Opened 3 years ago

Closed 2 years ago

#2211 closed enhancement (done)

Provide unit tests for samples and scripts creation and execution

Reported by: ascheibe Owned by: jkarder
Priority: medium Milestone: HeuristicLab 3.3.11
Component: Tests Version: 3.3.9
Keywords: Cc:

Description (last modified by jkarder)

Previously, all samples unit tests were added to the class SamplesTest. For maintainability reasons and to locate the corresponding unit test faster, each sample should have its separate unit test class. Additionally, the scripts provided on the StartPage of the Optimizer should also have a unit test for automated detection of API changes.

TODO:

  • Split SamplesTest class into individual classes
  • Create UnitTests for Script creation and execution

Change History (19)

comment:1 Changed 3 years ago by ascheibe

  • Owner changed from swagner to gkronber
  • Status changed from new to assigned

comment:2 Changed 3 years ago by ascheibe

  • Owner changed from gkronber to bburlacu

Please assign this ticket to jkarder after you have implemented your tests.

comment:3 Changed 3 years ago by mkommend

  • Description modified (diff)
  • Summary changed from Include all sample tests in SamplesTest class to Provide unit tests for samples and scripts creation and execution
  • Type changed from defect to enhancement

comment:4 Changed 3 years ago by bburlacu

  • Status changed from assigned to accepted

comment:5 Changed 3 years ago by bburlacu

r11450: Separated samples class into separate test classes. Added scripts unit tests (grid search classification/regression).

comment:6 Changed 3 years ago by bburlacu

  • Owner changed from bburlacu to jkarder
  • Status changed from accepted to assigned

r11466: Updated grid search start page scripts and unit tests, which now reside in HeuristicLab.Scripting and belong to test category "Scripting".

comment:7 Changed 3 years ago by jkarder

  • Owner changed from jkarder to bburlacu

Be advised that the CSharpScript.Execute method creates and starts a thread in which the script is actually executed. If you check the variable store immediately after starting the script, you most likely will not be able to retrieve any results since the script execution might not have finished yet. For now you can register an event handler on ScriptExecutionFinished and use e.g. a ManualResetEvent to synchronize the script and unit test execution.

comment:8 Changed 3 years ago by abeham

I would opt that we call it ExecuteAsync and have a synchronized Execute method. A similar change was already identified for algorithms in HL as well.

comment:9 Changed 3 years ago by bburlacu

r11483: Updated script unit tests to wait for the script thread to finish before validating results.

comment:10 Changed 2 years ago by jkarder

  • Owner changed from bburlacu to jkarder

comment:11 Changed 2 years ago by jkarder

  • Status changed from assigned to accepted

comment:12 Changed 2 years ago by jkarder

  • Description modified (diff)

r11514:

  • updated/added unit tests
    • added AssemblyInitialize method to load all plugins, create output directories for (script) samples and initialize the MainForm
    • script code is now stored in test resource files
    • refactored unit tests
  • updated (script) samples
  • added Test.cmd

comment:13 Changed 2 years ago by jkarder

@bburlacu: I oriented myself towards the sample unit tests and created additional unit tests for the script sample generation. I will reassign you this ticket after a successful build so you can tune the SVM parameters of your samples.

comment:14 Changed 2 years ago by jkarder

r11515: changed my mind about testing ui code

comment:15 Changed 2 years ago by jkarder

  • Owner changed from jkarder to bburlacu
  • Status changed from accepted to assigned

comment:16 Changed 2 years ago by bburlacu

r11890: Adjusted parameters ranges in the random forest grid search tests and fixed unit tests to match the new results.

comment:17 Changed 2 years ago by bburlacu

  • Owner changed from bburlacu to mkommend
  • Status changed from assigned to reviewing

comment:18 Changed 2 years ago by mkommend

  • Owner changed from mkommend to jkarder
  • Status changed from reviewing to readytorelease

comment:19 Changed 2 years ago by jkarder

  • Resolution set to done
  • Status changed from readytorelease to closed

r11907: merged r11450, r11466, r11483, r11514, r11515 and r11890 into stable: merged r11450, r11466, r11483, r11514, r11515 and r11890 into stable

Note: See TracTickets for help on using tickets.