Opened 13 years ago
Closed 13 years ago
#1659 closed feature request (done)
Add algorithm to execute performance benchmarks
Reported by: | ascheibe | Owned by: | swagner |
---|---|---|---|
Priority: | medium | Milestone: | HeuristicLab 3.3.6 |
Component: | Algorithms.Benchmarks | Version: | 3.3.6 |
Keywords: | Cc: |
Description (last modified by swagner)
Add an algorithm to execute performance benchmarks (Linpack, Dhrystone, Whetstone, etc.).
Change History (24)
comment:1 Changed 13 years ago by ascheibe
- Owner changed from ascheibe to spimming
- Status changed from new to assigned
comment:2 Changed 13 years ago by ascheibe
comment:3 Changed 13 years ago by ascheibe
r6920 added benchmarking algorithms
comment:4 Changed 13 years ago by ascheibe
mkommend suggested that we pull the common parts out of FixedDataAnalysisAlgorithm and put them in a base class in HeuristicLab.Optimization (OptimizerBase and ExecutableBase) because there is now a lot of duplicate code in our benchmarking algorithms.
comment:5 Changed 13 years ago by ascheibe
on behalf of spimming: r6934
- implemented abstract base class for benchmarking algorithms
- added License information
- corrected plugin dependencies
- corrected descriptions
comment:6 Changed 13 years ago by spimming
- restructuring of the benchmarking algorithms
- common interface for benchmarks
- 'benchmark' class discovers benchmarking algorithms
- added TimeLimit and ChunkSize parameters to benchmark
comment:7 Changed 13 years ago by spimming
- Owner changed from spimming to ascheibe
- Status changed from assigned to reviewing
comment:8 Changed 13 years ago by ascheibe
General:
- Getters/Setters should be placed in the code directly after the corresponding private data members.
Benchmark.cs:
- In DiscoverBenchmarks i wouldn't throw an exception if no benchmarks are found. If there are no benchmarks then no benchmarks can be selected and executed, i think that's no problem.
- There is some uncommented code that should be removed.
GUI:
- Mention in the description of the TimeLimit parameter that 0 means a fixed number of iterations.
- In the parameters tab where you can choose the Benchmarking algorithm there is below the drop down menu the text "No view available". Can we get rid of this?
- Input validation would be nice so that no negative values can be entered.
- In the Run Collection View if a run is selected, both "Benchmark Name" and "Benchmark Type" display "Benchmark" as value. Can we hide them? This point isn't really important though.
comment:9 Changed 13 years ago by spimming
- removed uncommented code
- removed ‘Benchmark Name’ and ‘Benchmark Type’ from result collection
- placed Getters/Setters after data members
- added input validation
- updated parameter description
comment:10 Changed 13 years ago by ascheibe
r7000 updated branch from trunk
comment:11 Changed 13 years ago by ascheibe
- fixed errors found by unit tests
- set references to copylocal=false
- added missing platforms to project files
comment:12 Changed 13 years ago by ascheibe
- renamed benchmarking algs to Benchmark where it was missing
- merged from trunk
comment:13 Changed 13 years ago by ascheibe
r7006 merged benchmarking algorithms branch back to trunk
comment:14 Changed 13 years ago by ascheibe
r7007 fixed small typo
comment:15 Changed 13 years ago by ascheibe
- Owner changed from ascheibe to swagner
r7008 added an empty view for IBenchmark so that "No view available" won't be displayed in HL
comment:16 Changed 13 years ago by ascheibe
swagner: Just in case you have some spare time you can have a look at it ;-)
comment:17 Changed 13 years ago by ascheibe
r7015 removed .resx files and moved Benchmark to Algorithms category
comment:18 Changed 13 years ago by ascheibe
r7016 deleted benchmarking branch
comment:19 Changed 13 years ago by abeham
- Version changed from branch to 3.3.5
comment:20 Changed 13 years ago by swagner
- Component changed from Hive.General to Algorithms.Benchmarks
- Description modified (diff)
- Summary changed from Add benchmarking algorithms to Add algorithm to execute performance benchmarks
- Type changed from enhancement to feature request
comment:21 Changed 13 years ago by swagner
Thanks for implementing this feature. I reviewed the code and made some additional changes:
- renamed Benchmark to BenchmarkAlgorithm and BenchmarkView to BenchmarkAlgorithmView
- removed unnecessary assembly references
- removed IBenchmarkView as the user should be informed that there is no specific view for benchmarks
- adapted descriptions of plugins and assemblies
- created new and unique assembly Guids
- sealed BenchmarkAlgorithm
- adapted item names and item descriptions
- adapted code formatting
- added base class Benchmark for benchmark implementations
- implemented ToString in Benchmark base class
- removed suffix Benchmark from class names of benchmark implementations
- sealed benchmark implementations Linpack, Dhrystone and Whetstone
- removed unnecessary problem-specific parts in BenchmarkAlgorithm
- enabled name and description changes in BenchmarkAlgorithm
- adapted parameter descriptions
- implemented ToString in BenchmarkAlgorithm
- renamed CreateDataChuck to CreateChunkData
- adapted code formatting
comment:22 Changed 13 years ago by swagner
- Owner changed from swagner to spimming
Please review my changes together with ascheibe and also test in the GUI once more. If you do not find any other issues, please set this ticket to ready to release. Thanks.
comment:23 Changed 13 years ago by spimming
- Owner changed from spimming to swagner
- Status changed from reviewing to readytorelease
comment:24 Changed 13 years ago by swagner
- Resolution set to done
- Status changed from readytorelease to closed
- Version changed from 3.3.5 to 3.3.6
r6917 create branch for benchmarking algorithms