Free cookie consent management tool by TermsFeed Policy Generator

Opened 13 years ago

Closed 13 years ago

#1659 closed feature request (done)

Add algorithm to execute performance benchmarks

Reported by: ascheibe Owned by: swagner
Priority: medium Milestone: HeuristicLab 3.3.6
Component: Algorithms.Benchmarks Version: 3.3.6
Keywords: Cc:

Description (last modified by swagner)

Add an algorithm to execute performance benchmarks (Linpack, Dhrystone, Whetstone, etc.).

Change History (24)

comment:1 Changed 13 years ago by ascheibe

  • Owner changed from ascheibe to spimming
  • Status changed from new to assigned

comment:2 Changed 13 years ago by ascheibe

r6917 create branch for benchmarking algorithms

comment:3 Changed 13 years ago by ascheibe

r6920 added benchmarking algorithms

comment:4 Changed 13 years ago by ascheibe

mkommend suggested that we pull the common parts out of FixedDataAnalysisAlgorithm and put them in a base class in HeuristicLab.Optimization (OptimizerBase and ExecutableBase) because there is now a lot of duplicate code in our benchmarking algorithms.

comment:5 Changed 13 years ago by ascheibe

on behalf of spimming: r6934

  • implemented abstract base class for benchmarking algorithms
  • added License information
  • corrected plugin dependencies
  • corrected descriptions

comment:6 Changed 13 years ago by spimming

r6948:

  • restructuring of the benchmarking algorithms
  • common interface for benchmarks
  • 'benchmark' class discovers benchmarking algorithms
  • added TimeLimit and ChunkSize parameters to benchmark

comment:7 Changed 13 years ago by spimming

  • Owner changed from spimming to ascheibe
  • Status changed from assigned to reviewing

comment:8 Changed 13 years ago by ascheibe

General:

  • Getters/Setters should be placed in the code directly after the corresponding private data members.

Benchmark.cs:

  • In DiscoverBenchmarks i wouldn't throw an exception if no benchmarks are found. If there are no benchmarks then no benchmarks can be selected and executed, i think that's no problem.
  • There is some uncommented code that should be removed.

GUI:

  • Mention in the description of the TimeLimit parameter that 0 means a fixed number of iterations.
  • In the parameters tab where you can choose the Benchmarking algorithm there is below the drop down menu the text "No view available". Can we get rid of this?
  • Input validation would be nice so that no negative values can be entered.
  • In the Run Collection View if a run is selected, both "Benchmark Name" and "Benchmark Type" display "Benchmark" as value. Can we hide them? This point isn't really important though.

comment:9 Changed 13 years ago by spimming

r6987:

  • removed uncommented code
  • removed ‘Benchmark Name’ and ‘Benchmark Type’ from result collection
  • placed Getters/Setters after data members
  • added input validation
  • updated parameter description

comment:10 Changed 13 years ago by ascheibe

r7000 updated branch from trunk

comment:11 Changed 13 years ago by ascheibe

r7002

  • fixed errors found by unit tests
  • set references to copylocal=false
  • added missing platforms to project files

comment:12 Changed 13 years ago by ascheibe

r7004

  • renamed benchmarking algs to Benchmark where it was missing
  • merged from trunk

comment:13 Changed 13 years ago by ascheibe

r7006 merged benchmarking algorithms branch back to trunk

comment:14 Changed 13 years ago by ascheibe

r7007 fixed small typo

comment:15 Changed 13 years ago by ascheibe

  • Owner changed from ascheibe to swagner

r7008 added an empty view for IBenchmark so that "No view available" won't be displayed in HL

comment:16 Changed 13 years ago by ascheibe

swagner: Just in case you have some spare time you can have a look at it ;-)

comment:17 Changed 13 years ago by ascheibe

r7015 removed .resx files and moved Benchmark to Algorithms category

comment:18 Changed 13 years ago by ascheibe

r7016 deleted benchmarking branch

comment:19 Changed 13 years ago by abeham

  • Version changed from branch to 3.3.5

comment:20 Changed 13 years ago by swagner

  • Component changed from Hive.General to Algorithms.Benchmarks
  • Description modified (diff)
  • Summary changed from Add benchmarking algorithms to Add algorithm to execute performance benchmarks
  • Type changed from enhancement to feature request

comment:21 Changed 13 years ago by swagner

Thanks for implementing this feature. I reviewed the code and made some additional changes:

r7246:

  • renamed Benchmark to BenchmarkAlgorithm and BenchmarkView to BenchmarkAlgorithmView
  • removed unnecessary assembly references
  • removed IBenchmarkView as the user should be informed that there is no specific view for benchmarks
  • adapted descriptions of plugins and assemblies
  • created new and unique assembly Guids
  • sealed BenchmarkAlgorithm
  • adapted item names and item descriptions
  • adapted code formatting

r7247:

  • added base class Benchmark for benchmark implementations
  • implemented ToString in Benchmark base class
  • removed suffix Benchmark from class names of benchmark implementations
  • sealed benchmark implementations Linpack, Dhrystone and Whetstone

r7248:

  • removed unnecessary problem-specific parts in BenchmarkAlgorithm
  • enabled name and description changes in BenchmarkAlgorithm
  • adapted parameter descriptions
  • implemented ToString in BenchmarkAlgorithm
  • renamed CreateDataChuck to CreateChunkData
  • adapted code formatting

comment:22 Changed 13 years ago by swagner

  • Owner changed from swagner to spimming

Please review my changes together with ascheibe and also test in the GUI once more. If you do not find any other issues, please set this ticket to ready to release. Thanks.

comment:23 Changed 13 years ago by spimming

  • Owner changed from spimming to swagner
  • Status changed from reviewing to readytorelease

comment:24 Changed 13 years ago by swagner

  • Resolution set to done
  • Status changed from readytorelease to closed
  • Version changed from 3.3.5 to 3.3.6
Note: See TracTickets for help on using tickets.