Opened 7 years ago

Closed 4 years ago

#1423 closed feature request (done)

Gradient descent algorithms for HL (already implemented in alglib)

Reported by: gkronber Owned by: gkronber
Priority: medium Milestone: HeuristicLab 3.3.8
Component: Algorithms.GradientDescent Version: 3.3.8
Keywords: Cc:

Description


Change History (26)

comment:1 Changed 7 years ago by gkronber

  • Component changed from ### Undefined ### to Algorithms.GradientDescent
  • Owner changed from swagner to gkronber
  • Version changed from 3.3.3 to branch

comment:2 Changed 7 years ago by gkronber

r5515: created branch for implementation of gradient descent algorithms.

comment:3 Changed 7 years ago by gkronber

r5518: added main project for automatic deployment

comment:4 Changed 7 years ago by gkronber

r5516: added project files for Levenberg-Maquardt optimization algorithm.

comment:5 Changed 7 years ago by gkronber

r5520: fixed problem with progress reporting and stored quality values in a datatable.

comment:6 Changed 5 years ago by gkronber

  • Milestone changed from HeuristicLab 3.3.x Backlog to HeuristicLab 3.3.8
  • Status changed from new to accepted
  • Version changed from branch to 3.3.7

comment:7 Changed 5 years ago by gkronber

development of LM-BFGS has partially been tracked in ticket #1902

comment:8 Changed 5 years ago by gkronber

r8398: merged gradient descent project from branch into trunk

comment:9 Changed 5 years ago by gkronber

r8401: moved LM-BFGS implementation from data-analysis into the gradient descent algorithm plugin.

comment:10 Changed 5 years ago by gkronber

r8402: deleted feature branch for gradient descent algorithm

comment:11 Changed 5 years ago by gkronber

  • Owner changed from gkronber to abeham
  • Status changed from accepted to reviewing

comment:12 Changed 5 years ago by abeham

  • Owner changed from abeham to gkronber
  • Status changed from reviewing to assigned

As discussed

  • Use RealVectorEncoding instead of test functions (-> SolutionCreator needs to be IRealVectorCreator)

comment:13 Changed 5 years ago by gkronber

  • Owner changed from gkronber to abeham
  • Status changed from assigned to reviewing

r9127:

  • changed LBFGS to take any single-objective optimization algorithm and throw an exception if the solution creator does not have the correct type in OnStarted().
  • added wiring for parameter names (please review).

comment:14 Changed 5 years ago by gkronber

r9128: removed reference to test functions

comment:15 Changed 5 years ago by abeham

r9177: Renamed project to add version number (3.3) and fixed build number

comment:16 Changed 5 years ago by abeham

Please consider following points:

  • LbfgsAlgorithm
    • The solution creator should be casted to an IRealVectorCreator instead of the concrete implementation RealVectorCreator
    • Assigning name and description from Item(Name|Description) in the default ctor usually needs to be done only if directly derived from NamedItem.
    • The problem specific analyzers are not run on the solution. In the case of test functions it could be interesting to e.g. view the trajectory of the solution in 2D problems.
    • You don't need to append "(placeholder)" to the operator's name. The type will be shown in the graph view if the name deviates, just put the name in braces "(Evaluator)" to have some distinction.
    • You could add another hidden parameter "GradientCheckStepSize", if it's > 0 you could enable gradient check in the initializer.
  • LbfgsInitializer
    • check if "GradientCheckStepSize" exists and is > 0 to enable gradient check
  • LbfgsMakeStep
    • throw appropriate exceptions on various terminationtype values reported by alglib
      • -7 gradient verification failed
      • -1 incorrect parameters

I think we should at least run the problem specific analyzers.

comment:17 Changed 5 years ago by abeham

  • Owner changed from abeham to gkronber
  • Status changed from reviewing to assigned

comment:18 Changed 4 years ago by ascheibe

  • Owner changed from gkronber to abeham

comment:19 Changed 4 years ago by gkronber

  • Owner changed from abeham to gkronber

comment:20 Changed 4 years ago by gkronber

r9408:9409:

  • LbfgsAlgorithm
    • The solution creator should be casted to an IRealVectorCreator instead of the concrete implementation RealVectorCreator
    • Assigning name and description from Item(Name|Description) in the default ctor usually needs to be done only if directly derived from NamedItem.
    • The problem specific analyzers are not run on the solution. In the case of test functions it could be interesting to e.g. view the trajectory of the solution in 2D problems.
    • You don't need to append "(placeholder)" to the operator's name. The type will be shown in the graph view if the name deviates, just put the name in braces "(Evaluator)" to have some distinction.
    • You could add another hidden parameter "GradientCheckStepSize", if it's > 0 you could enable gradient check in the initializer.
  • LbfgsInitializer
    • check if "GradientCheckStepSize" exists and is > 0 to enable gradient check

r9405:9406:

  • LbfgsMakeStep
    • throw appropriate exceptions on various terminationtype values reported by alglib
      • -7 gradient verification failed
      • -1 incorrect parameters
Last edited 4 years ago by gkronber (previous) (diff)

comment:21 Changed 4 years ago by gkronber

  • Owner changed from gkronber to abeham

Andreas, it would be great if you could add the code for inclusion of problem-specific analyzers.

Last edited 4 years ago by gkronber (previous) (diff)

comment:22 Changed 4 years ago by abeham

I'll do it tomorrow

comment:23 Changed 4 years ago by abeham

  • Owner changed from abeham to gkronber
  • Status changed from assigned to reviewing

r9439: Changed the way analyzers work for LM-BFGS

comment:24 Changed 4 years ago by gkronber

Reviewed r9439.

comment:25 Changed 4 years ago by gkronber

  • Status changed from reviewing to readytorelease

comment:26 Changed 4 years ago by swagner

  • Resolution set to done
  • Status changed from readytorelease to closed
  • Version changed from 3.3.7 to 3.3.8
Note: See TracTickets for help on using tickets.