Opened 13 years ago
Closed 11 years ago
#1423 closed feature request (done)
Gradient descent algorithms for HL (already implemented in alglib)
Reported by: | gkronber | Owned by: | gkronber |
---|---|---|---|
Priority: | medium | Milestone: | HeuristicLab 3.3.8 |
Component: | Algorithms.GradientDescent | Version: | 3.3.8 |
Keywords: | Cc: |
Description
Change History (26)
comment:1 Changed 13 years ago by gkronber
- Component changed from ### Undefined ### to Algorithms.GradientDescent
- Owner changed from swagner to gkronber
- Version changed from 3.3.3 to branch
comment:2 Changed 13 years ago by gkronber
comment:3 Changed 13 years ago by gkronber
r5518: added main project for automatic deployment
comment:4 Changed 13 years ago by gkronber
r5516: added project files for Levenberg-Maquardt optimization algorithm.
comment:5 Changed 13 years ago by gkronber
r5520: fixed problem with progress reporting and stored quality values in a datatable.
comment:6 Changed 12 years ago by gkronber
- Milestone changed from HeuristicLab 3.3.x Backlog to HeuristicLab 3.3.8
- Status changed from new to accepted
- Version changed from branch to 3.3.7
comment:7 Changed 12 years ago by gkronber
development of LM-BFGS has partially been tracked in ticket #1902
comment:8 Changed 12 years ago by gkronber
r8398: merged gradient descent project from branch into trunk
comment:9 Changed 12 years ago by gkronber
r8401: moved LM-BFGS implementation from data-analysis into the gradient descent algorithm plugin.
comment:10 Changed 12 years ago by gkronber
r8402: deleted feature branch for gradient descent algorithm
comment:11 Changed 12 years ago by gkronber
- Owner changed from gkronber to abeham
- Status changed from accepted to reviewing
comment:12 Changed 11 years ago by abeham
- Owner changed from abeham to gkronber
- Status changed from reviewing to assigned
As discussed
- Use RealVectorEncoding instead of test functions (-> SolutionCreator needs to be IRealVectorCreator)
comment:13 Changed 11 years ago by gkronber
- Owner changed from gkronber to abeham
- Status changed from assigned to reviewing
- changed LBFGS to take any single-objective optimization algorithm and throw an exception if the solution creator does not have the correct type in OnStarted().
- added wiring for parameter names (please review).
comment:14 Changed 11 years ago by gkronber
r9128: removed reference to test functions
comment:15 Changed 11 years ago by abeham
r9177: Renamed project to add version number (3.3) and fixed build number
comment:16 Changed 11 years ago by abeham
Please consider following points:
- LbfgsAlgorithm
- The solution creator should be casted to an IRealVectorCreator instead of the concrete implementation RealVectorCreator
- Assigning name and description from Item(Name|Description) in the default ctor usually needs to be done only if directly derived from NamedItem.
- The problem specific analyzers are not run on the solution. In the case of test functions it could be interesting to e.g. view the trajectory of the solution in 2D problems.
- You don't need to append "(placeholder)" to the operator's name. The type will be shown in the graph view if the name deviates, just put the name in braces "(Evaluator)" to have some distinction.
- You could add another hidden parameter "GradientCheckStepSize", if it's > 0 you could enable gradient check in the initializer.
- LbfgsInitializer
- check if "GradientCheckStepSize" exists and is > 0 to enable gradient check
- LbfgsMakeStep
- throw appropriate exceptions on various terminationtype values reported by alglib
- -7 gradient verification failed
- -1 incorrect parameters
- throw appropriate exceptions on various terminationtype values reported by alglib
I think we should at least run the problem specific analyzers.
comment:17 Changed 11 years ago by abeham
- Owner changed from abeham to gkronber
- Status changed from reviewing to assigned
comment:18 Changed 11 years ago by ascheibe
- Owner changed from gkronber to abeham
comment:19 Changed 11 years ago by gkronber
- Owner changed from abeham to gkronber
comment:20 Changed 11 years ago by gkronber
LbfgsAlgorithmThe solution creator should be casted to an IRealVectorCreator instead of the concrete implementation RealVectorCreatorAssigning name and description from Item(Name|Description) in the default ctor usually needs to be done only if directly derived from NamedItem.- The problem specific analyzers are not run on the solution. In the case of test functions it could be interesting to e.g. view the trajectory of the solution in 2D problems.
You don't need to append "(placeholder)" to the operator's name. The type will be shown in the graph view if the name deviates, just put the name in braces "(Evaluator)" to have some distinction.You could add another hidden parameter "GradientCheckStepSize", if it's > 0 you could enable gradient check in the initializer.
LbfgsInitializercheck if "GradientCheckStepSize" exists and is > 0 to enable gradient check
LbfgsMakeStepthrow appropriate exceptions on various terminationtype values reported by alglib-7 gradient verification failed-1 incorrect parameters
comment:21 Changed 11 years ago by gkronber
- Owner changed from gkronber to abeham
Andreas, it would be great if you could add the code for inclusion of problem-specific analyzers.
comment:22 Changed 11 years ago by abeham
I'll do it tomorrow
comment:23 Changed 11 years ago by abeham
- Owner changed from abeham to gkronber
- Status changed from assigned to reviewing
r9439: Changed the way analyzers work for LM-BFGS
comment:24 Changed 11 years ago by gkronber
Reviewed r9439.
comment:25 Changed 11 years ago by gkronber
- Status changed from reviewing to readytorelease
comment:26 Changed 11 years ago by swagner
- Resolution set to done
- Status changed from readytorelease to closed
- Version changed from 3.3.7 to 3.3.8
r5515: created branch for implementation of gradient descent algorithms.