Opened 9 years ago
Closed 5 years ago
#2435 closed enhancement (done)
Update AlgLib to most recent version
Reported by: | gkronber | Owned by: | mkommend |
---|---|---|---|
Priority: | medium | Milestone: | HeuristicLab 3.3.16 |
Component: | ExtLibs | Version: | trunk |
Keywords: | Cc: |
Description
Change History (32)
comment:1 Changed 9 years ago by gkronber
- Owner set to gkronber
- Status changed from new to accepted
comment:2 Changed 9 years ago by gkronber
comment:3 Changed 9 years ago by gkronber
r12790: updated alglib to version 3.9.0
comment:4 Changed 9 years ago by gkronber
r12791: updated alglib reference in test project (reverse merge)
comment:5 Changed 9 years ago by gkronber
r12792: updated some of the alglib calls (we should only use the external API whenever possible) (reverse merge)
comment:6 Changed 9 years ago by gkronber
r12796: updated alglib references to version 3.9.0 in FLA branch (reverse merge)
comment:7 Changed 9 years ago by gkronber
r12798: quick fix for new result values in unit tests because of update to alglib 3.9 (reverse merge)
Version increment of HeuristicLab.Algorithms.DataAnalysis necessary? This would allow us to adjust the persistence code also.
comment:8 Changed 9 years ago by gkronber
r12801: changed result value for gaussian process regression unit test because of changes of alglib version (reverse merge)
comment:9 Changed 9 years ago by gkronber
It seems there will be a new release soon. http://bugs.alglib.net/roadmap_page.php
comment:10 Changed 9 years ago by gkronber
r12817: reverse merge of all trunk changes for updating to alglib version 3.9.0 (r12790:12792, r12798, r12801)
comment:11 Changed 9 years ago by gkronber
comment:12 Changed 9 years ago by gkronber
As discussed by architects it is necessary to increment the minor version of all dependent plugins if the alglib version increment would lead to a change in behavior.
comment:13 Changed 9 years ago by gkronber
Results of evaluation of commercial version of alglib (including native implementation using Intel MKL)
- The native implementation automatically detects wether it is running on x86 or x64 systems at runtime and simply works.
- Native implementation is basically a C# wrapper to a compiled binary of C++ alglib that is statically linked to Intel MKL where useful. Thus, even the random forest implementation (even though it does not use MKL or parallelization) should be faster since it falls back to the native version.
- In several places we use 'internal' classes of alglib instead of the standard API (e.g. alglib.dforest.dfbuild(...) instead of alglib.dfbuild(...)). This poses a problem because the native implementation only provides a C# wrapper for the user-facing API.
- Code has to be rewritten to use 'official' alglib API calls. Especially for persistence (alglib now provides functions for serialization/deserialization of composed types)
- Our LMBFGS uses internal API extensively. The code has to be completely rewritten when using the external API and possibly it is not possible to implement it in the same way as now.
- We use a patched version of free alglib where we marked the PRNG as ThreadStatic. Alglib now has a different way of supporting multi-core parallel processing and thread-safety for the PRNG. Our code has to be adapted accordingly.
- The native implementation might produce slightly different results than the managed implementation. Also, the native implementation might produce different results on different processors (especially x86 vs. x64 systems).
- Updating alglib would lead to different algorithm results and therefore we need at least a minor version increment for all plugins that actually use alglib direcly or indirectly.
- Updating the version would also be useful because we could completely update the persistence format for alglib types.
comment:14 Changed 9 years ago by gkronber
- Summary changed from Update AlgLib to 3.9.0 to Update AlgLib to 3.10.0
comment:15 Changed 9 years ago by gkronber
- Milestone changed from HeuristicLab 3.3.13 to HeuristicLab 4.0.x Backlog
comment:16 Changed 8 years ago by gkronber
- r14999: merged r12790,r12791,r12792,r12796,r12798,r12801, and r12817 (reverse merge of all previous) from trunk to stable. (only merge info is affected)
- r15000: merged r12818 from trunk to stable (which was a reverse merge of r12796, only merge info is affected)
Therefore, all changes in the ticket so far have been merged to stable.
comment:17 Changed 7 years ago by gkronber
- Summary changed from Update AlgLib to 3.10.0 to Update AlgLib to most recent version
comment:18 Changed 7 years ago by gkronber
- Priority changed from medium to low
comment:19 Changed 6 years ago by gkronber
r16683: Created a new branch to add alglib version 3.15 as external library.
comment:20 Changed 6 years ago by gkronber
r16684: added 3.15.0 version of alglib as external library (build from source)
comment:21 Changed 6 years ago by gkronber
The branch is ready and can be merged to trunk. This would enable us to use the newer version for new developments in HL.
comment:22 Changed 6 years ago by gkronber
- Owner changed from gkronber to mkommend
- Status changed from accepted to reviewing
comment:23 Changed 6 years ago by gkronber
- Milestone changed from HeuristicLab 4.0 to HeuristicLab 3.3.16
- Version changed from 3.3.12 to branch
comment:24 Changed 5 years ago by mkommend
r17034: Updated branch with most recent trunk changes.
comment:25 Changed 5 years ago by mkommend
r17036: Changed rng in ALGLIB to have a separate instance per thread.
comment:26 Changed 5 years ago by mkommend
r17037: Merged branch into trunk.
comment:27 Changed 5 years ago by mkommend
- Priority changed from low to medium
- Status changed from reviewing to readytorelease
- Version changed from branch to trunk
r17038: Deleted branch.
comment:28 Changed 5 years ago by abeham
We currently have alglib 3.7 and 3.15 side-by-side in HL. This creates problems for scripting and dependent plugins (programmable problem, external evaluation problem) which always load all assemblies in the appdomain.
The script cannot be compiled: "The type 'alglib' exists in both '...\ALGLIB-3.15.0.dll' and '...\ALGLIB-3.7.0.dll'".
I fixed that by checking for duplicate assemblies (by version-stripped filename) and using only the most recent version.
r17110: added filter to include only assemblies of most recent version in case multiple versions exist
comment:29 Changed 5 years ago by abeham
- Status changed from readytorelease to reviewing
comment:30 Changed 5 years ago by mkommend
- Status changed from reviewing to readytorelease
Reviewed r17110.
comment:31 Changed 5 years ago by gkronber
- Keywords merged added
comment:32 Changed 5 years ago by jkarder
- Keywords merged removed
- Resolution set to done
- Status changed from readytorelease to closed
Script to test alglib changes (including persistence)