Vectorized/batch-mode interpreter for symbolic expression trees

[bburlacu]

This ticket explores the possibility of employing batching and vectorisation techniques (ie. using dedicated datatypes from System.Numerics) to speed up the interpretation of symbolic expression trees.

Batching consists in allocating a small buffer for each instruction and performing operations on the whole buffer (instead of individual values for each row in the dataset).

Vectorisation additionally involves using SIMD (Single Instruction Multiple Data) CPU instructions to speed up batch processing.

Managed (C#) interpreter

Batch processing using the Vector<double> class in System.Numerics allows us to achieve a 2-3x speed improvement compared to the standard linear interpreter.

Native interpreter

A tree interpreter in native code (C++) can offer a significant speed advantage due to more mature backends (msvc, gcc) and features like auto-vectorization and loop unrolling.

Preliminary results show 5-10x speed improvement compared to the linear tree interpreter. We should also investigate the potential benefit of integrating fast math libraries such as ​vdt (vectorized math]) to increase computation speed.

This functionality should be implemented as an external plugin.

[bburlacu]

Preliminary results

[bburlacu]

r16266: Add native interpreter dll wrapper as external lib.

r16269: Add C++ source code

r16274: Update dll files and C++ source code to the latest version.

r16276: Add SymbolicDataAnalysisExpressionTreeNativeInterpreter which calls into the native implementation.

r16277: SymbolicDataAnalysisExpressionTreeNativeInterpreter: add EvaluatedSolutions as parameter, similar to the other interpreters.

[bburlacu]

r16285: Add vectorized SymbolicDataAnalysisExpressionTreeBatchInterpreter and update project config (Nuget package System.Numerics).

r16286: Forgot to commit changes to project file

r16287: Keep the SymbolicDataAnalysisExpressionTreeBatchInterpreter, but remove vectorization.

[bburlacu]

r16293: Support additional symbols in the SymbolicDataAnalysisExpressionTreeBatchInterpreter

[bburlacu]

r16296: SymbolicDataAnalysisExpressionTreeBatchInterpreter: simplify Compile, add cache for variable values (helps a lot with performance).

r16297: Very minor refactor.

[bburlacu]

r16298: Add batch interpreter performance unit tests (for arithmetic and typecoherent grammar).

[bburlacu]

r16333: Native interpreter dlls: statically link against the Visual C++ runtime

r16334: Add support for sqrt in the interpreter and update dlls .

[gkronber]

reviewed the code and made some changes in the branch for #2915. Will merge back later.

[gkronber]

In r16356 I merged back changes to the native interpreter from the #2915 branch.

I tested a lot and found that the native interpreter produces exactly the same results as the managed interpreters except when using sqrt() or abs(). Do you have any idea why this might happen?

• changeset:16356/trunk/HeuristicLab.ExtLibs/HeuristicLab.NativeInterpreter/0.1/NativeInterpreter-0.1/src/instruction.h
• changeset:16356/trunk/HeuristicLab.ExtLibs/HeuristicLab.NativeInterpreter/0.1/NativeInterpreter-0.1/src/interpreter.h
• changeset:16356/trunk/HeuristicLab.ExtLibs/HeuristicLab.NativeInterpreter/0.1/NativeInterpreter-0.1/src/vector_operations.h

[gkronber]

The BatchInterpreter assumes that GetSymbolicExpressionTreeValues() is always called with the same dataset. On the first call it caches the supplied dataset and on later calls it just takes the values from the cache.

The API allows to call interpreters with different datasets so the supplied dataset must be checked against the cache on each call.

I just fell into this trap and only recognized the problem because I was skeptical of the results. Looking only at my code I would have never found that the interpreter actually ignores the dataset that I set as a parameter.

Please fix!

Probably this is true for the native interpreter as well.

[bburlacu]

r16378: Batch and Native interpreter: keep a cached reference to the dataset so we can detect when it changes.

[bburlacu]

r16379: NativeInterpreter: avoid memory leak (free pinned array handles when the cache changes)

[abeham]

r16542: changed reference to PluginInfrastructure from file to project

The project build order wasn't right due to the missing dependency on the project. NativeInterpreter could be built before PluginInfrastucture since the reference wasn't recorded as a project reference.

[gkronber]

The new interpreters do not support factor variables and several function symbols (average, ...).

[gkronber]

Reviewed r16378, r16379, r16542 and tested the new interpreters.

[gkronber]

There should be an exception when the evaluator does not support a symbol.

[gkronber]

r16762: added checks and exceptions if native and batch interpreters encounter an unsupported symbol.

[gkronber]

r16768: fixed supported operations in BatchInterpreter (fix failing unit test)

[mkommend]

r17071: Merged 16266, 16269, 16274, 16276, 16277, 16285, 16286, 16287, 16289, 16293, 16296, 16297, 16298, 16333, 16334 into stable.
r17073: Merged 16378, 16379, 16542 into stable.

[mkommend]

r17104: Merged 16762, 16768 into stable.

[gkronber]

r16301 not merged?

[gkronber]

r17170: merged r16301 from trunk to stable