Improve object graph traversing

[mkommend]

The traversing of an object graph was implemented to clean up all IStatefulItems when an algorithm is prepared or stopped (see #1424).

This traversing is rather slow especially when large arrays are enumerated (e.g., Datasets).

Additionally structs are not handled correctly because they are skipped due to a check for ValueType.

[mkommend]

r6181: Reorganized type checks and added a clause to exclude primitive arrays from being enumerated.

[mkommend]

The include structs in the traversing of the object graph, value types must be allowed for traversing. As this leads to an StackoverflowException I didn't change this behavior. It would be good if you could have a look at this issue.

[cneumuel]

Some further comments (open for discussion):

• The type should be checked for IsArray rather than for IEnumerable. By definition the implementation of IEnumerable could result an infinate number of elements.
• The check fot the ElementType should only be made if it is an array.
• String values should also be collected, as discussed with Stefan Wagner (this was changed in r6181)
• A possible performance optimization: The datatype of the objects could be given. In this case we could skip collecting strings or going through arrays with strings and primitive datatypes.

[swagner]

Replying to mkommend:

The include structs in the traversing of the object graph, value types must be allowed for traversing. As this leads to an StackoverflowException I didn't change this behavior. It would be good if you could have a look at this issue.

To avoid the StackOverflowException it is sufficient to exclude the type System.Reflection.Pointer. However, I am not quite sure about any possible side effects. We have to have a detailed look at this.

[swagner]

Replying to cneumuel:

• The type should be checked for IsArray rather than for IEnumerable. By definition the implementation of IEnumerable could result an infinate number of elements.

I experimented with checking for arrays and not for IEnumerables right before the 3.3.4 release. However, this change reduced the performance drastically and resulted in fewer collected objects (quite strange). We have to analyze this in more detail.

[mkommend]

In a discussion with swagner it was decided to exclude primitive types (int,short,string,char,...) from the returned objects, as they do not include any semantics and are immutable.

[cneumuel]

• The reason IsArray returns less objects than checking for IEnumerable is that it's quite different if an Enumerator of an IEnumerable is used rather than going deeper down into the object until an array is reached. An example is the Dictionary. dict is IEnumerable returns true, resulting in enumerating that dictionary. The implementation of the enumerator reveals that KeyValuePair<TKey, TValue> objects are created within the enumerator. If IsArray is used, dict.IsArray returns false, resulting in a further recursion into the dictionary object. In there an array of Entry<TKey, TValue>[] is found, which is then used. So in this example all the KeyValuePair objects are missing when IsArray is used (which is good!).
• I have not noticed a big performance difference between IsArray and is IEnumerable, so in my opinion IsArray would be the way to go. I could reproduce the performance difference, but I could mostly fix it by handling dictionaries in a special way.

[cneumuel]

r6192

• included ValueTypes (which includes structs and enums)
• avoided stack overflow by excluding Type
• improved performance by handling dictionaries in a special way
• avoided stack overflow by excluding ThreadLocal<> from (occured after many repetitions, when the field the field ConcurrentStack<int> s_availableIndices grew larger
• using IsArray instead of is IEnumerable

mkommend: Please test if performance is acceptable with your datasets.

[mkommend]

r6201: Updated object graph traversing.

[mkommend]

Please check if further simplifications and unifications can be made without affecting the results and the performance (e.g., check for delegates).

[cneumuel]

r6205

• simplified check for EventHandler (by simply checking for delegate)
• special handling types which use a hashtable internally (dictionaries, hashset, hashtable, ...) (as discussed with swagner)
• added testcase which shows a stack overflow if ThreadLocal is included and compiler is on Debug (it does not happen in Release)

[cneumuel]

r6207, r6208: fixed missing files. r6209: added comment for test method

[mkommend]

r6380: Replaced recursive object graph traversing with a linear solution using a stack to avoid stack overflows.