Remove "Logic" Classes in DataPreprocessing

[pfleck]

The "Logics" in the DataPreprocessing encapsulates different aspects of how the DataPreprocessingData can be accessed and manipulated. However, the "Logic"-classes mostly contain some helper methods that are only used within a single corresponding view. The logic classes should be removed and the code moved to the corresponding views.

By removing the "Logics", the initialization of the DataPreprocessingView should be simplified to avoid the highly coupled and entangled object initialization. Generally, the overall design of the data preprocessing should be more like the rest of HL.

[pfleck]

r15268 Added branch with the DataPreprocessing plugins.

[pfleck]

r15269: Removed SearchLogic

[pfleck]

r15270: Removed TransactionalPreprocessingData and moved relevant code to PreprocessingData.

[pfleck]

r15274:

• Removed FilterLogic.
• Made Contents storable and implemented proper cloning.

[pfleck]

r15283: Removed StatisticsLogic.

[pfleck]

r15285 removed ManipulationLogic

• Moved "ReplaceByAverage/Random/..." to DataGridContent only (functions are redundant in manipulation view)
  • Refactored some code for replacing values with ...
• Removed Smoothing (not very usefull and changes the data incomprehensibly)
• Moved Shuffling to DataGridContent
• Moved "Delete Rows/Columns with insufficient ..." to ManipulationContent
• Removed Logic Folder
  • Moved Filter folder out

[pfleck]

r15291: Added (Double/String/DateTime)PreprocessingDataColumn. (experimental state)

[pfleck]

r15309 Worked on type-save PreprocessingDataColumns.

[pfleck]

r15431: Removed experimental static-typed datacolumns. (reverse merge g15291, r15309)

[bburlacu]

The functionality is fine and consistent with the old behavior (tested side by side with the SARCOS problem data).

Some nitpicks:

• FilteredPreprocessingData.cs:
  • Line 45: typo "preporcessingData"

• Filtering logic seems unnecessarily complicated

• FilterView.cs:
  • Line 86: this piece of code is a little misleading and the whole logic in my opinion counter-intuitive

result1[row] = isAndCombination ? result1[row] || filterResult[row] : result1[row] && filterResult[row];

• the actual result is not the rows for which the filter combination is valid, but the rows for which the filter combination is NOT valid (and these rows get deleted in the SetFilter() method)
• i think the intent should be more clear and the result explicitly negated in order to get the rows which should be deleted
• names like "result1" should be more clear (eg., rowsToDelete, filteredRows)

[pfleck]

r15466: Simplified the overall filtering logic as suggested by bburlacu

• changed parameter names to actively reflect that filter means "remaining"
• moved filter combination logic to FilterContent
• simplified/restructured code

[bburlacu]

• unchecking a filter results in the wrong number of remaining and total rows displayed in the filter preview
• filtering is quite slow (takes ~7 seconds on my machine to remove 15k rows after filtering) - this problem also exists in the trunk version of the DataPreprocessing)

EDIT: The slowness is caused by the calls to IList.RemoveAt() which is called in succession for each row that needs to be removed inside the PreprocessingData.DeleteRow() method. Simply replacing the original values with the filtered values (instead of by row removal) results in a massive speed improvement.

[pfleck]

r15489

• Fixed wrong number of remaining/toral rows after filter changed
• Improved performance for filtering as suggested by bburlacu. Thanks for the patch.
• Start all charts with index zero.

[pfleck]

r15518: merged branch to trunk

[pfleck]

r15534: Fixed a potential InvalidOperationException when applying filters

[pfleck]

r15535: merged r15518,r15534 to stable

[pfleck]

r15536 deleted branch