Implement Hive Statistics

[pfleck]

r9522

Created Branch for Hive-Statistics.

[pfleck]

r9524

Added statistic tables in Hive database.

Updated InitializeHiveDatabase.sql script with new create statements.

Added Linq2Sql entity types for new tables.

[pfleck]

r9526

Added StatisticsGenerator for generating statistic data for Hive.

ServiceLocator returns the StatisticsGenerator.

Integrated StatisticsGenerator in Janitor for periodic calls.

[pfleck]

r9527

Provides a GUI for SelfHosting Hive-Service to start and stop the service.

Enables start and stop Janitor with adjustable frequency.

Run Cleanup and GenerateStatistics separately on click.

[pfleck]

r9533

StatisticsGenerator now updates Time, Job and User dimension-tables.

[pfleck]

r9538

StatisticsGenerator fills and updates client dimension table.

Removed week from time dimension table.

[pfleck]

r9541

UserId in ClientInfo fact table is nullable. UserId is removed from primary key therefore.

StatisticsGenerator insert new ClientInfo into fact table.

[gkronber]

Replying to pfleck:

r9541

UserId in ClientInfo fact table is nullable. UserId is removed from primary key therefore.

Not a good idea. Instead, use a surrogate key for the user id and keep an entry for the 'NULL' user in the DimUser table. This entry has non-null surrogate key value and NULL UserId value.

[pfleck]

r9545

ClientInfo facts are now generated correctly when resources are changed and new client dimensions are created.

Query username for statistics tables from user id.

[pfleck]

r9546

As suggested by gkronber, added user to primary key again in ClientFact table and added a 'NULL'-user in the DimUser table instead.

[pfleck]

r9547

Changed asp.net membership provider for querying usernames.

[pfleck]

r9557

Added DataSize column in TaskData (calculated column).

Created queries for fetching required data for generating task facts.

[pfleck]

r9560

Query last client for task facts and calculate task fact data. (TotalRuntime/Wait/TransferRime pending)

[pfleck]

r9561

Simplified and optimized query for task fact statistics.

[pfleck]

r9571

Calculating total waiting-, transfer- and calculating time for task statistics.

[pfleck]

r9578

Task statistics contains only current state and no change history.

Statistics calculation is no longer based on increment updates.

TaskId is primary key in task statistics and EndTime is nullable.

Added TaskState to Task statistics.

[pfleck]

r9589

Added Asp.net MVC project and all required libraries.

Added publish profile to web services and web pages.

[pfleck]

r9604

Set up basic webpage layout based on WebApplication branch.

Added Asp.Net MVC packages and some helper packages.

Implemented login.

[pfleck]

r9617

Started integrating Hive statistics into statistics web project.

Added jqPlot library for charting.

[pfleck]

r9625

Insert charts on index page for CPU utilization, cores and memory usage.

Added html helper for generating chart javascript.

Added ChartDataController for generating JSON data for the charts.

[pfleck]

r9643

Added time selection for charts.

Implemented redrawing of charts using Ajax.

[pfleck]

r9644

Fixed timezone bug in chart data.

Fixed zooming bug in jqplot dateAxisRenderer. (​https://bitbucket.org/cleonello/jqplot/pull-request/29/removed-unbalanced-addition-of-utc-offset/diff)

Reformatting

[pfleck]

r9646

Changed year in headers from 2012 to 2013.

Minor reformattings in chart helpers.

Refactored and simplified some statistics generator methods.

[pfleck]

r9661

Updated wrong endpoint address.

Re-enabled performance counters for slaves.

[pfleck]

r9662

Rearranged axis options in jqplot charts.

Renamed selectors into seriesSelectors in CreateSeriesData.

Changed default date range on index page.

[ascheibe]

The work from this ticket has already been released with #2388