Hive trunk integration

[ascheibe]

This ticket is for tracking the trunk integration of HeuristicLab Hive (#1233).

[ascheibe]

r6976 integrate the Hive client projects into trunk (Hive Job Manager and Administrator)

[ascheibe]

r6977

• removed unused files
• added missing license headers

[ascheibe]

r6979 fixed plugin dependencies

[ascheibe]

r6983

• added the Hive Services and Slave projects
• added missing svn ignores

[ascheibe]

r6985 added documentation for Hive

[ascheibe]

r6993

• removed unused files
• changed the plugin cache path of the Slave HL App so that HL doesn't discover Hive assemblies
• cleaned up config files
• incremented version number of installers to 3.3.6
• removed Execution time on Hive from Status page because it can't be calculated without the user statistics

[ascheibe]

r6994

• removed dead code
• added Hive assembly references to Tests project
• fixed problems found by tests

[ascheibe]

r6995 fixed typo which led to false assembly paths in the Slave App

[ascheibe]

r6997 added missing invoke

[ascheibe]

r6998 Changed again how plugin discovery works because of Hive. The reason is that it must be possible to move the plugin and working directories away from the original slave working directory. This is needed for the Slave App and also in the future for the windows service because we don't want it to run as the LocalSystem user. I have removed setting the PrivateBinPath and am now setting the ApplicationBase. This doesn't effect HL (because ApplicationBase is set by default to !pluginDir anyway) but makes Hive work. The reason why setting the PrivateBinPath doesn't work with moving plugin and working directories is (from msdn): "Private assemblies are deployed in the same directory structure as the application. If the directories specified for PrivateBinPath are not under ApplicationBase, they are ignored."

[ascheibe]

r7009 moved the Hive services parts to services project as suggested by swagner

[ascheibe]

r7014 increased HB interval to 20 secs

[ascheibe]

r7020

• increased max. object graph size which can be serialized to allow downloading of big jobs
• removed more magic numbers
• increased job polling interval

[ascheibe]

r7029 fixed a small bug when refreshing permissions

[ascheibe]

r7032 reverted change that exceptions are thrown in the job manager

[ascheibe]

r7045 switch more service calls to IsolationLevel=ReadUncommitted transactions to prevent db deadlocks

[ascheibe]

r7046 added Hive and Benchmarking project dependencies to HeuristicLab-3.3 project

[ascheibe]

r7047

• use transactions for status page
• removed speed up charts because they are only working with user statistics

[ascheibe]

r7048

• removed duplicate events
• got rid of compiler warnings

[ascheibe]

r7056

• disabled drag and drop for Hive Jobs
• fixed adding and deleting of Hive Tasks
• show Statelog after a Job is downloaded

[ascheibe]

r7059

• Permissions can now be deleted
• fixed overlay icons for permissions
• fixed overlay icons in job list

[ascheibe]

r7067

• allow drag and drop only for new jobs
• prepare optimizers on drag and drop

[ascheibe]

r7068

• added a default job name
• added check that a job name is set before upload

[ascheibe]

r7078 added missing invoke

[ascheibe]

r7103 cleaned up namespaces of the jobmanager

[ascheibe]

r7104 fixed graphical glitches and corrected tab order in the job manager

[abeham]

Are there any more changes coming to this ticket? If so, please take it again, it's currently in reviewing state.

[ascheibe]

r7115

• speed up download of tasks by avoiding unnecessary service calls
• display download progress correctly

[ascheibe]

r7125

• removed magic numbers for upload retries
• speed up job downloading by placing deserializing/downloading semaphores correctly
• increased max. number of parallel downloads/deserializations
• added more status messages when downloading to make it more clear what's actually happening
• renamed some variables

[ascheibe]

r7131 try to stop the slave service before uninstalling it

[ascheibe]

r7132

• fixed name of slave windows service

reviewing comments:

• reduced MaxParallelDownloads to 2 to not completely overload cpus
• renamed ServiceLocator to HiveServiceLocator

[abeham]

r7133

• Added missing configurations to Clients.Hive-3.3 project

[ascheibe]

r7135 some fixes for the slave tray ui:

• removed some more magic numbers
• fixed reconnecting to windows service when it was stopped
• added more time for stopping/starting windows service so that no exception is thrown because of a timeout

[abeham]

EngineHiveTask:

• please clean up the GetAsTaskData method, there's an uncommented lock. Is it necessary, why was it there in the beginning?

OptimizerHiveTask:

• GetNewRunName and GetRunNumber: idx will be 3 and not -1 if string cannot be found

TaskData:

• is contained in the file JobData.cs, should be renamed to TaskData.cs

PersistenceUtil:

• would be nicer to use the using pattern for MemoryStream

JobResultPoller:

• Why is stopRequested a property: private bool stopRequested { get; set; }

There are some singletons (e.g. HiveClient) which are not fully thread-safe (although for HiveClient this is probably not a problem). The ​MSDN implementation pattern for singletons suggests to either use double-checked locking or static initialization (private static readonly instance = new HiveClient();). Interestingly the article states that the static initialization is also lazily instantiated, because it is private and accessed only from Instance.

I'm a bit confused that the HiveServiceLocator.Instance property provides a public setter!?

[ascheibe]

r7142 implemented reviewing comments

[ascheibe]

Thanks for your comments. Concerning the EngineHiveTask: I don't think the locks are necessary. The GetAsTaskData is only used when uploading tasks and one task is only uploaded once, so this should be ok.

[ascheibe]

r7144 stop result polling before deleting a job

[ascheibe]

r7146 admin ui

• improved treeview
• added more icons and tooltips

[ascheibe]

r7152

• added missing invokes
• removed collecting types of object graph for retrieval of plugins as this isn't needed for MetaOpt. I think the problem was that MetaOpt always stored types of all available algorithms and problems which caused the problem (#1527) with Hive.

[ascheibe]

r7156 fixed starting, pausing and stopping of jobs and tasks

[ascheibe]

r7157 improved event logging in the Hive service

[ascheibe]

r7158 another event logging fix

[abeham]

I just tested downloading a running job (1250 tasks). It worked to download, but the JobManager never exited the populating / updating display state. I then switched the view to another job and back to the just downloaded job: The application locked up without using any CPU. I gave you permission to view the job it's called "QAPLIB SA parameter range (lipa20b - tai50b)". I'm pretty sure it'll still be running tomorrow, so you can test for yourself.

[ascheibe]

r7162 throw exceptions in Job Manager so that we can see if something went wrong

[ascheibe]

r7164 set taskDataInvalid when deserializing fails

[ascheibe]

r7165 build the task tree first and then display it. This should be more light on the CPU.

[ascheibe]

r7166

• don't serialize the results 2 times before uploading
• made slave a little bit more robust

[ascheibe]

r7171 communication with ui should be more stable now

[ascheibe]

r7177 fixed setting of priorities in the Job Manager

[ascheibe]

r7178 disabled checking if there are parent tasks which should to be calculated because this case doesn't exist at the moment

[ascheibe]

r7182 switched to HeuristicLab Log and LogView for the Slave UI

[ascheibe]

r7185

• possible fix for the slave hang problem: don't host the service on the thread it was created on
• added a trigger for deleting slavestatistics when statistics are deleted

[ascheibe]

r7187

• increased times between life cycles on the server
• some smaller performance improvements on the server

[ascheibe]

r7189

• increased timeout for slaves/tasks to 3 minutes
• moved the cleanup functionality to an own windows service. This will hopefully increase performance because it was done within the heartbeat calls up until now.

[ascheibe]

r7190

• fixed a typo
• increased transferring timeout

[ascheibe]

r7192 some small job manager ui fixes

[ascheibe]

r7191

• tooltip now shows the name and the id of a task
• the date is now shown on the x-axis for runs spanning multiple days

[abeham]

Deleting a job while it's being downloaded causes HL to crash

[ascheibe]

Thanks for the bug report!
r7200: don't allow deleting jobs while they are uploading/downloading

[ascheibe]

r7217 fixed compiler warning

[ascheibe]

r7218 renamed some jobs to tasks

[ascheibe]

r7219 renamed wrongly named folder

[ascheibe]

r7222 don't crash if there are no child hive tasks

[abeham]

The last test with several computers joining was successful and no further bugs have been identified.