Opened 15 months ago

Last modified 8 weeks ago

#2839 accepted feature request

Implement project management for Hive

Reported by: jkarder Owned by: jkarder
Priority: medium Milestone: HeuristicLab 3.3.16
Component: Hive.General Version: trunk
Keywords: Cc:

Description (last modified by fholzing)

Hive jobs should be assigned to projects instead of resources. We could better track resource usage per project and introduce some kind of credit system. Furthermore, this would allow us to assign one resource to different projects, which is currently not possible using resource groups.

fHolzing: In the "Select Project"-View, changing the selection of HiveSlaves via up/down arrow won't update the "Computing Resources"-Details.
By checking the Checkbox besides the Project (in my case HEAL), the slaves cannot be unchecked independently.
By Right-clicking on any Slave, the "Computing Resources" are updated, just like the selection (for a fraction of a second). The selection gets reverted und jumps back to the previously selected item, but the "Computing Resources" still displays the information about the now unselected item.

Change History (112)

comment:1 Changed 15 months ago by jkarder

  • Status changed from new to accepted

comment:2 Changed 15 months ago by jkarder

r15377: created branch for hive project management

comment:3 Changed 15 months ago by jkarder

r15378: worked on database model

comment:4 Changed 15 months ago by jkarder

r15379:

  • added Project and ProjectPermission DAOs and service methods
  • made Project.EndDate nullable

comment:5 Changed 15 months ago by jkarder

r15380: worked on Project and ProjectPermission service methods

comment:6 Changed 15 months ago by jkarder

  • Version set to branch

comment:7 Changed 15 months ago by abeham

  • Milestone changed from HeuristicLab 3.3.15 to HeuristicLab 3.3.16

comment:8 Changed 15 months ago by jkarder

r15399: worked on database model

comment:9 Changed 15 months ago by jkarder

r15401:

  • worked on hive administrator view
  • updated service clients

comment:10 Changed 15 months ago by jkarder

r15411: worked on database model

comment:11 Changed 15 months ago by jkarder

r15412:

  • worked on resources and projects views
  • changed resource selector to be able to select projects and assigned resources
  • updated service clients

comment:12 Changed 14 months ago by jkarder

r15422: worked on resources and projects views

comment:13 Changed 13 months ago by jzenisek

r15496: Added Dao for ResourcePermission entity

comment:14 Changed 13 months ago by jzenisek

r15497: worked on permission check at AddTask

comment:15 Changed 13 months ago by jzenisek

r15500: worked on resource permission checks (v2 = accelerated version)

comment:16 Changed 13 months ago by jzenisek

r15503: added ResourcePermission handling (still in progress)

comment:17 Changed 12 months ago by jzenisek

r15508: finalized permission checks in AddTask and revised implementation of ResourcePermission methods

comment:18 Changed 12 months ago by jzenisek

r15523: worked on db model and updated dbml

comment:19 Changed 12 months ago by jzenisek

r15526: reverted changes from last revision (r15523)

comment:20 Changed 12 months ago by jzenisek

r15527:

  • updated dbml (removed ResourcePermission and ResourceIds in Job-Table)
  • updated Resource and Project Daos

comment:21 Changed 12 months ago by jzenisek

r15528: added AssignedJobResource to dbml and Daos

comment:22 Changed 12 months ago by jzenisek

r15530:

  • worked on Job operations add&update
  • worked on ProjectPermission handling
  • worked on Project-Resource assignment

comment:23 Changed 12 months ago by jzenisek

r15540: added checks for the administration of project-resource assignments

comment:24 Changed 12 months ago by jzenisek

r15546: worked on ProjectResource assignment: (cascading) assigning & unassigning

comment:25 Changed 12 months ago by jzenisek

r15547: worked on ProjectResource assignment: (cascading) assigning & unassigning (reverted changes of config files from r15546)

comment:26 Changed 12 months ago by jzenisek

r15552: worked on permission checks in listing methods

comment:27 Changed 12 months ago by jzenisek

r15557: fixed tree building routine in ResourcesView

comment:28 Changed 12 months ago by jzenisek

r15559: worked on views for Projects and ProjectResources

Last edited 12 months ago by jzenisek (previous) (diff)

comment:29 Changed 12 months ago by jzenisek

r15567: worked on views for project and resource administration

Last edited 12 months ago by jzenisek (previous) (diff)

comment:30 Changed 12 months ago by jzenisek

r15576: worked on client side mgmt of project-resource assignments and project-user permissions

Last edited 12 months ago by jzenisek (previous) (diff)

comment:31 Changed 12 months ago by jzenisek

r15577: worked on service side mgmt of project-resource assignments and project-user permissions

Last edited 12 months ago by jzenisek (previous) (diff)

comment:32 Changed 11 months ago by jzenisek

r15580:

  • minor bugfix in HiveService.cs
  • corrected behavior of ProjectPermissionsView

comment:33 Changed 11 months ago by jzenisek

r15627: worked on Hive Job Manager: improved project/resource selector to show existent assignments and preserve selections

comment:34 Changed 11 months ago by jzenisek

r15628:

  • updated TaskDao towards independency of the formerly used Task-Resource assignment entity
  • updated several permission/assignment handling service methods for client
  • added AssignedJobResource DTO

comment:35 Changed 11 months ago by jzenisek

r15630:

  • updated Heartbeat processing (regarding: checking against AssignedJobResources and handling of the updated Job deletion routine)
  • updated Job deletion routine (still in progress at GenerateStatistics)

comment:36 Changed 11 months ago by jzenisek

r15641: finished implementation of job deletion routine

comment:37 Changed 11 months ago by jzenisek

r15642:

  • adapted handling of cascading revocations of project-user permissions and project-resource assignments: from deleting all AssignedJobResource (formerly) to deleting only entries which are effected by the previously mentioned revocations (now)
  • implemented job update (client side)
  • corrected & improved stateful project/resource selection dialog
  • corrected resource statistic calculation in project/resource selection dialog

comment:38 Changed 11 months ago by jzenisek

r15643:

  • worked on UpdateJob(..)
  • adapted permission and assignment handling methods
  • adpated dbml of HiveDataContext using the designer (added delete rules and onNull-handling)

comment:39 Changed 11 months ago by jzenisek

r15644: removed AssignedTaskResource entity

comment:40 Changed 11 months ago by jzenisek

r15658:

  • worked on (restricted) accessibility of hive's administration area for non-admin roles
  • adapted HiveClient & HiveAdminClient entity loading (client- & service-side)

comment:41 Changed 11 months ago by jzenisek

r15659:

  • added DimProject and FactProjectInfo entities to statistics generation
  • implemented tracking for projects

comment:42 Changed 11 months ago by jzenisek

r15666: implemented project facts-logging in HiveStatisticsGenerator

comment:43 Changed 11 months ago by jzenisek

r15671:

  • adapted computation of DimClient and FactClientInfo stats
  • adapted web app according to new stats computation

comment:44 Changed 11 months ago by jzenisek

r15715: improved permission checking of HiveService methods

comment:45 Changed 11 months ago by jzenisek

r15716: worked on stats generation

comment:46 Changed 10 months ago by jzenisek

r15737:

  • updated sql scripts (necessary foreign key option alterations & introduction of statistic tables)
  • updated HiveService according to changed client side (deletion-routine, permission checking,...)

comment:47 Changed 10 months ago by jzenisek

r15742: worked on HiveAdministrator:

  • corrected and modified CRUD operations
  • improved usability by providing detailed state information, adding dialogs etc.

comment:48 Changed 10 months ago by jzenisek

r15760:

  • minor changes regarding project- & resource selection
  • adapted DeleteOnNull rules in dbml

comment:49 Changed 10 months ago by jzenisek

r15761: renamed branch according to guidelines

comment:50 Changed 10 months ago by jzenisek

r15767: fixed cyclic-relation bug in projects- and resources-view by preventing parents from being dragged onto descendants

comment:51 Changed 10 months ago by jzenisek

r15768: fixed cyclic-relation bug (part 2: HiveAdminClient)

comment:52 Changed 10 months ago by jzenisek

r15777: improved project- & resource selection handling

comment:53 Changed 10 months ago by jzenisek

r15792: preliminary integration of several HiveAdministrator related improvements from #2877 (full merge pending)

  • slaves and slave groups are sorted by name
  • slave state is indicated by different colors
  • number of contained slaves is shown for each group
  • CPU utilization is shown for each calculating slave
  • last heartbeat date is shown for each offline slave

comment:54 Changed 10 months ago by jzenisek

r15813: improved HiveAdmin interactions:

  • collaps groups with slaves only
  • update scheduleView only if currently visible
  • made schedule scrollable for non-admins
  • changed selection backcolor
  • fixed add/delete project permission bug for non-admins

comment:55 Changed 10 months ago by jzenisek

r15819: implemented refreshing list of available (i.e. for non-admins assignable) resources depending on currently selected project

comment:56 Changed 10 months ago by pfleck

I would also like to have some additional features (mainly GUI wise) for the "Nr. of needed cores" for the Hive tasks.

  • Currently, the combobox to set the nr of cores is not working (no event registered), although, the rest of Hive already propagates the nr of needed cores correctly.
  • For a scenario where a batch run of algorithms should be distributed, setting the nr of cores needed for all tasks is quite tedious. Maybe, setting the nr of cores needed on the batch run propagates the demand to its children.
  • An option for Nr of needed cores that represents "all cores" could be useful (also with respect to executing ParallelEngines in Hive). Such a tasks should only be scheduled on a slave where no other tasks are currently running and the task will use up all available cores.

comment:57 Changed 8 months ago by jkarder

r15908: worked on hive project management

  • fixed some bugs
  • cleaned up

comment:58 Changed 8 months ago by jzenisek

r15913: updated genealogy computation for hive job administrator

comment:59 Changed 8 months ago by jzenisek

r15914: fixed several bugs in resource selector

comment:60 Changed 8 months ago by jzenisek

r15920: fixed several bugs in JobAdmin:

  • disabled play button if no project or resource is selected
  • extract stats for selected or alternatively for checked resources
  • collapse slaves-only groups if no slave is assigned explicitly
  • prevent unnecessary resource tree rebuilds (reset and style instead)
  • suppress double clicks in resource tree (i.e. prevent win forms check-error)

...

comment:61 Changed 8 months ago by jzenisek

r15922:

  • replaced use of TreeView with NoDoubleClickTreeView everywhere allegedly reasonable for Hive
  • updated genealogy computation for HiveAdminClient and corresponding views (ResourcesView & ProjectResourcesView)

comment:62 Changed 8 months ago by jzenisek

r15925: fixed bug in HeartbeatManager (more specific: corrected check if a task via its parenting job is still granted for a certain resource)

comment:63 Changed 7 months ago by jzenisek

r15933: several fixes:

  • rebuilt run in hive
  • adapted numeric selector (added FixedNumericUpDown.cs)

comment:64 Changed 6 months ago by jzenisek

r15953: Several changes and fixes:

  • added button tooltips
  • changed position of buttons
  • added refresh functionality to view of lightweight users ...

comment:65 Changed 6 months ago by jzenisek

r15954: updated behavior of permission assignment for project owners (client and server side)

comment:66 Changed 6 months ago by jzenisek

r15955:

  • fixed refresh in resource view
  • fixed assignment/include behavior in permission view

comment:67 Changed 6 months ago by jzenisek

r15956:

  • fixed project/resource selector in HiveJobManager
  • added error provider to project/resource selector
  • changed datetime pickers in HiveAdmin...

comment:68 Changed 6 months ago by jzenisek

r15966: Fixed several bugs and added project duration handling

  • fixed project duration handling (client- & service-side)
  • fixed tagging in HiveJobAdmin
  • added ProjectJobs view (under construction)
  • added necessary service methods

comment:69 Changed 6 months ago by jzenisek

r15969: finalized ProjectJobsView

comment:70 Changed 6 months ago by jzenisek

r15978: applied several fixes:

  • show full project-path in project/resource selector
  • handle lost of project-ownership by not withdrawing permissions
  • update automatically after hand-down save
  • lock jobs for which statistics/deletion is pending
  • lock the disabled checkbox in ProjectResourcesView...

comment:71 Changed 5 months ago by jzenisek

r15992: fixed couple of minor issues:

  • changed tags in resource selector
  • added project information in job list and adapted sortation
  • fixed hand-down save by withdrawing additional offset-rights (permissions, resources),...

comment:72 Changed 5 months ago by jzenisek

r15995: adapted illustration of project and resource ancestry in HiveAdministrator and HiveJobAdministrator

comment:73 Changed 5 months ago by jzenisek

r16040: fixed handling of project related updates for project owners, who are no admins and do not own parent projects

comment:74 Changed 5 months ago by jzenisek

r16043: adapted handling of project start/end boundaries for non-admins

comment:75 Changed 5 months ago by jzenisek

r16044:

  • adapted icons: replaced ".PublishToWeb" with ".Save"
  • pre-select project with all resources for new jobs, if it is the only available

comment:76 Changed 4 months ago by jkarder

r16050: fixed connection string

comment:77 Changed 4 months ago by jkarder

r16057:

comment:78 Changed 4 months ago by jkarder

r16060: worked on hive project management

  • added .ToList() calls within some service methods to prevent TransactionExceptions
  • added check for InvokeRequired
  • improved RunInHiveMenuItem
  • removed .resx files from projects

comment:79 Changed 4 months ago by jkarder

r16062: worked on hive project management

  • removed .resx file from project

comment:80 Changed 4 months ago by jkarder

r16064: worked on hive project management

  • fixed bug in StyleTreeNode

comment:81 Changed 4 months ago by jkarder

r16066: worked on hive project management

  • improved RunInHiveMenuItem

comment:82 Changed 4 months ago by jkarder

r16068: worked on hive project management

  • derived UserListView, UserGroupListView and RoleListView from ItemCollectionView<T>

comment:83 Changed 4 months ago by jzenisek

r16072: worked on HiveProjectMgmt:

  • adapted enabled state of buttons

comment:84 Changed 4 months ago by jzenisek

r16089: updated setting enabled state of controls in project view

comment:85 Changed 4 months ago by jkarder

r16091: worked on hive project management

  • updated app/web configs and settings

comment:86 Changed 4 months ago by jkarder

r16092: merged [16057-16091/trunk] into branch

comment:87 Changed 4 months ago by jkarder

r16093: worked on hive project management

  • removed usage of null-conditional operator

comment:88 Changed 4 months ago by jkarder

r16094: worked on hive project management

  • removed usage inline out variables and string interpolation

comment:89 Changed 4 months ago by jkarder

r16095: worked on hive project management

  • removed usage of null-conditional operator

comment:90 Changed 4 months ago by jkarder

r16116: worked on hive project management

  • fixed plugin dependencies
  • fixed NullReferenceException in HiveTaskView
  • added cloning ctor for AssignedJobResource and AssignedProjectResource

comment:92 Changed 4 months ago by jkarder

  • Version changed from branch to trunk

comment:93 Changed 4 months ago by jkarder

r16122: worked on hive project management

  • fixed handling of enabled states in ResourcesView

comment:94 Changed 3 months ago by fholzing

  • Description modified (diff)

comment:95 Changed 3 months ago by fholzing

  • Description modified (diff)

comment:96 Changed 3 months ago by fholzing

  • Description modified (diff)

comment:97 Changed 3 months ago by abeham

Exceptions

Create a project and then delete it before it is saved:

System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
   at System.ThrowHelper.ThrowKeyNotFoundException()
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at HeuristicLab.Clients.Hive.Administrator.Views.ProjectsView.<removeButton_Click>d__13.MoveNext() in c:\...\HeuristicLab.Clients.Hive.Administrator\3.3\Views\ProjectsView.cs:line 268
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

Click on a user in the Permissions tab:

System.ArgumentException: View "HeuristicLab.Clients.Access.Views.RefreshableLightweightUserInformationView" cannot display content "HeuristicLab.Clients.Access.LightweightUser".
   at HeuristicLab.MainForm.WindowsForms.ViewHost.set_ViewType(Type value) in c:\...\HeuristicLab.MainForm.WindowsForms\3.3\Controls\ViewHost.cs:line 126
   at HeuristicLab.Clients.Hive.Administrator.Views.ProjectPermissionsView.treeView_AfterSelect(Object sender, TreeViewEventArgs e) in c:\...\HeuristicLab.Clients.Hive.Administrator\3.3\Views\ProjectPermissionsView.cs:line 120
   at System.Windows.Forms.TreeView.OnAfterSelect(TreeViewEventArgs e)
   at System.Windows.Forms.TreeView.TvnSelected(NMTREEVIEW* nmtv)
   at System.Windows.Forms.TreeView.WmNotify(Message& m)
   at System.Windows.Forms.TreeView.WndProc(Message& m)
   at HeuristicLab.Clients.Hive.Views.TreeView.NoDoubleClickTreeView.WndProc(Message& m) in c:\...\HeuristicLab.Clients.Hive.Views\3.3\TreeView\NoDoubleClickTreeView.cs:line 31
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

There is probably some dangling event handler that does not get deregistered. Opening Services > Access > User Information and clicking on the "Refresh" button results in the following exception after playing around with the projects:

ObjectDisposedException: Cannot access a disposed object.
Object name: 'ProjectView'.
   at System.Windows.Forms.Control.CreateHandle()
   at System.Windows.Forms.Control.get_Handle()
   at HeuristicLab.MainForm.WindowsForms.ControlExtensions.SuspendRepaint(Control control) in c:\...\HeuristicLab.MainForm.WindowsForms\3.3\Controls\ControlExtensions.cs:line 38
   at HeuristicLab.MainForm.WindowsForms.View.SuspendRepaint() in c:\...\HeuristicLab.MainForm.WindowsForms\3.3\Views\View.cs:line 255
   at HeuristicLab.MainForm.WindowsForms.View.set_Enabled(Boolean value) in c:\...\HeuristicLab.MainForm.WindowsForms\3.3\Views\View.cs:line 82
   at HeuristicLab.Clients.Hive.Administrator.Views.ProjectsView.SetEnabledStateOfControls() in c:\...\HeuristicLab.Clients.Hive.Administrator\3.3\Views\ProjectsView.cs:line 126
   at HeuristicLab.Clients.Hive.Administrator.Views.ProjectsView.AccessClient_Instance_Refreshing(Object sender, EventArgs e) in c:\...\HeuristicLab.Clients.Hive.Administrator\3.3\Views\ProjectsView.cs:line 183
   at System.EventHandler.Invoke(Object sender, EventArgs e)
   at HeuristicLab.Clients.Access.AccessClient.OnRefreshing() in c:\...\HeuristicLab.Clients.Access\3.3\AccessClient.cs:line 96
   at HeuristicLab.Clients.Access.AccessClient.<>c__DisplayClass8.<ExecuteActionAsync>b__6() in c:\...\HeuristicLab.Clients.Access\3.3\AccessClient.cs:line 74

Issues

  • Creating a new project, then changing its name -> treeview is not updated until saved
  • It's a bit odd that the save button for details is above the treeview. What does that button save?
  • The "Slaves" tab should be renamed "Resources"
  • I would use a fixed (approx.) 300px width of panel1 in the split control and keep panel1 size fixed during resize
  • Trying to add HEAL resource group to a project that is not a sub-project of the HEAL project is not possible. The resources can be selected and save can be clicked. There is no exception or message that the save was not successful.
    • Btw, first creating that project under group HEAL, then dragging this to root level maintains the assigned resources. I can then create sub-projects where I can assign the same resources.

Questions

  • Project Details: Why a refresh after the Owner combobox?
  • What does "Permissions" of projects actually mean? What kind of permissions are there? Read, Execute?
  • How can I create a new project that is not a sub-project of some other?
    • jkarder showed that this can be done using drag & drop. I think there should be a more obvious way.
  • What kind of privilege is required to open the "Services > Access > User Administration". Being "Hive Administrator" doesn't seem to qualify: "Access Service Administrator - You do not seem to have the permissions to use the Access Service Administrator. If that's not the case or you have any questions please write an email to support..."
    • Access Service Administrator
Last edited 3 months ago by abeham (previous) (diff)

comment:98 Changed 3 months ago by gkronber

r16117 broke the HeuristicLab.MetaOptimization addon

HiveEngine.cs(298, 24): error CS1061: 'Job' does not contain a definition for 'ResourceNames' and no extension method 'ResourceNames' accepting a first argument of type 'Job' could be found (are you missing a using directive or an assembly reference?)

HiveEngine.cs(321, 60): error CS1061: 'IHiveService' does not contain a definition for 'DeleteJob' and no extension method 'DeleteJob' accepting a first argument of type 'IHiveService' could be found (are you missing a using directive or an assembly reference?)

comment:99 Changed 3 months ago by jkarder

r16173:

  • fixed compilation errors in HiveEngine
  • minor changes

comment:100 Changed 3 months ago by jzenisek

r16184: fixed bug appeared when clicking on user

Last edited 3 months ago by jzenisek (previous) (diff)

comment:101 Changed 3 months ago by jzenisek

r16185:

  • updated enabled state of controls for projects
  • updated enabled state of controls for resources

comment:102 Changed 3 months ago by jzenisek

r16186: fixed lightweightUserInformationView

comment:103 Changed 3 months ago by jzenisek

r16187:

  • fixed enabled state of controls bug for initial project selection regarding permission detail views
  • fixed problem at resource assignment for root projects

comment:104 Changed 3 months ago by jzenisek

r16202: fixed several bugs:

  • renamed tab "Slaves" to "Resources"
  • fixed bugs in job manager > resource selector reported by fholzinger
  • adapted client-side handling of disabled resource ancestors
  • adapted parentship check in HiveAdminClient
  • adapted seting of enabled state of controls for project details, project resources
  • introduced "inactive" tag in ProjectsView

comment:105 Changed 3 months ago by jzenisek

r16203: fixed job listing method for hive administrators

comment:106 Changed 3 months ago by jzenisek

r16204: added assembly reference

comment:107 Changed 3 months ago by jzenisek

r16205: adapting project job view

comment:108 Changed 3 months ago by jzenisek

r16208: updated job execution implementation in ProjectJobsView

comment:109 Changed 3 months ago by jzenisek

r16209:

  • adapted job execution implementation at ProjectJobsView
  • prohibited resource checking for non-admins

comment:110 Changed 3 months ago by jzenisek

r16211:

  • disabled „Start Time“, „End Time“, „All Day Event“ checkbox for non admins
  • reload ProjectResources on change of resources
  • fixed bug in ProjectResources regarding ungroupedGroupNode

comment:111 Changed 2 months ago by jzenisek

r16219: fixed several bugs:

  • made additional slaves checkable
  • changed default collapsing of additional slave group
  • added info in project name text box

comment:112 Changed 8 weeks ago by jkarder

r16257: worked on hive project management

  • fixed GetAvailabilityStatsPerProjectQueryString
Note: See TracTickets for help on using tickets.