Changeset 17574 for trunk/HeuristicLab.Services.Hive.DataAccess/3.3/Daos/StateLogDao.cs

Timestamp:

05/29/20 13:28:25 (4 years ago)

Author:

jkarder

Message:

#3062: overhauled statistics generation and cleanup

• switched to a single thread for database cleanup and statistics generation (executed sequentially)
• switched to preemptive deletion of items that are in status DeletionPending (for jobs: statelogs, taskdata, tasks)
• added code that aborts tasks whose jobs have already been marked for deletion
• added method UseTransactionAndSubmit in addition to UseTransaction in PersistenceManager
• updated DAO methods and introduced more bare metal sql
• introduced DAO methods for batch deletion
• fixed usage of enum values in DAO sql queries
• deleted unnecessary triggers tr_JobDeleteCascade and tr_TaskDeleteCascade in Prepare Hive Database.sql
• changed scheduling for less interference with janitor and other heartbeats
  • increased scheduling patience from 20 to 70 seconds (to wait longer to get the mutex for scheduling)
  • changed signature of ITaskScheduler.Schedule
  • added base class for TaskSchedulers and moved assignment of tasks to slaves into it
  • changed RoundRobinTaskScheduler to use bare metal sql
• made MessageContainer a storable type (leftover)
• updated HiveJanitorServiceInstaller.nsi

File:

• trunk/HeuristicLab.Services.Hive.DataAccess/3.3/Daos/StateLogDao.cs (modified) (2 diffs)

trunk/HeuristicLab.Services.Hive.DataAccess/3.3/Daos/StateLogDao.cs

@@ -40 +40 @@
     }
 
+    public int DeleteObsolete(int batchSize) {
+      return DataContext.ExecuteCommand(DeleteObsoleteQueryString, batchSize);
+    }
+
     #region Compiled queries
     private static readonly Func<DataContext, Guid, StateLog> GetByIdQuery =
@@ -54 +58 @@
          select stateLog).First(x => x.SlaveId != null));
     #endregion
+
+    #region String queries
+    private const string DeleteObsoleteQueryString = @"
+delete top ({0}) s
+from statelog s
+  join task t on t.taskid = s.taskid
+  join job j on j.jobid = t.jobid
+where j.jobstate = 'deletionpending'
+    ";
+    #endregion
   }
 }