Opened 12 years ago
Last modified 5 years ago
#1888 assigned task
Optimization-as-a-Service
Reported by: | spimming | Owned by: | swagner |
---|---|---|---|
Priority: | medium | Milestone: | HeuristicLab 3.3.x Backlog |
Component: | General | Version: | branch |
Keywords: | Cc: |
Description
Change History (55)
comment:1 Changed 12 years ago by spimming
- Status changed from new to assigned
comment:2 Changed 12 years ago by spimming
comment:3 Changed 12 years ago by spimming
r8232 scripts for sql azure db adapted
comment:4 Changed 12 years ago by spimming
r8235 Hive server web role initial commit
comment:5 Changed 12 years ago by spimming
- slave worker role initial commit
- start client communication service optionally
- make slave client communication interchangeable ('PipeCom', 'TraceCom')
comment:6 Changed 12 years ago by spimming
r8243 added pre-build event to project properties
comment:7 Changed 12 years ago by spimming
- added signing certificates
- added status pages
comment:8 Changed 12 years ago by spimming
- Change hive server from service configuration file
- Created service configuration settings for server address and certificate
- Added HeuristicLab.Clients.Hive-3.3 project to solution
comment:9 Changed 12 years ago by spimming
- calling roles caused an exception
- fix: call default provider instance directly
comment:10 Changed 12 years ago by spimming
- disabled some transactions since dtc is not supported in sql azure
- settings in dataaccess updated
comment:11 Changed 12 years ago by spimming
r8270 Caution: disabled transactions in the TransactionManager
comment:12 Changed 12 years ago by fschoepp
- Added IConnectionProvider interface + implementation based on jheinzelreiters transaction management
- Fixed transaction being promoted to dtc; transactions enabled again
- Changed HiveDao: transactions are managed properly
comment:13 Changed 12 years ago by fschoepp
- Owner changed from spimming to fschoepp
comment:14 Changed 12 years ago by fschoepp
- Created a project which contains the back-end controller for the Optimization WebSite
- Added a WCF-back-end-controller which generates all available optimization problems (currently in-memory solution: PlaceholderControllerService.cs)
- Created a WebRole using ASP.NET MVC 3 for the Optimization Web Site
- WebSite authenticates users with the HeuristicLab.Authentication membership provider and database
- WebSite crawls and displays all available optimization scenarios by using the WCF-back-end controller (test with: http://localhost:.../optimization)
comment:15 Changed 12 years ago by fschoepp
- Web project now creates custom html for each type we want to enter / display
- Added endpointConfigurationName to HiveServiceLocator (because Web Project contains more than one endpoint configuration)
- Removed logging statement from ConfigurationService to prevent exception during failure of loading ConfigurationSettings
- ApplicationManager: Changed default implementation to WebApplicationManager (instead of LightWeight) for testing purposes within Web Project
- WebApplicationManager: The application manager which returns plugin descriptors from the currently loaded assemblies (instead of LightweightAppManager)
- HiveService: Fixed a transaction bug
- IControllerService: Created a method to dispatch Scenarios to certain IScenarioManager (in this case HiveScenarioManager)
- Added more mappable types to ControllerModel
- PlaceholderControllerService dispatches all Scenarios to the HiveScenarioManager
- Web project now dispatches the scenario to the controller after pressing "Run Job"
comment:16 Changed 12 years ago by fschoepp
- Controller is now capable of gathering Hive Jobs
- Hive Jobs will be mapped to independent Job-Class (shared between Controller and Frontend)
- HiveScenarioManager is capable of gathering Hive Jobs + their results
- Job Results will be mapped to string properties
- Frontend renders all results after opening the job details
- Misc: Frontend now passes User-object to Controller so that it is able to connect to the Hive Service (hardcoded values removed)
comment:17 Changed 12 years ago by fschoepp
- Added a parser for independent scenarios (using the model of the optimization backend)
- Optimization scenario sample can be found in mappings folder of the web project.
- Added IScenarioMapper interface which provides functionality to map from the optimization data model to a backend model (e.g. Heuristic Lab data model)
- Implementations of IScenarioMapper have to be provided as C# code (strings) which will be compiled by using a CSharpCodeProvider. Once compiled, the implementations of the IScenarioMapper are being cached within the platform for further usage.
- Fixed a bug in web template DecimalMatrix (using i instead of j)
- Added missing thumprint of localhost certificate to the optimization web project (ServiceConfiguration.Local.cscfg / ServiceConfiguration.Cloud.cscfg)
- Test project now provides following test cases: Mapping types using IronPython and mapping types using Otis
comment:18 Changed 12 years ago by fschoepp
- Added a administrator web interface for job management
- Fixed Hive Client (PluginValidator) to find the assemblies within the right directories
- Reorganized controller classes (Folders HL, Interfaces, Azure)
- You may now successfully schedule and run jobs with the web ui.
comment:19 Changed 12 years ago by fschoepp
Backend changes:
- Simplified job state detection (only one hive call will be made to detect all states now, instead of one additional call per job)
- Reorganized classes (moved model classes into Model folder)
Website changes:
- Website now heavily uses JavaScript to achieve better user experience
- JavaScript degrades gracefully, except for plots
- Tables: Added jquery-datatable-plugin to extend tables (pagination + search functionality)
- OaaS-Website now uses the design of the HL websites (found in WebApplication branch)
- Added jqplot to render zoomable line plots for HL-Datatables
- Styling.js: Plots will be generated by using an ajax call; additional jquery-styling occurs within this file.
- Added jquery-ui-1.9.2 which is capable of handling/rendering tabs, accordions and resizers.
comment:20 Changed 12 years ago by fschoepp
- Model: OptimizationScenario may be a tree of algorithms (and problems)
- Model: Renamed InputParameters to ProblemParameters (as they are the parameters of a problem)
- Model: Added JobExecutionDetails which contain Repetitions + Group (resource to use)
- ScenarioParser parses the new XML scenario files
- Website + Model: You are now able to add/remove rows from a table (no JavaScript involved yet)
- Website + Controller: Added repetitions (enables batch jobs) and group (resource to use) to OaaS which will be used by the controller to schedule the job
- Website: Updated templates to use new model structure
- Website + Scenarios: Added the new algorithm Benchmark Algorithm
- Controller: Added a singleton to make the (Azure/Mockup)-DAL exchangeable
- Controller: Added mockup classes for DAL + IScenarioManager
- Website/Result Page: Line Diagrams will be added via JavaScript, crawling their data using AJAX
- Website: Most configuration parameters can be set in the ServiceDefinition directly
- Added a mockup for the Membership classes: These can be used if no network connection is available or if other parts of the app shall be tested
- Scenarios: Updated TSP mappings to new xsd
comment:21 Changed 12 years ago by fschoepp
- Janitor is now working as expected in Windows Azure
- Added basic support for experiments (draggable experiments)
- Added methods to save/read experiments from TableStorage
- The job status can now be retrieved by using the GetTasks/GetTaskData methods
- Added a class to convert JSON-objects to Algorithm instances
- Web page: Added experiment button to navigation
comment:22 Changed 12 years ago by fschoepp
- Experiments will be saved as JSON elements within the blob store.
- Added simple model and JSON converters.
- Backend stores and runs experiments.
- Updated interfaces to save/read experiments.
- Added a binding to automatically map incoming JSON ajax requests to experiment models.
- Added the javascript DatatypeMapper to map parameter inputs to the right html elements and vice versa.
- Added smartwizard to generate Wizards for creating new experiments (New.cshtml).
comment:23 Changed 11 years ago by fschoepp
- Fixed build of OaaS-HL (removed an unnecessary reference to a .resx file).
- Build order is as follows: HeuristicLab.ExtLibs, HeuristicLab 3.3, HeuristicLab 3.3 Services (Administrator)
- Added missing respositories.config for nuget (Heuristiclab/branches/OaaS/packages).
- Added missing nuget references (Json).
comment:24 Changed 11 years ago by fschoepp
- Added Update / GetExperiment... methods to the controller for updating and querying experiments.
- The AlgorithmConverter class now properly converts from/to JSON format.
- Integrated backbone js as MVC provider for JavaScript + jquery.
- Added experiment.model.js + experiment.view.js + experiment.controller.js containing the MVC impl. for the Experiment pages.
- Added new methods to the ExperimentController usable by the backbone js model implementation.
- Added the experiment dialog from HL 3.3.7 (variate experiment parameters). It's capable of variating the algorithm parameters.
- JavaScript is a must have for the "Edit Experiment" / "Build Experiment" pages.
comment:25 Changed 11 years ago by fschoepp
- DAL: Added a Delete method which deletes by experiment id.
- HL DataTables will now be transposed and mapped as double[ROWS][COLUMNS] (transposed)
- JS: Moved all classes into "modules" to prevent namespace pollution (using OAAS_MODEL for model classes, OAAS_VIEW for views and OAAS_CONTROLLER for controllers)
- JS: Moved DatatypeMapper classes into Backbone views
- JS: Models now correctly send DELETE requests
- Added a new job overview page (which also renders run details) using AJAX
- Using moment.min.js to format DateTime as string
- Controllers now inherit from BaseController which provides a RedirectToLoginIfNecessary-method
- Added loading animations to several AJAX bound places (loading experiments / scenarios)
- Added a section to _Layout.cshtml which allows page-specific JavaScript includes (<script> only for a certain page)
- Fixed Build/Edit of experiment menu redirecting to the wrong page
- The Experiment Variation Dialog disables input fields, if the property has not been activated before
comment:26 Changed 11 years ago by fschoepp
- The Edit-Experiment-Page now correctly displays the experiments again.
- Disabled key navigation for all wizards.
- Added a RunCollectionView to render different Plots for the run results of a job.
- Added a Boxplot-Chart and a Bubble-Chart for rendering the results.
- Added underscoreAddon.js which provides methods to calculate mean/sum of an JavaScript array/list.
comment:27 Changed 11 years ago by fschoepp
- Added input parameters to the run class. They will be populated by the back-end and returned to the web pages which renders them.
- Added a ParameterMapper class which converts HL specific model classes to OaaS independent classes. The conversion gets delegated to IParameterHandler which have to be implemented manually and registered for a certain Type before. All parameters which can be converted to IStringConvertible, IStringConvertibleMatrix, IString* will be converted into a OaaS-StringValue instances.
- Added IParameterHandlers for PathTSPTour and PermutationType (required for TSP).
- AlgorithmConverter now makes sure that the id of a run is unique. (All runs of a RunList will be shown now.)
- Web pages are capable of rendering both the results of a run and their input parameters (added a accordion to wrap their content).
- Renamed "Traveling Salesman Problem" to "Genetic Algorithm - TSP".
- Changed js-files to render both input and result parameters of a Run.
comment:28 Changed 11 years ago by fschoepp
- Added IVisualExtensionDao to the backend representing js extension that create new viewable elements of an algorithm.
- Started to upgrade the frontend to render those new javascript UI extensions.
comment:29 Changed 11 years ago by spimming
- Merged revisions from trunk
comment:30 Changed 11 years ago by fschoepp
- WorkerRole of Slave now logs all exceptions during startup to the blobstore (slavelog).
- The PluginManager throws an exception during CheckWorkingDirectories() (Save method) which will be caught now, preventing crashes in Windows Azure.
- "db.DeferredLocal = false" has been removed to prevent loading bugs.
- HiveScenarioManager doesn't crash anymore, if he can't find an algorithm within a job during retrieval of the run results.
comment:31 Changed 11 years ago by fschoepp
comment:32 Changed 11 years ago by fschoepp
- Added visual extensions (dynamic JavaScript) which will be used to render additional result parameters specific to scenarios (e. g. create a graphical representation of a TSP).
- Added relationship between jobs and experiments (otherwise, it's not possible to get the job's experiment).
- Updated Admin page to allow removal/addition of visual extensions.
- Added back-end logic to store/retrieve/delete visual extensions.
- Added visual extension functionality to the JavaScript views/controllers (job.*.js).
- Added tsp.js which is a visual extension for the "Genetic Algorithm - TSP" scenario. It adds a graphical representation of the TSP (just like the C# version) to the results.
comment:33 Changed 11 years ago by fschoepp
r9508: HL:
- Web projects requires different users to interact with hive. The singleton HiveServiceLocator.Instance doesn't allow different users at the same time, resulting in serialization during access of HiveClient methods.
The following changes have been introduced in favor of a parallel use of the HL libs:
- HiveClient, TaskDownloader and ConcurrentTaskDownloader may now use a different IHiveServiceLocator than HiveServiceLocator.Instance (all methods have appropriate overloads now).
- The default instance is still HiveServiceLocator.Instance.
Automated Scaling of Instances:
- Added Scaler project to solution which represents a WorkerRole that scales the slave instances based on the global cpu utilization of all slaves.
- Scaler is based on WASABi, rules can be adjusted in rulesstore.xml. Basic rule is: if < 45% global cpu utilization => remove an instance; if > 65% cpu => add an instance. Minimum boundary is 1 and maximum boundary is 8 slave instances.
- Adjusted Slave project to automatically register itself to a SlaveGroup during WebRole startup (can be adjusted in service configuration).
Web-Frontend:
- Added basic error messages to the dialogs when an ajax call fails.
- Removed Styling.js from scripts.
comment:34 Changed 11 years ago by fschoepp
- Removed unnecessary files from web project.
- Cleaned up slave.
comment:35 Changed 11 years ago by spimming
- Added billing test client
- Use entity framework for DAL
- Modified service interface and model classes
comment:36 Changed 11 years ago by spimming
- Added dal and service interfaces and implementations
- Added dbcontext for billing
comment:37 Changed 11 years ago by spimming
- Revised model classes
- Modified relationship definitions according to model types
comment:38 Changed 11 years ago by fschoepp
- Added an overview for users to inspect their orders
- Order Administrators may now suspend or reactivate orders
- When creating an order, its necessary to enter payment information (paypal, credit card,...) before
- Also, the billing period and type must be entered during the creation of an order.
comment:39 Changed 11 years ago by fschoepp
- Billing Component may now be deactivated by setting the BillingEnabled flag within the OaaS service configuration.
- Added views for invoices and usage data.
- Changed appearance of the SaveOrder view.
comment:40 Changed 11 years ago by fschoepp
- Removed readme from scaler
- Added BillingEnabled flag to configuration files
comment:41 Changed 11 years ago by spimming
- Workaround for enums in model classes
- Implemented dao methods for model
- Added test data in db seed method
- changed EF configuration
- Implemented some BL methods
comment:42 Changed 11 years ago by spimming
- Added new BillingService methods
- Disabled proxy generation and lazy loading!
- Extended see method with additional test data
- Added properties to order and invoice model
- Initial commit of BillingEngine module
comment:43 Changed 11 years ago by spimming
- Owner changed from fschoepp to spimming
comment:44 Changed 11 years ago by spimming
- Users now contain ContactInformation
- Fixed formatting in invoice template
- Fixed calculation for next invoice date
- Worked on indentation for invoice fields
comment:45 Changed 11 years ago by spimming
- Model classes implement new interface to track current state of entity
- Mark EntityState property as 'not mapped'
- Set hook to initialize entity as unchanged
- Extension method for DbContext to apply changes only on modified entity
comment:46 Changed 11 years ago by spimming
- Set flag to indicate a modified model object
- Add dao query method to specify includes as parameter
- cleaned up the using of namespaces to qualify types
comment:47 Changed 11 years ago by spimming
- Added model and view for contact information
- Insert contact information on SaveOrder page
- Set correct entity state
comment:48 Changed 11 years ago by gkronber
Please accept this ticket.
comment:49 Changed 11 years ago by spimming
- Status changed from assigned to accepted
comment:50 Changed 11 years ago by spimming
- enabled transactions
- enabled tracing output
- set correct invoice data
- saving invoices
comment:51 Changed 10 years ago by ascheibe
- Owner changed from spimming to ascheibe
- Status changed from accepted to assigned
comment:52 Changed 8 years ago by ascheibe
- Owner changed from ascheibe to swagner
comment:53 Changed 6 years ago by jkarder
- Milestone changed from HeuristicLab 3.3.x Backlog to HeuristicLab 3.3.16
HL architects: swagner will rename this branch
comment:54 Changed 6 years ago by abeham
r16115: renamed branch
comment:55 Changed 5 years ago by abeham
- Milestone changed from HeuristicLab 3.3.16 to HeuristicLab 3.3.x Backlog
Note: See
TracTickets for help on using
tickets.
r8228 initial version for optimization as a service branch