Changes between Version 5 and Version 6 of Documentation/Howto/OptimizeExternalApplications

Timestamp:

06/04/10 15:16:09 (14 years ago)

Author:

abeham

Comment:

--

Documentation/Howto/OptimizeExternalApplications

@@ -11 +11 @@
 Among the many possibilities and technologies that have emerged to provide a base for performing "distributed computing" in a wide sense, one of the first technologies was Remote Procedure Call (RPC). The idea is very simple: Instead of calling a local procedure to do some kind of calculation, a procedure is called that is not defined within the same executable or one of its dependencies. The client thus is in some ways hands the parameters of the method to another program or server, waits for the computation and then reads back the return value. In such a broad sense this is how most web applications nowadays work and indeed it is not until the invention of Representational State Transfer (REST) that there was a revolutionary change from this early RPC paradigm.
 
-Recently [http://www.google.com Google], one of the biggest players in the world wide web released parts of their core technologies to the general public. The framework that they call [http://code.google.com/p/protobuf/ Protocol Buffers] combines a domain specific language (DSL) for describing messages with an RPC framework that is used to pass these messages among different servers. The RPC framework itself is not released, but the DSL for describing messages and translating them to objects of several supported programming languages was released under the BSD license. Google directly supports C++, Java, and Python, but many developers have programmed ports of protocol buffers to different languages such as C#, Visual Basic, Objective C, Perl, Haskell, and [http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns many more]. The [http://code.google.com/apis/protocolbuffers/docs/overview.html documentation] on protocol buffers is extremely helpful in understanding the framework. A similar framework called Thrift has evolved that backs the operation of [http://www.facebook.com Facebook] and was also released to the public. These technologies provide convenient ways to define messages, manipulate them, and serialize them to small sizes at high speeds.
+Recently [http://www.google.com Google], one of the biggest players in the world wide web released parts of their core RPC technologies to the general public. The framework that they call [http://code.google.com/p/protobuf/ Protocol Buffers] combines a domain specific language (DSL) for describing messages with an RPC framework that is used to pass these messages among different servers. The RPC framework itself is not released, but the DSL for describing messages and translating them to objects of several supported programming languages was released under the BSD license. Google directly supports C++, Java, and Python, but many developers have programmed ports of protocol buffers to different languages such as C#, Visual Basic, Objective C, Perl, Haskell, and [http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns many more]. The [http://code.google.com/apis/protocolbuffers/docs/overview.html documentation] on protocol buffers is extremely helpful in understanding the framework. A similar framework called Thrift has evolved that backs the operation of [http://www.facebook.com Facebook] and was also released to the public. These technologies provide convenient ways to define messages, manipulate them, and serialize them to small sizes at high speeds.
 
 In HeuristicLab using remote procedure calls seems to fit very well with what we are trying to achieve in "exporting" the problem definition. The problem, to an optimizer, is basically the evaluation function and the solution representation is their common knowledge. A client is provided in HeuristicLab as well as a framework that enables developers to write a service which exposes the evaluation function to HeuristicLab. Using this service foreign language applications and problems can effectively communicate with HeuristicLab and have their parameters optimized by HeuristicLab's powerful optimization library. Please note that while we're talking about RPC, the provided frameworks are not compatible with the RPC standard, but are rather simplified to ease application. Think of RPC as a paradigm rather than a standard.
 
-Communication however also requires a given media over which to exchange the information. So far HeuristicLab offers two choices for the underlying media:
+Communication also requires a given media over which to exchange the information. So far HeuristicLab offers two choices for the underlying media:
  1. The external program can be started as a process from HeuristicLab and the communication occurs via the process' stdin and stdout. This requires that the external program can be executed under Windows. If the developer controls the standard input and output streams and does not need to write or read other data through them, this might be a simple solution.
  2. The external program is started independently from HeuristicLab and opens a TCP/IP port for communicating over a network. This is independent of the platform and a universal solution that should work in most cases.
@@ -37 +37 @@
 == Architecture Details ==
 
+Optimizing external applications divides into two parts:
+ 1. Providing an evaluation service for HeuristicLab in the external application
+ 2. Preparing an `ExternalEvaluationProblem` in HeuristicLab
+
+These two tasks are described in this section in more detail.
+
+=== Providing an evaluation service ===
+
 The following class diagram displays the classes and interfaces present in the java service framework. There is an abstract base class `Channel` on the one hand which provides methods for sending and receiving messages and several concrete implementations and on the other hand an abstract base class `Service` that provides the concrete `PollService` and `PushService`. Each Channel has a corresponding factory which implements `IChannelFactory`.
 
@@ -44 +52 @@
 
 [[Image(ExternalEvaluationServiceTestCD.png)]]
+
+=== Preparing an `ExternalEvaluationProblem` ===
+
+In the HeuristicLab Optimizer a number of algorithms and problems are available which can be created, viewed and parameterized. Among them is a special problem designed for the purpose of calling an external evaluation function called `ExternalEvaluationProblem`. The problem allows the user to define a customized solution representation, as well as customizing the operators that are available to solve the problem. The following screenshot shows the default parameters of this problem.
+
+[[Image(ExternalEvaluationProblemParameters.png)]]
+
+The list of parameters that are available there are briefly described:
+ * '''!BestKnownQuality''' - Displays the best known quality of the problem, this is updated through a special analyzer.
+ * '''!BestKnownSolution''' - Houses the best known solution so far, that is the scope that contains the solution representation
+ * '''Client''' - The client that transmits the solution to the external application. The user configures the client by defining the appropriate channel and channel connection information.
+ * '''Evaluator''' - The evaluator is an operator that collects variables from the scope and includes them in a message which will be sent to the external application. The user has to configure the evaluator such that it can find the variables to collect.
+ * '''Maximization''' - Is necessary for the algorithm to know whether it should minimize or maximize the quality value.
+ * '''Operators''' - Contains a list of operators that the problem provides to the algorithm. In this list any Crossover, Manipulation or other operator will be passed to the algorithm and will be made selectable there.
+ * '''!SolutionCreator''' - Is an operator that creates a solution. This operator can also be adjusted by the user as needed. One can use the included representation generators in the !HeuristicLab.Encodings namespace, or use own generators from an own plugin.
+
+In the tutorial section it will be shown how to configure these parameters to solve a real problem.
+
 == Tutorial ==