Quick guide: Implement an Evaluator

This quick guide examines how to implement evaluators. We will simply use an existing evaluator and have a look at how it works. In the following you find the source of the evaluator for the OneMax problem:

  [Item("OneMaxEvaluator", "Evaluates solutions for the OneMax problem.")]
  [StorableType("c33d28ed-d5c8-424c-85ff-c3e25c1f63f6")]
  public class OneMaxEvaluator : InstrumentedOperator, IOneMaxEvaluator {
    public ILookupParameter<DoubleValue> QualityParameter {
      get { return (ILookupParameter<DoubleValue>)Parameters["Quality"]; }
    }

    public ILookupParameter<BinaryVector> BinaryVectorParameter {
      get { return (ILookupParameter<BinaryVector>)Parameters["BinaryVector"]; }
    }

    [StorableConstructor]
    protected OneMaxEvaluator(StorableConstructorFlag _) : base(_) { }
    protected OneMaxEvaluator(OneMaxEvaluator original, Cloner cloner) : base(original, cloner) { }
    public OneMaxEvaluator()
      : base() {
      Parameters.Add(new LookupParameter<DoubleValue>("Quality", "The evaluated quality of the OneMax solution."));
      Parameters.Add(new LookupParameter<BinaryVector>("BinaryVector", "The OneMax solution given in path representation which should be evaluated."));
    }

    public override IDeepCloneable Clone(Cloner cloner) {
      return new OneMaxEvaluator(this, cloner);
    }

    public sealed override IOperation InstrumentedApply() {
      BinaryVector v = BinaryVectorParameter.ActualValue;

      double quality = 0;
      for (int i = 0; i < v.Length; i++) {
        if (v[i])
          quality += 1.0;
      }

      QualityParameter.ActualValue = new DoubleValue(quality);

      return base.InstrumentedApply();
    }
  }

An evaluator is just a normal HL operator that implements at least either ISingleObjectiveEvaluator or IMultiObjectiveEvaluator. This is important as an evaluator must have a QualityParameter that is used for returning the result. Normally, as evaluators are problem-specific, there are problem-specific interfaces and may also base-classes. So in this example, we have the IOneMaxEvaluator. But if you have e.g. a look at the test-functions problem, you can see that there is a base-class (SingleObjectiveTestFunctionProblemEvaluator) that you should use. To find out which interface/base-class to use you have to look at the source of the problem you want to work with.

Operators in HL normally inherit from InstrumentedOperator to give users the ability to add other operators to an operator for pre- and post processing tasks.

The evaluator has two parameters: The before-mentioned Quality parameter and the BinaryVector parameter. The second one is again problem specific. If you think again of test functions, this would not be of type BinaryVector but RealVector. So these are the names how the parameters are called. Now how the solutions are really called in the scopes is set by the problem and may be different for every problem. Normally, the problem looks up all operators and tells them the real (actual) name. You have to be cautions here as this may not work for operators that you add after you created a new problem in HL, so always check if the parameters of your evaluator are correctly named an set and otherwise set them yourself. If you want to find out for a problem how it calls solutions, you can just select the solution creator and click the "Show hidden parameters" button. So e.g. in the test-function problem the solution is called "Point", in the TSP it's called "TSPTour" and in OneMax the solution is called "OneMaxSolution" and the Quality value "NumberOfOnes".

Evaluators get applied to scopes by the algorithm. All HL operators work on scopes. Scopes are just containers for data (e.g. the quality value and solution vector) and are organized in an hierarchical manner. And parameters are used to access the values in these scopes. You can use e.g. the Debug Engine and check "Breakpoint" on the evaluator parameter to get a visualization of how this looks like:

Here you can see all the scopes, each containing the solution and, if they were already evaluated, the quality value named as discussed before. So if the evaluator was configured with the correct names, it can look up the solution (first line in InstrumentedApply()) through the BinaryVector parameter, compute the quality value and return it by setting the quality parameter.