#2923 closed enhancement (done)

Penn ML Benchmark Problem Instances

Reported by: bburlacu Owned by: gkronber
Priority: medium Milestone: HeuristicLab 3.3.16
Component: Problems.Instances Version: trunk
Keywords: Cc:

Description (last modified by bburlacu)

The EpistasisLab at the University of Pennsylvania maintains a curated set of benchmark datasets for classification and regression [1].

A large part of the regression datasets have already been used for an in-depth comparison of symbolic regression methods [2].

This ticket will track integration of these benchmark problems in HeuristicLab via a new problem instance provider.

[1] https://github.com/EpistasisLab/penn-ml-benchmarks

[2] https://arxiv.org/abs/1804.09331

Attachments (3)

PennML GBT Results.xlsx (34.5 KB) - added by gkronber 14 months ago.
PennML - GBT Experiment.hl (9.0 MB) - added by gkronber 14 months ago.
PennML RF Results.xlsx (54.8 KB) - added by bburlacu 14 months ago.
Random Forest results

Change History (11)

comment:1 Changed 15 months ago by bburlacu

  • Description modified (diff)
  • Status changed from new to accepted

comment:2 Changed 15 months ago by bburlacu

  • Owner changed from bburlacu to gkronber
  • Status changed from accepted to reviewing

r15951: Add PennML problems and implement instance provider. A reusable descriptor was also implemented, taking advantage of the structure of the data (by convention, target is named "target" and is always the last column). The instance provider generates the descriptors on the fly from the archive.

comment:3 Changed 15 months ago by bburlacu

r15952: Fix namespace.

comment:4 Changed 14 months ago by gkronber

I made a quick test using our GBT implementation with standard settings and 3000 iterations.

See the two attachments.

Changed 14 months ago by gkronber

Changed 14 months ago by gkronber

Changed 14 months ago by bburlacu

Random Forest results

comment:5 Changed 11 months ago by gkronber

  • Status changed from reviewing to readytorelease

Reviewed r15951 and r15952. Thanks!

comment:6 Changed 11 months ago by gkronber

r16163: merged r15951, r15952 from trunk to stable

comment:7 Changed 11 months ago by gkronber

r15951 depends on r15855 (#2908)

comment:8 Changed 11 months ago by gkronber

  • Resolution set to done
  • Status changed from readytorelease to closed
Note: See TracTickets for help on using tickets.