Free cookie consent management tool by TermsFeed Policy Generator

Opened 6 years ago

Closed 6 years ago

#2923 closed enhancement (done)

Penn ML Benchmark Problem Instances

Reported by: bburlacu Owned by: gkronber
Priority: medium Milestone: HeuristicLab 3.3.16
Component: Problems.Instances Version: trunk
Keywords: Cc:

Description (last modified by bburlacu)

The EpistasisLab at the University of Pennsylvania maintains a curated set of benchmark datasets for classification and regression [1].

A large part of the regression datasets have already been used for an in-depth comparison of symbolic regression methods [2].

This ticket will track integration of these benchmark problems in HeuristicLab via a new problem instance provider.

[1] https://github.com/EpistasisLab/penn-ml-benchmarks

[2] https://arxiv.org/abs/1804.09331

Attachments (3)

PennML GBT Results.xlsx (34.5 KB) - added by gkronber 6 years ago.
PennML - GBT Experiment.hl (9.0 MB) - added by gkronber 6 years ago.
PennML RF Results.xlsx (54.8 KB) - added by bburlacu 6 years ago.
Random Forest results

Change History (11)

comment:1 Changed 6 years ago by bburlacu

  • Description modified (diff)
  • Status changed from new to accepted

comment:2 Changed 6 years ago by bburlacu

  • Owner changed from bburlacu to gkronber
  • Status changed from accepted to reviewing

r15951: Add PennML problems and implement instance provider. A reusable descriptor was also implemented, taking advantage of the structure of the data (by convention, target is named "target" and is always the last column). The instance provider generates the descriptors on the fly from the archive.

comment:3 Changed 6 years ago by bburlacu

r15952: Fix namespace.

comment:4 Changed 6 years ago by gkronber

I made a quick test using our GBT implementation with standard settings and 3000 iterations.

See the two attachments.

Changed 6 years ago by gkronber

Changed 6 years ago by gkronber

Changed 6 years ago by bburlacu

Random Forest results

comment:5 Changed 6 years ago by gkronber

  • Status changed from reviewing to readytorelease

Reviewed r15951 and r15952. Thanks!

comment:6 Changed 6 years ago by gkronber

r16163: merged r15951, r15952 from trunk to stable

comment:7 Changed 6 years ago by gkronber

r15951 depends on r15855 (#2908)

comment:8 Changed 6 years ago by gkronber

  • Resolution set to done
  • Status changed from readytorelease to closed
Note: See TracTickets for help on using tickets.