Skip to content

Latest commit

 

History

History
27 lines (18 loc) · 1.22 KB

README.md

File metadata and controls

27 lines (18 loc) · 1.22 KB

EvaluationEngine

Sources of the Java Evaluation Engine. The evaluation engine performs backend tasks for OpenML:

  • process_datasets: download a dataset, describe information per feature (e.g., the datatype and number of values per feature) and upload the results.
  • evaluate_run: download a dataset and its splits, run the run's predictor on this dataset, and upload the evaluation (e.g., mean absolute error).
  • extract_features_all: download a dataset, describe some features such as maximum number of distinct values of nominal features, and upload the results.
  • extract_features_simple: same as extract_features_all, but with a subset of features.

Usage

See docker directory for how to run this using docker.

Prerequisites

  • Java. It works with 11 LTS, compatibility with other versions is unknown.
  • Maven

Compile / package

mvn clean compile                 # only compile
mvn clean package -DskipTests     # package (generate jar)

Run

Generate the jar (see "Compile / package"). Then

java -jar target/EvaluationEngine-*-SNAPSHOT-jar-with-dependencies.jar -f process_dataset -id 1
java -jar target/EvaluationEngine-*-SNAPSHOT-jar-with-dependencies.jar -f evaluate_run -id 1