Home

Chalab Wiki

Chalab is a tool, which helps you design data science or machine learning challenges. A step-by-step Wizard guides you through the process. When you are done, you can compile your challenge as a self-contained zip file (competition bundle) and upload it to a challenge platform. Currently, only Codalab accepts competition bundles created by Chalab. You can view a sample competition of the style you can create with Chalab.

Mini challenge tutorial

Your point of entry into Chalab is the wizard home page, which allows you to select a challenge to edit or create a new challenge. You are then led to the Wizard page allowing you to design a challenge, one step at a time! Conveniently, you may use as template challenges previously created by others (i.e. there is a lot of information already filled in that provides you with further guidance). To understand how to select or create a template, see the Profile and the Group pages.

The Chalab challenge design includes 6 steps:

Data:

Data science challenges designed with Chalab propose supervised learning tasks of CLASSIFICATION or REGRESSION. You must prepare your dataset in the AutoML challenge format, which supports data represented as feature vectors. Full and sparse ([LIBSVM-style])(http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html#f307)) formats are supported. See the data page for details. We supply several example datasets, which you can choose from in a menu, if you are not ready yet to upload your own data.

Split:

Chalab wants to split your data 3-way into a:

1. training set (labels are supplied to the participants to train their learning machine)
2. validation set
3. training set

   Metric:

   Protocol:

   Baseline:

   Documentation: