Skip to content

Submission Docker Container Layout

Jump to bottom
Tyler edited this page Dec 20, 2019 · 8 revisions

Overview:

When you make a submission to Codalab, the information and file are saved to the database. Afterwards a Celery task gets sent to the compute worker (Default or a compute worker attached to a custom queue). From there the compute worker spins up another docker container with either the default docker image for submissions, or a custom supplied one from the organizer.

Site Worker

The site worker can be thought of exactly how it sounds. It's a worker for the site to handle different Celery tasks and other miscellaneous functions. This is what is responsible for creating competition dumps, unpacking competitions, firing off the tasks for re-running submissions, etc.

Compute Worker

A compute worker is a worker that works with submissions generally. We do not really use compute workers for any other type of processing.

Submission Container

The submission container is the container that participant's submissions get ran in. The image used to create this container is either the codalab competitions default if the organizer's haven't specified an image, or if they did specify an image, that one is used instead.

This container is also created with some specific directories, some of which are only available at specific steps of scoring.

Here are the following directories:

  • /app/input_data: Where input data will be (Only exists if input data is supplied for the task)
  • /app/output: Where all submission output should go (Should always exist)
  • /app/ingestion_program: Where the ingestion program files should exist (Only available in ingestion)
  • /app/program: Where the scoring program and or the ingestion program should be located (Should always exist)
  • /app/input: Where any input for this step should be. (IE: Previous predictions from ingestion for scoring. Only exists on scoring step)
  • /app/input/ref: Where the reference data should live (Not available to submissions. Only available on scoring step.)
  • /app/input/res: Where predictions/output from the prediction step should live. (Only available on scoring step)
  • /app/shared: Where any data that needs to be shared between the ingestion program and submission should exist. (Only available on scoring steps.)
  • /app/ingested_program: Where submission code should live if it is a code submission. (Only available in ingestion)
Clone this wiki locally