Skip to content

Commit cacc36b

Browse files
authored
docs: add guide for adding a new dataset and a new task (#628)
* Add guide for adding a new dataset and a new task * Format Markdown files
1 parent f8c4800 commit cacc36b

File tree

2 files changed

+17
-0
lines changed

2 files changed

+17
-0
lines changed

docs/adding-new-dataset-and-task.md

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Adding a new dataset and a new task
2+
3+
1. Add code for the new dataset [1] (e.g. `snu_sr.{cc,h}`) to `flutter/cpp/datasets` and corresponding building rule(s) in `flutter/cpp/datasets/BUILD`.
4+
2. Add corresponding code to `flutter/cpp/binary/main.cc` so we can test the dataset with CLI.
5+
3. Add a new task specific `benchmark_setting` to backends so that accelerators could be used. E.g. adding a new setting to TFLite backend by modifying `mobile_back_tflite/cpp/backend_tflite/tflite_settings_android.h`, for an example see [d23d16c](https://github.com/mlcommons/mobile_app_open/commit/d23d16c6cec110786379fa8d3a5e2b49e1b80b0e)
6+
4. Add a new task to the `flutter/assets/tasks.pbtxt` file.
7+
5. Add a new task to Flutter code part.
8+
6. Add an integration test for the new task.
9+
10+
See the [PR #574](https://github.com/mlcommons/mobile_app_open/pull/574) for an example how to add the new SNUSR dataset.
11+
12+
See the [PR #608](https://github.com/mlcommons/mobile_app_open/pull/608) for an example how to add the new super resolution task.
13+
14+
See the [issue #595](https://github.com/mlcommons/mobile_app_open/issues/595) for a complete discussion how we added the new super resolution task.
15+
16+
[1] What does "a dataset" mean? A dataset here is to implement LoadGen's Query Sample Library (QSL). Actually, we can share some common methods and the MLPerfDriver uses interface defined in `flutter/cpp/dataset.h`, all the datasets in `flutter/cpp/datasets/` now inherit the `mlperf::mobile::Dataset`.

docs/readme.md

+1
Original file line numberDiff line numberDiff line change
@@ -15,5 +15,6 @@ Refer to the [root README file](../README.md) for general overview of the repo.
1515
* [Build and Run the app](./build-and-run.md)
1616
* [Adding support for a new custom backend](./adding-custom-backend.md)
1717
* [Using custom benchmark configuration](./custom-tasks.md)
18+
* [Adding a new dataset and a new task](./adding-new-dataset-and-task.md)
1819
* [Working with Firebase](./firebase.md)
1920
* [Formal result specification](./result-spec.md)

0 commit comments

Comments
 (0)