Skip to content

Updating Documentation and config #51

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Updating Documentation and config #51

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7ad86e8
Updating Documentation and config
depksingh Jun 19, 2025
cafdd51
Removed repo name from internal scripts
depksingh Jun 20, 2025
772b27b
Updated current dir ref
depksingh Jun 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions recml/inference/benchmarks/DLRM_DCNv2/ckpt_load_and_eval.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,24 +6,22 @@ export XLA_FLAGS=

export TPU_NAME=<TPU_NAME>
export LEARNING_RATE=0.0034
export BATCH_SIZE=135168
export BATCH_SIZE=4224
export EMBEDDING_SIZE=128
export MODEL_DIR=/tmp/
export FILE_PATTERN=gs://qinyiyan-vm/mlperf-dataset/criteo_merge_balanced_4224/train-*
export NUM_STEPS=28000
export CHECKPOINT_INTERVAL=1500
export EVAL_INTERVAL=1500
export EVAL_FILE_PATTER=gs://qinyiyan-vm/mlperf-dataset/criteo_merge_balanced_4224/eval-*
export EVAL_FILE_PATTERN=gs://qinyiyan-vm/mlperf-dataset/criteo_merge_balanced_4224/eval-*
export EVAL_STEPS=660
export MODE=eval
export EMBEDDING_THRESHOLD=21000
export LOGGING_INTERVAL=1500
export RESTORE_CHECKPOINT=true



python recml/inference/models/jax/DLRM_DCNv2/dlrm_main.py \

--learning_rate=${LEARNING_RATE} \
--batch_size=${BATCH_SIZE} \
--embedding_size=${EMBEDDING_SIZE} \
Expand Down
5 changes: 2 additions & 3 deletions recml/inference/benchmarks/DLRM_DCNv2/train_and_checkpoint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,22 +6,21 @@ export XLA_FLAGS=

export TPU_NAME=<TPU_NAME>
export LEARNING_RATE=0.0034
export BATCH_SIZE=135168
export BATCH_SIZE=4224
export EMBEDDING_SIZE=128
export MODEL_DIR=/tmp/
export FILE_PATTERN=gs://qinyiyan-vm/mlperf-dataset/criteo_merge_balanced_4224/train-*
export NUM_STEPS=28000
export CHECKPOINT_INTERVAL=1500
export EVAL_INTERVAL=1500
export EVAL_FILE_PATTER=gs://qinyiyan-vm/mlperf-dataset/criteo_merge_balanced_4224/eval-*
export EVAL_FILE_PATTERN=gs://qinyiyan-vm/mlperf-dataset/criteo_merge_balanced_4224/eval-*
export EVAL_STEPS=660
export MODE=train
export EMBEDDING_THRESHOLD=21000
export LOGGING_INTERVAL=1500
export RESTORE_CHECKPOINT=true

python recml/inference/models/jax/DLRM_DCNv2/dlrm_main.py \

--learning_rate=${LEARNING_RATE} \
--batch_size=${BATCH_SIZE} \
--embedding_size=${EMBEDDING_SIZE} \
Expand Down
4 changes: 2 additions & 2 deletions recml/inference/benchmarks/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,10 @@ gcloud alpha compute tpus tpu-vm ssh ${TPU_NAME} --project ${PROJECT} --zone ${Z
gcloud alpha compute tpus tpu-vm ssh ${TPU_NAME} --project ${PROJECT} --zone ${ZONE} --worker=all --command="pip install -U tensorflow dm-tree flax google-metrax"
```

#### Run workload
#### Make script executable and Run workload

Note: Please update the MODEL_NAME & TASK_NAME before running the below command

```
gcloud alpha compute tpus tpu-vm ssh ${TPU_NAME} --project ${PROJECT} --zone ${ZONE} --worker=all --command="TPU_NAME=${TPU_NAME} ./inference/benchmarks/<MODEL_NAME>/<TASK_NAME>"
gcloud alpha compute tpus tpu-vm ssh ${TPU_NAME} --project ${PROJECT} --zone ${ZONE} --worker=all --command="cd RecML && chmod +x ./recml/inference/benchmarks/<MODEL_NAME>/<TASK_NAME> && TPU_NAME=${TPU_NAME} ./recml/inference/benchmarks/<MODEL_NAME>/<TASK_NAME>"
```
2 changes: 1 addition & 1 deletion requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ platformdirs==4.3.7
pluggy==1.5.0
pre-commit==4.2.0
promise==2.3
protobuf==5.29.4
protobuf==4.21.12
psutil==7.0.0
pyarrow==19.0.1
pygments==2.19.1
Expand Down