WIP: Try to use monarch to run torchtitan.

[ahmadsharif1]

Repro:

source dev/xl_conda.sh activate xlformers_llama4_flagship_conda:latest_conveyor_build
git clone https://github.com/pytorch/torchtitan.git
cd torchtitan
pip install -r dependencies.txt

python scripts/download_tokenizer.py --repo_id meta-llama/Meta-Llama-3.1-8B --tokenizer_path "original" --hf_token=$HF_TOKEN

GROUP_WORLD_SIZE=1 WORLD_SIZE=2 python -m torchtitan.train_monarch \
--job.config_file \
./torchtitan/models/llama3/train_configs/llama3_8b.toml

Currently there is an error:

KeyError: "Invalid mesh_dim_names ('dp_cp',) specified. Valid mesh_dim_names are ['dp_shard']."