Search results

Llama4 training does not automatically use bfloat16 when FSDP2 is enabled

bugSomething isn't workingSomething isn't working

Status: Open.

Bug

#1332 In pytorch/torchtitan;

· danielvegamyhre opened on Jun 24, 2025

Error in backward pass when importing and using Llama4 MoE layer

bugSomething isn't workingSomething isn't working

Status: Open.

Bug

#1265 In pytorch/torchtitan;

· danielvegamyhre opened on Jun 5, 2025

float8 rowwise vanilla TP low throughput

bugSomething isn't workingSomething isn't working

module: float8

Status: Open.

#1207 In pytorch/torchtitan;

· danielvegamyhre opened on May 20, 2025

debug model training hangs on NVIDIA B200 with >1 GPU

bugSomething isn't workingSomething isn't working

module: c10d

Status: Open.

#810 In pytorch/torchtitan;

· vkuzo opened on Jan 28, 2025

unable to run 8b llama

bugSomething isn't workingSomething isn't working

Status: Open.

#807 In pytorch/torchtitan;

· asahni-sc opened on Jan 27, 2025

[Bug] Unexpected performance drop with float8 training + compiling only nn.Linear layers + using selective per op AC

bugSomething isn't workingSomething isn't working

Status: Open.

#786 In pytorch/torchtitan;

· danielvegamyhre opened on Jan 10, 2025

BUG: early_step_in_backward with pipeline parallelism and len(model_parts) > 1

bugSomething isn't workingSomething isn't working

Status: Open.

#777 In pytorch/torchtitan;

· cassanof opened on Jan 7, 2025

PP hangs when pipeline_parallel_microbatches < pipeline_parallel_degree

bugSomething isn't workingSomething isn't working

module: pipelining

Status: Open.

#775 In pytorch/torchtitan;

· cassanof opened on Jan 6, 2025

Model init with HuggingFace model

bugSomething isn't workingSomething isn't working

huggingface integration

module: checkpoint

questionFurther information is requestedFurther information is requested

Status: Open.

#743 In pytorch/torchtitan;

· neeldani opened on Dec 16, 2024

Low bit Optimizers & FA-3

bugSomething isn't workingSomething isn't working

questionFurther information is requestedFurther information is requested

Status: Open.

#742 In pytorch/torchtitan;

· asahni-sc opened on Dec 16, 2024

2D whole model compile fails at embedding layer

bugSomething isn't workingSomething isn't working

Status: Open.

#534 In pytorch/torchtitan;

· tianyu-l opened on Aug 20, 2024

numerical difference for SDPA between non-dtensor vs dtensor, when math attention and fp16 are used

bugSomething isn't workingSomething isn't working

Status: Open.

#317 In pytorch/torchtitan;

· tianyu-l opened on May 8, 2024