Skip to content

Commit c71b379

Browse files
committed
Hopefully this fixes MLM
1 parent ef205dd commit c71b379

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

megatron/data/mlm_dataset.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -330,7 +330,7 @@ def __init__(
330330
if hasattr(args, "encoder_seq_length") and args.encoder_seq_length is not None:
331331
# T5 style
332332
assert self.inputs_length == args.encoder_seq_length
333-
assert self.targets_length == args.decoder_seq_length
333+
assert self.targets_length == args.decoder_seq_length + 1
334334
else:
335335
assert self.inputs_length + self.targets_length == args.seq_length
336336

0 commit comments

Comments
 (0)