Removed collate_fn for samsum dataset as DataCollatorForSeq2Seq works for all the use cases supported.

quic-meetkuma · quic-meetkuma · commit d5319e15b8d1 · 2025-04-15T18:05:20.000+05:30
Signed-off-by: Meet Patel &lt;quic_meetkuma@quicinc.com&gt;
diff --git a/QEfficient/finetune/dataset/dataset_config.py b/QEfficient/finetune/dataset/dataset_config.py
@@ -38,5 +38,4 @@
 }
 DATALOADER_COLLATE_FUNC = {
     "custom_dataset": get_data_collator,
-    "samsum_dataset": get_samsum_collate_fn,
 }

Original file line number	Diff line number	Diff line change
`@@ -38,5 +38,4 @@`
`38`	`38`	`}`
`39`	`39`	`DATALOADER_COLLATE_FUNC = {`
`40`	`40`	`"custom_dataset": get_data_collator,`
`41`		`- "samsum_dataset": get_samsum_collate_fn,`
`42`	`41`	`}`