Skip to content

Added finetuning support for BERT based models on IMDB dataset. #292

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 16 commits into from
Apr 17, 2025
Merged

Added finetuning support for BERT based models on IMDB dataset. #292

merged 16 commits into from
Apr 17, 2025

Conversation

quic-meetkuma
Copy link
Contributor

@quic-meetkuma quic-meetkuma commented Feb 28, 2025 •
edited
Loading

  • Added support for SequenceClassification task for BERT based models.
  • Added support for IMDB dataset.
  • Introduced a new flag --task_type with default value as "generation". For BERT training, it can be set to "seq_classification".

@vbaddi vbaddi marked this pull request as draft February 28, 2025 10:43
@quic-meetkuma quic-meetkuma marked this pull request as ready for review March 28, 2025 09:10
quic-meetkuma
quic-meetkuma commented Mar 28, 2025
View reviewed changes
Copy link
Contributor Author

@quic-meetkuma quic-meetkuma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Address minor comments along with the other reviewers' comments.

@quic-meetkuma quic-meetkuma force-pushed the bert_ft branch 9 times, most recently from 1911bc4 to 016331c Compare April 9, 2025 11:16
@quic-meetkuma quic-meetkuma force-pushed the bert_ft branch 2 times, most recently from e4c5045 to 977f881 Compare April 14, 2025 05:42
@quic-meetkuma quic-meetkuma force-pushed the bert_ft branch 2 times, most recently from e682111 to ad3c27f Compare April 15, 2025 13:06
quic-mamta
quic-mamta requested changes Apr 16, 2025
View reviewed changes
@quic-mamta quic-mamta requested a review from vbaddi April 16, 2025 05:46
@quic-meetkuma quic-meetkuma force-pushed the bert_ft branch 3 times, most recently from 3174547 to 8d201d0 Compare April 17, 2025 05:51
@quic-meetkuma
Added finetuning support for BERT based models on IMDB dataset.
214e85c 
Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Added finetuning support for BERT based models on IMDB dataset.
390bce8 
Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Added finetuning support for BERT based models on IMDB dataset.
a9637b1 
Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Added torchmetrics as dependency and fixed loss computation for ddp c…
2a09de2 
…ase.

Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Fixed atol value.
41766dd 
Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Fixed collate fn for bs>1. It will work fine for bs>1 for llama as we…
74ede95 
…ll on single device.

Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Fixed train accuracy computation
1047357 
Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Fixed formatting issues.
4e21d3b 
Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Fixed few comments. Need to rebase first and then test
5c154aa 
Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Removed collate_fn for samsum dataset as DataCollatorForSeq2Seq works…
e97fed8 
… for all the use cases supported.

Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Ruff formatted code.
b3c707e 
Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Removed samsum collate_fn as it is dead code.
8d983f2 
Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Refactored evaluation fun and renamed variable names to generic names.
bf0f083 
Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Fixed metric value for json logging
9ef972f 
Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Minor fixes to variable names
2713fdc 
Signed-off-by: Meet Patel <[email protected]>
@quic-meetkuma
Addressed few offline comments.
b6d0ce9 
Signed-off-by: Meet Patel <[email protected]>
@quic-swatia quic-swatia self-requested a review April 17, 2025 11:49
@quic-mamta quic-mamta merged commit b1fb57d into quic:main Apr 17, 2025
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@quic-mamta quic-mamta quic-mamta approved these changes

@quic-swatia quic-swatia quic-swatia approved these changes

@quic-rishinr quic-rishinr Awaiting requested review from quic-rishinr quic-rishinr is a code owner

@ochougul ochougul Awaiting requested review from ochougul ochougul is a code owner

@vbaddi vbaddi Awaiting requested review from vbaddi

Assignees

@quic-meetkuma quic-meetkuma

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@quic-meetkuma @quic-mamta @quic-swatia