Fix outlines task1706: pin transformers and tokenizers versions#39
Merged
akhatua2 merged 1 commit intocooperbench:mainfrom Mar 12, 2026
Merged
Conversation
The Dockerfile and runner.sh install transformers and tokenizers without version pins. transformers>=5.0.0 combined with tokenizers>=0.22 breaks the test suite in two ways: 1. tokenizers 0.22 turns the BPE.__init__ DeprecationWarning into a hard error, causing GPT2Tokenizer initialization to fail. 2. transformers 5.x has a code path in _from_pretrained that calls import_protobuf_decode_error() inside an except clause, which raises ImportError when protobuf is not installed — even for non-protobuf tokenizers like GPT2. Pin transformers<5 and tokenizers<0.21 in both the Dockerfile and runner.sh to restore compatibility. Verified locally: both feature 4 and feature 6 tests pass (9/9 and 11/11) with the pinned versions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Collaborator
|
Thanks Kevin! This looks great to me. |
Contributor
Author
|
@akhatua2 do we need to rebuild and reupload akhatua/cooperbench-dottxt-ai-outlines:task1706? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The outlines task1706 Dockerfile and runner.sh install
transformersandtokenizerswithout version pins. With the latest releases (transformers>=5.0.0,tokenizers>=0.22), all tests that use themodel_transformersfixture fail:tokenizers>=0.22turns theBPE.__init__DeprecationWarninginto a hard error, causingGPT2Tokenizerinitialization to failtransformers>=5.0.0has a code path in_from_pretrainedthat callsimport_protobuf_decode_error()inside anexceptclause, which raisesImportErrorwhen protobuf is not installed — even for non-protobuf tokenizers like GPT2The pre-built Docker image (
akhatua/cooperbench-dottxt-ai-outlines:task1706) ships withtransformers==5.0.0+tokenizers==0.22.2and is affected by both issues — 8 of 9 tests error withDeprecationWarning/ImportErroron every run.Fix
Pin
transformers<5andtokenizers<0.21in both the Dockerfile and runner.sh.Verification
Built the fixed image locally and ran the full test suite against the oracle
combined.patch:The pre-built image also needs to be rebuilt with these pins.
Test plan
akhatua/cooperbench-dottxt-ai-outlines:task1706) reproduces the failure → 1 passed, 8 errors