fix: _get_offset_tokenizer immune to global fastokens patch (concurrent-pool race)#86

hallerite merged 2 commits into

mainPrimeIntellect-ai/renderers:mainfrom

fix/offset-tokenizer-fastokens-racePrimeIntellect-ai/renderers:fix/offset-tokenizer-fastokens-raceCopy head branch name to clipboard