Yourbench Stable 0.9

What's Changed

• code style fix by @sumukshashidhar in #169
• Fix failing test in test_configuration_engine.py by @sumukshashidhar in #171
• Add various examples and prompts by @sumukshashidhar in #170
• Add CLI support for base_url and model configuration options by @sumukshashidhar in #172
• Minor changes docs and examples by @patrickfleith in #173
• Tutorial with harry potter wiki plots by @patrickfleith in #177
• Adding comprehensive dataset columns documentation by @patrickfleith in #180
• Bug/174 model card verbosity and missing elements by @patrickfleith in #176
• Enable Extra Parameters in the LLM configuration by @sumukshashidhar in #182
• make push_to_hub configurable by @lzamparo in #184
• Simplify configuration system with OmegaConf schema defaults by @sumukshashidhar in #185
• Bugfix: Resolved question rewriting pipeline issues by @patrickfleith in #181
• fix: handle UTF-8 text files correctly in ingestion (#186) by @sumukshashidhar in #188
• refactor: remove analysis module, web UI, and simplify inference layer by @sumukshashidhar in #191
• feat: enhanced logging with structured context and file output by @sumukshashidhar in #192
• refactor: split dataset_engine.py into smaller modules by @sumukshashidhar in #193
• simplification effort by @sumukshashidhar in #195
• feat: add custom Pydantic question schema support by @sumukshashidhar in #196
• refactor: clean up code smells and fix minor issues by @sumukshashidhar in #197
• Release v0.9.0: CLI improvements, single_hop rename, and custom schema demo by @sumukshashidhar in #198

New Contributors

• @lzamparo made their first contribution in #184

Full Changelog: v0.6...v0.9

v0.6

What's Changed

• Update configuration engine by @sumukshashidhar in #168

Full Changelog: v0.5.3...v0.6

v0.5.3

What's Changed

• Update configuration_engine.py by @sumukshashidhar in #167

Full Changelog: v0.5.2...v0.5.3

v0.5.2

What's Changed

• Hotfix api by @sumukshashidhar in #166

Full Changelog: v0.5.1...v0.5.2

v0.5.1

What's Changed

• Quickfix pathnames and env variable checks by @sumukshashidhar in #165

Full Changelog: v0.5.0...v0.5.1

v0.5.0

What's Changed

• don't overwrite base url by @sumukshashidhar in #163
• Local Dataset JSONL exports by @sumukshashidhar in #164

Full Changelog: v0.4.3...v0.5.0

v0.4.3

What's Changed

• Fix path resolution for --docs ./ parameter by @sumukshashidhar in #158
• remove additional instructions for quick mode by @sumukshashidhar in #159
• Version bump to 0.4.2 by @sumukshashidhar in #160
• Clean up dataset card insertions by @sumukshashidhar in #162
• Refactor: Simplify CLI interface by @sumukshashidhar in #161

Full Changelog: v0.4.1...v0.4.3

v0.4.1

What's Changed

• add new readme, remove legacy figure by @sumukshashidhar in #99
• Pipeline integration tests by @sumukshashidhar in #88
• Update README.md by @sumukshashidhar in #100
• Delete yourbench/utils/load_task_config.py by @sumukshashidhar in #103
• Analyze sample questions by @sumukshashidhar in #107
• Update quickstart with correct run command by @patrickfleith in #108
• fix summarization and refactor by @sumukshashidhar in #106
• refactor loading engine by @sumukshashidhar in #104
• Refactor chunking by @sumukshashidhar in #112
• Long task stability by @alozowski in #111
• Fix: Ensure summarization uses correct model by aligning step name by @alozowski in #117
• Improve Ingestion for Web Documents by @sumukshashidhar in #116
• Fix citation score filtering by @sumukshashidhar in #115
• Refactor question generation and inference by @alozowski in #121
• Potential fix for code scanning alert no. 1: Workflow does not contain permissions by @sumukshashidhar in #101
• overhaul dataset engine by @sumukshashidhar in #118
• utf-8 encoding of advanced sample yaml file in windows by @praveenmec67 in #128
• Add support for cross-document question generation in multi-hop pipeline by @alozowski in #129
• Expanded example instructions for self-hoseted non /v1 endpoints like OpenWebUI which uses /api by @mmajurski in #126
• add simple configuration generator by @sumukshashidhar in #119
• refactor citation score filtering by @sumukshashidhar in #105
• Pushing an automatically generated readme datacard with yourbench dataset by @patrickfleith in #125
• Add Deduplication for Generated Questions by @alozowski in #130
• Add llm ingestion and consolidate upload_ingest_to_hub by @sumukshashidhar in #133
• fix viewer by @sumukshashidhar in #132
• Add question re-writing stage by @sumukshashidhar in #127
• improve multi-document chunking by @sumukshashidhar in #131
• Fix local saving by @alozowski in #136
• fix #138 by @sumukshashidhar in #140
• docs/clarified-contribution-guidelines by @patrickfleith in #139
• Add API names to .env.template by @alozowski in #141
• Gradio integration by @alozowski in #137
• Add prompts folder by @sumukshashidhar in #143
• add stargazers and clear up ambiguity in main readme by @sumukshashidhar in #145
• Add new config tests by @sumukshashidhar in #148
• Add config builder by @sumukshashidhar in #146
• Configuration engine update by @sumukshashidhar in #147
• Add predetermined configs and examples by @sumukshashidhar in #144
• Fix code quality issues by @sumukshashidhar in #150
• Add quick mode CLI with --model and --docs arguments by @sumukshashidhar in #155
• fix prompt use by @sumukshashidhar in #157
• add better readme and add additional instructions for uvx by @sumukshashidhar in #156
• Add JSONL export functionality by @sumukshashidhar in #153
• Fix missing dataset template in PyPI package by @sumukshashidhar in #152
• Optimize CLI startup performance with lazy imports by @sumukshashidhar in #151

New Contributors

• @patrickfleith made their first contribution in #108
• @praveenmec67 made their first contribution in #128
• @mmajurski made their first contribution in #126

Full Changelog: v0.3.1...v0.4.1

v0.4.0-alpha

What's Changed

• add new readme, remove legacy figure by @sumukshashidhar in #99
• Pipeline integration tests by @sumukshashidhar in #88
• Update README.md by @sumukshashidhar in #100
• Delete yourbench/utils/load_task_config.py by @sumukshashidhar in #103
• Analyze sample questions by @sumukshashidhar in #107
• Update quickstart with correct run command by @patrickfleith in #108
• fix summarization and refactor by @sumukshashidhar in #106
• refactor loading engine by @sumukshashidhar in #104
• Refactor chunking by @sumukshashidhar in #112
• Long task stability by @alozowski in #111
• Fix: Ensure summarization uses correct model by aligning step name by @alozowski in #117
• Improve Ingestion for Web Documents by @sumukshashidhar in #116
• Fix citation score filtering by @sumukshashidhar in #115
• Refactor question generation and inference by @alozowski in #121
• Potential fix for code scanning alert no. 1: Workflow does not contain permissions by @sumukshashidhar in #101
• overhaul dataset engine by @sumukshashidhar in #118
• utf-8 encoding of advanced sample yaml file in windows by @praveenmec67 in #128
• Add support for cross-document question generation in multi-hop pipeline by @alozowski in #129
• Expanded example instructions for self-hoseted non /v1 endpoints like OpenWebUI which uses /api by @mmajurski in #126
• add simple configuration generator by @sumukshashidhar in #119
• refactor citation score filtering by @sumukshashidhar in #105
• Pushing an automatically generated readme datacard with yourbench dataset by @patrickfleith in #125
• Add Deduplication for Generated Questions by @alozowski in #130
• Add llm ingestion and consolidate upload_ingest_to_hub by @sumukshashidhar in #133
• fix viewer by @sumukshashidhar in #132
• Add question re-writing stage by @sumukshashidhar in #127
• improve multi-document chunking by @sumukshashidhar in #131
• Fix local saving by @alozowski in #136
• fix #138 by @sumukshashidhar in #140
• docs/clarified-contribution-guidelines by @patrickfleith in #139
• Add API names to .env.template by @alozowski in #141
• Gradio integration by @alozowski in #137
• Add prompts folder by @sumukshashidhar in #143
• add stargazers and clear up ambiguity in main readme by @sumukshashidhar in #145
• Add new config tests by @sumukshashidhar in #148
• Add config builder by @sumukshashidhar in #146
• Configuration engine update by @sumukshashidhar in #147
• Add predetermined configs and examples by @sumukshashidhar in #144

New Contributors

• @patrickfleith made their first contribution in #108
• @praveenmec67 made their first contribution in #128
• @mmajurski made their first contribution in #126

Full Changelog: v0.3.1...v0.4.0-alpha

Special Shoutout to @patrickfleith for their contributions!

v0.3.1

What's Changed

• add bill_to parameter and dependencies by @sumukshashidhar in #83
• Respect run flag in pipeline stages by @alozowski in #84
• Add Support for Multi-Choice and Open-Ended Question Types by @alozowski in #86
• Fixed local datasets loading/saving by @stepdi in #81
• fix push to hub by @sumukshashidhar in #91
• feat: add cost tracking (clean PR) by @sumukshashidhar in #90
• Add offline mode support (cherry-pick) by @alozowski in #93
• v0.3.1 release PR by @sumukshashidhar in #92

New Contributors

• @stepdi made their first contribution in #81

Full Changelog: v0.3...v0.3.1