Skip to content

Issues: UKGovernmentBEIS/inspect_ai

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

48 Open 329 Closed
48 Open 329 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

CUDA memory leak after closing APIs for local models
#1543 opened Mar 24, 2025 by adamdejl
8
Improve handling of pathlib.Path objects for read_eval_log
#1542 opened Mar 24, 2025 by nix-apollo
1
Bug: Logprobs returned as float when using deepseek-r1 via openrouter
#1540 opened Mar 23, 2025 by MattFisher
3
scorers should not take up two lines in the header
#1533 opened Mar 23, 2025 by JasonGross
@dragonstyle
2
hendrycks/competition_math has been removed from huggingface
#1526 opened Mar 22, 2025 by federicoandreis
agent protocol (handoffs, improved substitutability / reduced coupling)
#1496 opened Mar 17, 2025 by jjallaire
JS Console Error in inspect view: TypeError: Cannot read properties of undefined (reading 'replace') upstream
#1487 opened Mar 15, 2025 by JasonGross
@dragonstyle
1
basic_agent initialise infinite tools when being wrapped in an orchestrator for multi-agent evals
#1471 opened Mar 13, 2025 by eaguaida
4
Feature Request: support for arbitrary provider adhering to known API format
#1467 opened Mar 12, 2025 by nathanneuro
6
Type of sample_id in eval is incompatible with lists of strings
#1465 opened Mar 12, 2025 by devon-research
4
Add line numbers to python and other code
#1464 opened Mar 12, 2025 by dragonstyle
@dragonstyle
Support passing file-like objects (StringIO, BytesIO) to read_eval_log and similar functions
#1443 opened Mar 6, 2025 by tadamcz
O1PreviewChatAPIHandler.input_with_tools breaks message context
#1440 opened Mar 6, 2025 by alexb-palisaderesearch
4
inspect view does not know how to render tool calls with arguments of type dict[str, str]
#1387 opened Feb 25, 2025 by JasonGross
@dragonstyle
Filter by sample metadata in log viewer
#1378 opened Feb 24, 2025 by tadamcz
Search message contents in log viewer
#1377 opened Feb 24, 2025 by tadamcz
The "Value" type in score is overly restrictive for dictonaries
#1333 opened Feb 17, 2025 by max-kaufmann
@dragonstyle
4
Side by side comparison of two models
#1327 opened Feb 16, 2025 by yassersouri
@dragonstyle
5
Scorers which don't return a numeric Score cause crashes in the log viewer
#1324 opened Feb 15, 2025 by max-kaufmann
2
[feature request] progress of forked solvers / subtasks are not visible in terminal viewer
#1296 opened Feb 12, 2025 by skinnerjc
2
[feature request] set default cache policy
#1292 opened Feb 11, 2025 by kaifronsdal
1
MATH dataset used for mathematics tutorial no longer accessible
#1283 opened Feb 10, 2025 by ChiWilliams
3
Custom Displays
#1267 opened Feb 7, 2025 by Baidicoot
1
Log viewer: scroll bar resets when using the scorer drop-down
#1220 opened Jan 30, 2025 by tadamcz
@dragonstyle
[Feature request] Support for LiteLLM and LiteLLM proxy
#1213 opened Jan 30, 2025 by danrr
ProTip! Follow long discussions with comments:>50.