Skip to content

[BUG] Misleading metric type of LightevalTaskConfig #710

Open
@wangtsuh

Description

@wangtsuh

Describe the bug

In the documentation and function LightevalTaskConfig, the metric argument has denoted list[str] as the type annotation. However, in the actual code, we can see that the expected type should be ListLike[Metric | Metrics]. The incorrect type annotation is misleading and causes users hard to know what list of string should be passed.

Version info

0.9.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdocumentationImprovements or additions to documentationgood first issueGood for newcomershelp wantedExtra attention is needed

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions