Skip to content

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented Jul 16, 2025

This PR addresses typing compatibility issues in azure-ai-evaluation that prevent successful type checking with mypy version 1.14.1, which will be merged on 2025-07-14.

Issues Fixed

The following mypy errors have been resolved:

1. Type Annotations (var-annotated errors)

  • Added explicit type annotations for risk_assessment variables in _red_team_result.py
  • Added type annotations for collection variables in multiple files:
    • validated_prompts and valid_prompts_by_category in _attack_objective_generator.py
    • session_state in _conversation.py
    • tasks in _adversarial_simulator.py
    • evaluators_dict in _safety_evaluation.py

2. Return Type Compatibility (return-value errors)

  • Fixed to_eval_qr_json_lines() method return type from str to List[str]
  • Updated empty return value to return [] instead of ""

3. Assignment Compatibility (assignment errors)

  • Fixed evaluator assignment issues in _get_evaluators() method by adding proper type annotation

4. List Type Casting (arg-type errors)

  • Converted integer lists to float lists for list_mean_nan_safe() function calls in safety evaluation

Files Changed

  • azure/ai/evaluation/red_team/_red_team_result.py
  • azure/ai/evaluation/_safety_evaluation/_safety_evaluation.py
  • azure/ai/evaluation/red_team/_attack_objective_generator.py
  • azure/ai/evaluation/simulator/_conversation/_conversation.py
  • azure/ai/evaluation/simulator/_adversarial_simulator.py

Verification

All original mypy 1.14.1 compatibility errors mentioned in the issue have been successfully resolved. The changes are minimal and surgical, focusing only on type safety improvements without altering functionality.

Example of fixes applied:

# Before: Missing type annotation
risk_assessment = conversation.get("risk_assessment", {})

# After: Explicit type annotation
risk_assessment: Any = conversation.get("risk_assessment", {})
# Before: Incorrect return type
def to_eval_qr_json_lines(self) -> str:
    # ... returns a list

# After: Correct return type
def to_eval_qr_json_lines(self) -> List[str]:
    # ... returns a list

Fixes #40620.


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

@Copilot Copilot AI changed the title [WIP] azure-ai-evaluation needs typing updates for mypy version 1.14.1 Fix azure-ai-evaluation typing issues for mypy version 1.14.1 compatibility Jul 16, 2025
@Copilot Copilot AI requested a review from luigiw July 16, 2025 21:49
Copilot finished work on behalf of luigiw July 16, 2025 21:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

azure-ai-evaluation needs typing updates for mypy version 1.14.1
2 participants