FC-73 Add XqueueViewset with xwatcher services (Feature: XQueueViewSet - Compatible API Endpoints for External Grader Integration) #287
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit updates the XQueue Migration ADR to be more accessible while maintaining critical technical information. Key changes include: - Streamline context section focusing on core system limitations - Reorganize decision section with clearer structure - Add implementation approach section for gradual migration - Maintain technical details in consequences section - Keep all original references for further documentation Part of the XQueue migration initiative. refactor: squashed migrations - Delete all migrations - Run again makemigrations - Test in secure environment refactor: change submission queue record name functionalities - Change SubmissionQueueManager to ExternalGraderDetailManager - Change create_submission_queue_record to create_external_grader_detail - Change validation to queue name in create_submission - Remove unnecessary test to check queue name error refactor(submissions): Rename SubmissionQueueRecord to ExternalGraderDetail - Rename model SubmissionQueueRecord to ExternalGraderDetail - Update all references in Python files - Create Django migration for model renaming - Update related imports and references feat: add ADR for SubmissionQueueRecord migration from XQueue Add Architecture Decision Record (ADR) documenting the design and implementation of the SubmissionQueueRecord model as part of the gradual migration from XQueue to edx-submissions. This ADR covers: - Context of current XQueue architecture - Technical details of new model implementation in edx submissions - Migration and compatibility considerations - Impact analysis and consequences feat: add architecture decisions to documentation - Create new decisions folder - Configurate in index.rst test: add validation tests for SubmissionQueueRecord state transitions - Add test_clean_invalid_transitions to verify error cases: - pending -> retired (not allowed) - pending -> invalid_status - pulled -> pending (not allowed) - failed -> retired (not allowed) - Enhance test_clean_valid_transitions with proper assertions This improves test coverage for state transition validations. fix: change validation SubmissionQueueRecord clean method - Add an optional status can_transition_to - Implement old status in clean method This resolves a bug in the implementation of transition validation when using the clean method. fix: remove try except in test_clean_new_instance - Remove unnecessary try/except in test_clean_new_instance - Add validations when new queue record call clean method - Extend coverage measure in test_models test: Add tests to extend coverage for SubmissionQueueRecord model - Status transition rules - Failure count tracking - Queue length calculations - Submission retrieval logic The new tests improve the robustness of the submissions app by covering. docs: Improve create_submission docstring formatting - Fix Sphinx documentation warnings - Correct formatting and indentation issues - Clarify parameter and return value descriptions Resolves documentation build warnings related to: - Inline strong start-string without end-string - Unexpected indentation fix: correct docstring formatting in create_submission - Add proper indentation and blank lines between sections - Fix example code block formatting - Replace >>> with ... for continued lines in example - Ensure consistent spacing throughout docstring This resolves the ReadTheDocs build failure and docstring formatting warning. fix: code style and quality improvements - Fix pylint warnings: - Remove protected access warnings in test files - Remove pointless string statements - Add missing final newlines - Fix isort import ordering in: - api.py - test_models.py - test_api.py - Fix pycodestyle issues: - Remove extra blank lines - Fix operator spacing - Add proper spacing before inline comments - Fix long lines in migration files - Fix line spacing in test files - Fix docstring formatting in api.py create_submission method Testing: make test_quality passes without warnings feat: add grading metadata fields to submission queue record - Add grader_file_name CharField (128 chars max) with empty default - Add points_possible PositiveIntegerField with default value of 1 - Update create_submission_queue_record to handle new fields fix: Remove duplicate nested create_submission function definition - Removes the redundant nested function definition feat: add nullable pullkey and grader_reply to Submission model feat(SubmissionQueueRecord): add pullkey and grader_reply fields - Add pullkey CharField(128) for tracking submission processing key - Add grader_reply TextField to store grader's response - Fields support queue processing and feedback management to xwatcher test: add comprehensive tests for Submission and SubmissionQueueRecord models Add missing test suite for Submission model and add new tests for SubmissionQueueRecord: Submission model tests (previously non-existent): - Basic model functionality (creation, retrieval) - String and repr representations - Large answer handling - Field mutability behavior - Submission ordering - Soft deletion mechanism - JSON serialization of answers SubmissionQueueRecord model tests: - Default status and initialization - Valid and invalid status transitions - Failure count tracking - Record processability logic - Queue name validation - One-to-one relationship with Submission - Status time updates Updates: - Fix repr test to handle UUID and string conversions - Document actual mutability behavior - Add validation for large answers - Ensure proper status transitions and timing - Verify queue processing logic The test suite ensures both models work correctly together and maintain data integrity through their lifecycle. Note: This commit adds the first test coverage for the Submission model, which previously lacked any automated testing. refactor: update submission queue record tests - Update test cases to use new public create_submission_queue_record function - Update assertions to use SubmissionQueueCanNotBeEmptyError instead of ValueError - Adjust test function parameters to match new interface (queue_name and files) - Add new test cases for empty queue_name validation The changes ensure test coverage for: - Direct queue record creation with valid and invalid inputs - Integration with create_submission function - Multiple submission queue records - Error handling scenarios fix: add blank line in models - This fixed the pylint warning feat(error): add SubmissionQueueCanNotBeEmptyError class error - This class is to is raised when queue name is empty fix: improve submission queue record validation - Update condition for creating submission queue record to properly check for queue_name - Use `event_data.get()` for safer dictionary access - Change ValueError to SubmissionQueueCanNotBeEmptyError for more specific error handling - Add proper validation in parent function to prevent None access errors The changes ensure that: 1. Queue name validation is more robust 2. Error handling is more specific and clear 3. Null checks are properly implemented Test coverage remains unchanged. refactor(submission): simplify queue record creation with explicit field - Replace dynamic unpacking of event_data with explicit queue_name field - Remove unnecessary field expansion to improve code clarity and maintainability test: add queue record unit and integration tests - Add unit tests for _create_submission_queue_record helper function - Basic queue record creation - Queue name validation - Database error handling - Add integration tests with create_submission - Queue record creation via submission creation - Multiple queue records with same name - Error handling with missing queue name - Database error integration testing Organized tests into clearly separated unit and integration sections for better maintainability and clarity. feat: add submission queue record functionality - Add _create_submission_queue_record helper function to handle queue record creation - Modify create_submission to accept event_data as kwargs - Update documentation to reflect new parameters and functionality - Allow dynamic field assignment using event_data kwargs in queue record creation The main changes: - Added ability to create SubmissionQueueRecord with dynamic fields - Improved error handling for queue record creation - Made event_data more flexible using kwargs pattern - Maintained backwards compatibility with existing submission creation feat(submissions): add SubmissionQueueRecord model - Add SubmissionQueueManager to process queries - Replace multiple timestamps with a single status_time field - Introduce explicit state machine for submission processing - Add atomic state transitions with validation - Optimize database indexes for queue operations Key changes: - Add STATUS_CHOICES and VALID_TRANSITIONS for state management - Consolidate pull_time, push_time, return_time into status_time - Add transaction-safe update_status() method - Improve queue manager with safer submission retrieval - Add validation for state transitions - Optimize indexes for common query patterns The changes maintain backwards compatibility with existing queue processing while providing a more robust and maintainable approach to state management.
Add extensive test suite covering edge cases and error handling - Test file processing with different input types (bytes, file objects) - Implement error handling tests for IO errors and invalid files - Verify complete file processing workflow feat: implement submission file workflow - Create Submission file model - Develop SubmissionFileManager for centralized DB query handling - Add comprehensive test suite for submission file actions - Implement validation tests for file addition in create_external_grader_detail - Document architectural decisions with ADR - Add test queue folder in .gitignore
- Remove unused import statements from refactored modules
- Delete redundant migrations causing conflicts during rebase
- Fix import ordering to follow project style guidelines
- Ensure all dependencies are properly declared
refactor: remove unnecessary print sentence
feat (docs): add Xqueue viewset implementation ADR
test(coverage): Expand test suite for ViewSets
- Increase unit test coverage for ViewSet classes
- Add comprehensive test scenarios for edge cases
- Enhance validation and error handling test scenarios for dispatch method
fix(lint): Resolve Pylint warnings across multiple files
- Fix code style and quality issues identified by Pylint
- Improve overall code consistency and readability
- Address static code analysis warnings
- Enhance code maintainability
refactor(session): Enhance session management in XqueueViewSet
- Implement robust cookie parsing method
- Improve session loading and validation workflow
- Add detailed logging for session-related events
- Handle edge cases in session key and user authentication
Merge XQU-15-add-fields-to-submission-mode in XQU-31-extend-coverage-in-file-mode
refactor(submissions): Rename SubmissionQueueRecord to ExternalGraderDetail
- Rename model SubmissionQueueRecord to ExternalGraderDetail
- Update all references in Python files
- Create Django migration for model renaming
- Update related imports and references
fix: code style and quality improvements
- Fix pylint warnings:
- Remove protected access warnings in test files
- Remove pointless string statements
- Add missing final newlines
- Fix isort import ordering in:
- api.py
- test_models.py
- test_api.py
- Fix pycodestyle issues:
- Remove extra blank lines
- Fix operator spacing
- Add proper spacing before inline comments
- Fix long lines in migration files
- Fix line spacing in test files
- Fix docstring formatting in api.py create_submission method
Testing:
make test_quality passes without warnings
feat(error): add SubmissionQueueCanNotBeEmptyError class error
- This class is to is raised when queue name is empty
fix(submissions): Update testing for SubmissionQueueRecord model
Update tests after rebase on branch XQU-15-add-fields-to-submission-mode
- Synchronize test suite with latest model changes
- Ensure compatibility with recent modifications to SubmissionQueueRecord
feat(submissions): Add file processing to submission queue record creation
- Extend `create_submission_queue_record` function to handle file processing
- Add optional file processing step using SubmissionFileManager
- Retrieve files from event_data and process them if present
- Add tests to validate test functionalities
feat: implement files manager in get submission
- Use SubmissionFileManager for retrieving files for grader
- Add tests for checking xqueue_files handling in get_submission endpoint
- Add tests for single file, multiple files and file URL format verification
- Fix pylint issues (variables not used, protected access warnings)
Merge XQU-24-add-files-in-submissions-workflow into XQU-26-merge-files-submissions-with-get-submission
feat: add grading metadata fields to submission queue record
- Add grader_file_name CharField (128 chars max) with empty default
- Add points_possible PositiveIntegerField with default value of 1
- Update create_submission_queue_record to handle new fields
refactor: standardize response handling in XQueueViewSet to guarantee backward compatibility with xwatcher
Changes:
- Replace direct dict responses with _compose_reply method
- Add custom authentication class XQueueSessionAuthentication
- Restructure submission data format in get_submission endpoint
- Update response format for success and error cases
Details:
1. Response Standardization:
- Replace manual {'success': True/False, 'content': ...} with _compose_reply
- Update HTTP status codes to be consistent across endpoints
2. Submission Data Structure:
- Add structured grader_payload and student_info
- Format student response with proper JSON encoding
- Maintain consistent xqueue_body format
3. Authentication:
- Add XQueueSessionAuthentication for CSRF exemption on put_result
- Keep SessionAuthentication for other endpoints
refactor: update XQueue tests for submission handling
Changes:
- Fix test_get_submission_invalid_transition to use pending status
- Update test_get_submission_success to verify student_response field
- Standardize response data validation across tests
- Add client login assertions in all test cases
- Use consistent response structure checks
update get_submission and test complete
fix: Update SessionAuthentication import for tox testing in edx-submission
When running tox, there was an import error with openedx related to
authentication classes. This change ensures compatibility during local
repository testing by directly using SessionAuthentication instead of
DefaultSessionAuthentication.
The issue occurs specifically during tox tests, as edx platform normally
uses edx-submission as a dependency without this problem. This fix
resolves the import conflict for local test environments.
feat(tests): Add authentication and core methods tests for XqueueViewSet
Add comprehensive test coverage for authentication-related endpoints and core
utility methods in XqueueViewSet:
- Test get_permissions() logic for login and other actions
- Add dispatch() tests for valid and invalid sessions
- Add login endpoint tests (success/failure scenarios)
- Add logout endpoint test
- Add _validate_grader_reply() tests
- Add _compose_reply() tests
- Add status endpoint test
This extends the existing test suite while maintaining compatibility with
previously implemented submission-related tests.
feat(security): Improve session validation in custom dispatch method
- Add robust session key format validation using regex
- Validate session key against a strict alphanumeric pattern (min 32 characters)
- Implement explicit checks for:
* Invalid session key format
* Failed session loading
* Expired session
- Use JSONSerializer for session deserialization
- Log warning messages for different session validation scenarios
- Maintain default Django dispatch flow on validation failures
Security enhancement to prevent potential session-related vulnerabilities
get_submission views complete
feat(xqueue): Add session management and authentication to XqueueViewSet
- Implement ViewSet for xwatcher service interactions
- Add custom session validation in dispatch method
- Configure permission handling for login/protected endpoints
- Set up logging for request and session debugging
The ViewSet now properly handles:
- Cookie-based session management
- Support for both sessionid and lms_sessionid
- Dynamic permission assignment per action
- Detailed request logging
test(submissions): Add comprehensive tests for XqueueViewSet
- Implement TestXqueueViewSet with multiple test scenarios
- Cover success, failure, and edge cases for XqueueViewSet
- Test endpoint behaviors including:
* Successful grade submission
* Invalid submission ID handling
* Incorrect submission key validation
* Malformed request data processing
* Error handling and logging
* Submission failure retry mechanism
- Ensure robust test coverage for XqueueViewSet functionality
feat(submissions): Add SubmissionQueueRecordFactory for test data generation
- Implement SubmissionQueueRecordFactory for creating SubmissionQueueRecord instances
- Define factory attributes with default values
- Support test data generation for submission queue records
refactor(submissions): Improve GraderViewSet implementation
- Change class name to XqueueViewSet
- Add comprehensive docstring to XqueueViewSet class
- Refactor exception handling in put_result method
- Optimize submission record status management
- Enhance logging and error tracking
feat: add nullable pullkey and grader_reply to Submission model
feat(SubmissionQueueRecord): add pullkey and grader_reply fields
- Add pullkey CharField(128) for tracking submission processing key
- Add grader_reply TextField to store grader's response
- Fields support queue processing and feedback management to xwatcher
feat(submissions): add SubmissionQueueRecord model
- Add SubmissionQueueManager to process queries
- Replace multiple timestamps with a single status_time field
- Introduce explicit state machine for submission processing
- Add atomic state transitions with validation
- Optimize database indexes for queue operations
Key changes:
- Add STATUS_CHOICES and VALID_TRANSITIONS for state management
- Consolidate pull_time, push_time, return_time into status_time
- Add transaction-safe update_status() method
- Improve queue manager with safer submission retrieval
- Add validation for state transitions
- Optimize indexes for common query patterns
The changes maintain backwards compatibility with existing queue
processing while providing a more robust and maintainable approach
to state management.
feat(grading): add initial GraderViewSet implementation
Add preliminary GraderViewSet for handling grader responses and score submissions.
This is a temporary implementation that will require significant changes in the future.
- Add GraderViewSet with score submission handling
- Register viewset in URL configuration
- Include basic validation and error handling for grader replies
- TODO: Implement proper authentication (IsAuthenticated currently commented out)
Note: This is an initial implementation that will need substantial revisions
refactor: move submissions viewset from views.py to __init__.py
Move the get_submissions_for_student_item view handler from views.py to __init__.py
within the submissions module. This view handles retrieving all submissions
associated with a specific student item based on course, student, and item IDs.
- No functional changes to the view logic
- Maintains login_required decorator and error handling
- Updates file organization while preserving existing functionality
openedx-webhooks
added
the
open-source-contribution
|
Thanks for the pull request, @leoaulasneo98! This repository is currently maintained by Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review. 🔘 Get product approvalIf you haven't already, check this list to see if your contribution needs to go through the product review process.
🔘 Provide contextTo help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:
🔘 Get a green buildIf one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green. 🔘 Update the status of your PRYour PR is currently marked as a draft. After completing the steps above, update its status by clicking "Ready for Review", or removing "WIP" from the title, as appropriate. Where can I find more information?If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources: When can I expect my changes to be merged?Our goal is to get community contributions seen and reviewed as efficiently as possible. However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:
💡 As a result it may take up to several weeks or months to complete a review and merge your PR. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #287 +/- ##
==========================================
+ Coverage 93.53% 96.23% +2.70%
==========================================
Files 18 22 +4
Lines 1995 3084 +1089
Branches 90 135 +45
==========================================
+ Hits 1866 2968 +1102
+ Misses 118 106 -12
+ Partials 11 10 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
* Fix code block syntax in Decision section * Replace technical language with clearer descriptions * Update References with current repository links * Resolve indentation and spacing issues
This comment was marked as resolved.
This comment was marked as resolved.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
FC-73 Feature: XQueueViewSet - Compatible API Endpoints for External Grader Integration
ExternalGraderDetailandSubmissionFileinfrastructure deployed in previous PRs (FC-73). Please ensure those changes are fully deployed before merging.Description
This pull request implements the XQueueViewSet, providing compatible API endpoints that allow external graders (XWatcher) to interact with the new submission and grading architecture. This viewset consolidates authentication, submission retrieval, and result processing services while leveraging the new
ExternalGraderDetailandSubmissionFilemodels.Motivation
The current XQueue implementation requires a strategic update to:
Key Improvements
Authentication Services
XQueueSessionAuthenticationclass with:Submission Distribution
get_submissionendpointSubmissionFileManagerResult Processing
put_resultendpointTechnical Details
Session Management
Score Processing
set_scoreError Handling
Testing Strategy
Comprehensive test coverage includes:
Open edX Compliance
This implementation adheres to Open edX standards through:
Performance Considerations
BREAKING CHANGES: None. Designed for full backward compatibility with existing XWatcher services.
Documentation
Updated documentation will include:
Implementation References