Refactor JSON review processing script and GitHub Actions workflow

Updated the workflow to remove the exit code on script failure, allowing for continued execution. Enhanced the script to provide clearer feedback on failed files, indicating their movement to a dedicated 'failed' directory and improving overall error handling and user feedback.

Author: stephen-rowan

.github/workflows/process_json_reviews.yml

@@ -35,7 +35,7 @@ jobs:
       env:
         GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       run: |
-        python scripts/process_json_reviews.py || exit 1
+        python scripts/process_json_reviews.py
     
     - name: Commit and push processed files
       if: success() || failure()  # Run even if previous step failed

Scripts/process_json_reviews.py

@@ -332,16 +332,25 @@ def main() -> None:
     print(f"  Total files: {len(json_files)}")
 
     if failed_files:
-        print(f"\nFailed files:")
+        print(f"\nFailed files (moved to {FAILED_DIR}/):")
         for filename, error in failed_files:
             print(f"  - {filename}: {error}")
+        print(f"\n⚠ Note: Failed files have been moved to {FAILED_DIR}/ and will not be retried.")
 
-    if failed_count > 0:
-        print(f"\n✗ Exiting with error code 1 due to {failed_count} failed file(s)")
-        sys.exit(1)
-    else:
-        print(f"\n✓ All files processed successfully")
+    # Exit successfully if we handled all files (either processed or moved to failed/)
+    # We only exit with error if we couldn't handle the files at all
+    if processed_count > 0 or (failed_count > 0 and processed_count + failed_count == len(json_files)):
+        # All files were handled (either processed or moved to failed directory)
+        if failed_count > 0:
+            print(f"\n✓ Handled {processed_count + failed_count} file(s) ({processed_count} processed, {failed_count} moved to failed/)")
+            print(f"  Failed files are in {FAILED_DIR}/ and can be reviewed manually.")
+        else:
+            print(f"\n✓ All {processed_count} file(s) processed successfully")
         sys.exit(0)
+    else:
+        # This shouldn't happen, but if it does, something went wrong
+        print(f"\n✗ Error: Could not handle all files properly")
+        sys.exit(1)
 
 
 if __name__ == "__main__":