Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the GitHub Actions workflow by introducing more granular control over test execution and improving failure analysis. It allows for more precise selection of test applications, dynamic generation of test matrices based on APIs, and focused parsing of test results. Furthermore, it integrates automated reporting of failed jobs to an external analysis tool and refines the job retry mechanism, contributing to a more robust and efficient CI/CD pipeline. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Ignored Files
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| name: build-${{ inputs.platform }}-${{ inputs.unity_version }} | ||
| runs-on: ${{ inputs.build_os }} | ||
| env: | ||
| xcodeVersion: "16.2" | ||
| steps: | ||
| - uses: lukka/get-cmake@latest | ||
| with: | ||
| cmakeVersion: "~3.31.0" | ||
| - id: matrix_info | ||
| shell: bash | ||
| run: | | ||
| echo "info=${MATRIX_UNITY_VERSION}-${MATRIX_OS}-${MATRIX_PLATFORM}-${MATRIX_IOS_SDK}" >> $GITHUB_OUTPUT | ||
| echo "artifact_suffix=${MATRIX_UNITY_VERSION}-${MATRIX_OS}-${MATRIX_IOS_SDK}" >> $GITHUB_OUTPUT | ||
| env: | ||
| MATRIX_UNITY_VERSION: ${{ inputs.unity_version }} | ||
| MATRIX_OS: ${{ inputs.build_os }} | ||
| MATRIX_PLATFORM: ${{ inputs.platform }} | ||
| # Pass NA for ios_sdk during the single build step, or dynamic if needed later | ||
| MATRIX_IOS_SDK: "NA" | ||
| - uses: actions/checkout@v4 | ||
| - name: Setup python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.9' | ||
| - name: Install python deps | ||
| timeout-minutes: 10 | ||
| shell: bash | ||
| run: pip install -r scripts/gha/requirements.txt | ||
| - name: setup Xcode version | ||
| if: runner.os == 'macOS' | ||
| run: sudo xcode-select -s /Applications/Xcode_${{ env.xcodeVersion }}.app/Contents/Developer | ||
| - id: unity_setup | ||
| uses: ./gha/unity | ||
| timeout-minutes: 30 | ||
| with: | ||
| version: ${{ inputs.unity_version }} | ||
| platforms: ${{ inputs.platform }} | ||
| username: ${{ secrets.UNITY_USERNAME }} | ||
| password: ${{ secrets.UNITY_PASSWORD }} | ||
| serial_ids: ${{ secrets.SERIAL_ID }} | ||
| - name: Workaround tvOS XCode 15 issue | ||
| if: ${{ contains(inputs.platform, 'tvOS') && contains(inputs.unity_version, '2020') }} | ||
| shell: bash | ||
| run: | | ||
| find /Applications/Unity/Hub/Editor -type f -name 'UnityViewControllerBase.h' -exec sed -i '' 's/#import <GameController\/GCController.h>/#import <GameController\/GCEventViewController.h>/g' {} \; | ||
| - name: Prepare for integration tests | ||
| timeout-minutes: 10 | ||
| shell: bash | ||
| run: | | ||
| python scripts/gha/restore_secrets.py --passphrase "${{ secrets.TEST_SECRET }}" | ||
| - name: Setup Android keystore | ||
| if: ${{ contains(inputs.platform, 'Android') }} | ||
| shell: bash | ||
| run: | | ||
| mkdir -p ~/.android | ||
| gpg --quiet --batch --yes --decrypt --passphrase="${{ secrets.TEST_SECRET }}" \ | ||
| --output ~/.android/debug.keystore "scripts/gha-encrypted/debug_keystore.gpg" | ||
| - name: Fetch prebuilt packaged SDK from previous run | ||
| if: ${{ inputs.packaged_sdk_run_id != '' }} | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: 'firebase_unity_sdk.zip' | ||
| github-token: ${{ github.token }} | ||
| run-id: ${{ inputs.packaged_sdk_run_id }} | ||
|
|
||
| - name: Build integration tests | ||
| timeout-minutes: 240 | ||
| shell: bash | ||
| run: | | ||
| if [[ -n "${PACKAGED_SDK_RUN_ID}" ]]; then | ||
| unzip -q firebase_unity_sdk.zip -d ~/Downloads/ | ||
| else | ||
| curl -L "https://firebase.google.com/download/unity" -o ~/Downloads/firebase_unity_sdk.zip | ||
| unzip -q ~/Downloads/firebase_unity_sdk.zip -d ~/Downloads/ | ||
| fi | ||
| CLEAN_APIS=$(echo ${APIS} | tr -d "'" | tr -d "\"") | ||
| python scripts/gha/build_testapps.py \ | ||
| --t ${CLEAN_APIS} \ | ||
| --u ${UNITY_VERSION} \ | ||
| --p "${PLATFORM}" \ | ||
| --ios_sdk "NA" \ | ||
| --plugin_dir ~/Downloads/firebase_unity_sdk \ | ||
| --output_directory "${{ github.workspace }}" \ | ||
| --artifact_name "${STEPS_MATRIX_INFO_OUTPUTS_INFO}" \ | ||
| --notimestamp \ | ||
| --force_latest_runtime \ | ||
| --ci | ||
| env: | ||
| PACKAGED_SDK_RUN_ID: ${{ inputs.packaged_sdk_run_id }} | ||
| APIS: ${{ inputs.apis }} | ||
| PLATFORM: ${{ inputs.platform }} | ||
| UNITY_VERSION: ${{ inputs.unity_version }} | ||
| STEPS_MATRIX_INFO_OUTPUTS_INFO: ${{ steps.matrix_info.outputs.info }} | ||
|
|
||
| - name: Return Unity license | ||
| if: always() | ||
| uses: ./gha/unity | ||
| with: | ||
| version: ${{ inputs.unity_version }} | ||
| release_license: "true" | ||
|
|
||
| - name: Prepare results summary artifact | ||
| if: ${{ !cancelled() }} | ||
| shell: bash | ||
| run: | | ||
| if [ ! -f build-results-${STEPS_MATRIX_INFO_OUTPUTS_INFO}.log.json ]; then | ||
| echo "__SUMMARY_MISSING__" > build-results-${STEPS_MATRIX_INFO_OUTPUTS_INFO}.log.json | ||
| fi | ||
| env: | ||
| STEPS_MATRIX_INFO_OUTPUTS_INFO: ${{ steps.matrix_info.outputs.info }} | ||
|
|
||
| - name: Upload build results artifact | ||
| uses: actions/upload-artifact@v4 | ||
| if: ${{ !cancelled() }} | ||
| with: | ||
| name: build_and_test_results-build-${{ steps.matrix_info.outputs.info }} | ||
| path: build-results-${{ steps.matrix_info.outputs.info }}* | ||
| retention-days: 2 | ||
|
|
||
| - name: Upload Mobile integration tests artifact | ||
| uses: actions/upload-artifact@v4 | ||
| if: ${{ contains('Android,iOS,tvOS', inputs.platform) && !cancelled() }} | ||
| with: | ||
| name: testapps-${{ inputs.platform }}-${{ steps.matrix_info.outputs.artifact_suffix }} | ||
| path: testapps-${{ steps.matrix_info.outputs.info }} | ||
| retention-days: 2 | ||
|
|
||
| - name: Upload Desktop integration tests artifact | ||
| uses: actions/upload-artifact@v4 | ||
| if: ${{ !contains('Android,iOS,tvOS', inputs.platform) && !cancelled() }} | ||
| with: | ||
| name: testapps-${{ inputs.platform }}-${{ steps.matrix_info.outputs.artifact_suffix }} | ||
| path: testapps-${{ steps.matrix_info.outputs.info }}/${{ inputs.platform }} | ||
| retention-days: 2 | ||
|
|
||
| prepare_tests: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
| needs: build_testapp | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| test_matrix: ${{ steps.set-matrix.outputs.test_matrix }} | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - name: Generate Test Matrix | ||
| id: set-matrix | ||
| run: | | ||
| matrix=$(python scripts/gha/print_matrix_configuration.py -test_matrix \ | ||
| -platforms ${{ inputs.platform }} \ | ||
| -os ${{ inputs.build_os }} \ | ||
| -unity_versions ${{ inputs.unity_version }} \ | ||
| -mobile_test_on ${{ inputs.mobile_device_types }} \ | ||
| -apis "${{ inputs.apis }}") | ||
| echo "test_matrix=$matrix" >> $GITHUB_OUTPUT | ||
|
|
||
| integration_test: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
| needs: [build_testapp, prepare_tests] | ||
| name: test-${{ matrix.platform }}-${{ matrix.test_device }}-${{ matrix.api }} | ||
| runs-on: ${{ matrix.test_os }} | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: ${{ fromJSON(needs.prepare_tests.outputs.test_matrix) }} | ||
| env: | ||
| xcodeVersion: "16.2" | ||
| steps: | ||
| - id: matrix_info | ||
| shell: bash | ||
| run: | | ||
| echo "info=${MATRIX_UNITY_VERSION}-${MATRIX_BUILD_OS}-${MATRIX_PLATFORM}-${MATRIX_TEST_DEVICE}-${MATRIX_TEST_OS}-${MATRIX_API}" >> $GITHUB_OUTPUT | ||
| echo "artifact_path=testapps-${MATRIX_PLATFORM}-${MATRIX_UNITY_VERSION}-${MATRIX_BUILD_OS}-NA" >> $GITHUB_OUTPUT | ||
| env: | ||
| MATRIX_UNITY_VERSION: ${{ matrix.unity_version }} | ||
| MATRIX_BUILD_OS: ${{ matrix.build_os }} | ||
| MATRIX_PLATFORM: ${{ matrix.platform }} | ||
| MATRIX_TEST_DEVICE: ${{ matrix.test_device }} | ||
| MATRIX_TEST_OS: ${{ matrix.test_os }} | ||
| MATRIX_API: ${{ matrix.api }} | ||
| - uses: actions/checkout@v4 | ||
| - name: Setup python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.9' | ||
| - name: Install python deps | ||
| timeout-minutes: 10 | ||
| shell: bash | ||
| run: pip install -r scripts/gha/requirements.txt | ||
| - name: setup Xcode version | ||
| if: runner.os == 'macOS' | ||
| run: sudo xcode-select -s /Applications/Xcode_${{ env.xcodeVersion }}.app/Contents/Developer | ||
| - name: Download Testapp artifacts | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| path: testapps | ||
| name: ${{ steps.matrix_info.outputs.artifact_path }} | ||
| - name: Set up Node (22) | ||
| if: ${{ matrix.test_device == 'github_runner' }} | ||
| uses: actions/setup-node@v5 | ||
| with: | ||
| node-version: 22 | ||
| - name: Setup java for Firestore emulator | ||
| if: ${{ matrix.test_device == 'github_runner' }} | ||
| uses: actions/setup-java@v3 | ||
| with: | ||
| distribution: 'temurin' | ||
| java-version: '21' | ||
| - name: Setup Firestore Emulator | ||
| if: ${{ matrix.test_device == 'github_runner' }} | ||
| uses: nick-invision/retry@v2 | ||
| with: | ||
| shell: bash | ||
| timeout_minutes: 5 | ||
| max_attempts: 3 | ||
| command: npm install -g firebase-tools | ||
|
|
||
| - name: Run Desktop integration tests | ||
| if: ${{ matrix.test_device == 'github_runner' }} | ||
| timeout-minutes: 30 | ||
| shell: bash | ||
| run: | | ||
| firebase emulators:start --only firestore & | ||
| python scripts/gha/desktop_tester.py --testapp_dir testapps \ | ||
| --testapp_name ${{ matrix.api }} \ | ||
| --logfile_name "${STEPS_MATRIX_INFO_OUTPUTS_INFO}" | ||
| env: | ||
| STEPS_MATRIX_INFO_OUTPUTS_INFO: ${{ steps.matrix_info.outputs.info }} | ||
|
|
||
| - name: Run Mobile integration tests on real device via FTL | ||
| id: ftl_test | ||
| if: ${{ matrix.device_type == 'real' }} | ||
| uses: FirebaseExtended/github-actions/firebase-test-lab@v1.4 | ||
| with: | ||
| credentials_json: ${{ secrets.FIREBASE_SERVICE_ACCOUNT_CREDENTIALS }} | ||
| testapp_dir: testapps | ||
| test_type: "game-loop" | ||
| timeout: 1200 | ||
| test_devices: ${{ matrix.device_detail }} | ||
| test_device_selection: random | ||
| max_attempts: 3 | ||
| validator: ${GITHUB_WORKSPACE}/scripts/gha/read_ftl_test_result.py | ||
|
|
||
| - name: Read FTL Test Result | ||
| if: ${{ matrix.device_type == 'real' && !cancelled() }} | ||
| timeout-minutes: 60 | ||
| shell: bash | ||
| run: | | ||
| python scripts/gha/read_ftl_test_result.py --test_result "${STEPS_FTL_TEST_OUTPUTS_TEST_SUMMARY}" \ | ||
| --testapp_name ${{ matrix.api }} \ | ||
| --output_path testapps/test-results-"${STEPS_MATRIX_INFO_OUTPUTS_INFO}".log | ||
| env: | ||
| STEPS_FTL_TEST_OUTPUTS_TEST_SUMMARY: ${{ steps.ftl_test.outputs.test_summary }} | ||
| STEPS_MATRIX_INFO_OUTPUTS_INFO: ${{ steps.matrix_info.outputs.info }} | ||
|
|
||
| - name: Print available devices | ||
| if: ${{ matrix.device_type == 'virtual' && !cancelled() }} | ||
| run: | | ||
| xcrun xctrace list devices | ||
| - name: Create keychain (macOS Simulator) | ||
| if: ${{ contains('iOS,tvOS', matrix.platform) && matrix.device_type == 'virtual'}} | ||
| shell: bash | ||
| run: | | ||
| echo "Creating temporary keychain" | ||
| security delete-keychain tmp-keychain 2> /dev/null || true | ||
| security -i <<EOF | ||
| create-keychain -p ${{ secrets.TEST_SECRET }} tmp-keychain | ||
| set-keychain-settings tmp-keychain | ||
| unlock-keychain -p ${{ secrets.TEST_SECRET }} tmp-keychain | ||
| EOF | ||
| security list-keychains -d user -s tmp-keychain | ||
| security default-keychain -s tmp-keychain | ||
|
|
||
| - name: Run Mobile integration tests on virtual device locally | ||
| timeout-minutes: 120 | ||
| if: ${{ matrix.device_type == 'virtual' && !cancelled() }} | ||
| run: | | ||
| python scripts/gha/test_simulator.py --testapp_dir testapps \ | ||
| --tvos_device "${{ matrix.test_device }}" \ | ||
| --ios_device "${{ matrix.test_device }}" \ | ||
| --android_device "${{ matrix.test_device }}" \ | ||
| --testapp_name ${{ matrix.api }} \ | ||
| --logfile_name "${{ steps.matrix_info.outputs.info }}" \ | ||
| --ci | ||
|
|
||
| - name: Delete keychain (macOS Simulator) | ||
| if: ${{ always() && contains('iOS,tvOS', matrix.platform) && matrix.device_type == 'virtual' }} | ||
| shell: bash | ||
| run: | | ||
| security list-keychains -d user -s login.keychain | ||
| security delete-keychain tmp-keychain || true | ||
|
|
||
| - name: Prepare results summary artifact | ||
| if: ${{ !cancelled() }} | ||
| shell: bash | ||
| run: | | ||
| if [ -d "testapps/testapps-${STEPS_MATRIX_INFO_OUTPUTS_INFO}" && ! -f testapps/test-results-${STEPS_MATRIX_INFO_OUTPUTS_INFO}.log.json ]; then | ||
| mkdir -p testapps && echo "__SUMMARY_MISSING__" > testapps/test-results-${STEPS_MATRIX_INFO_OUTPUTS_INFO}.log.json | ||
| fi | ||
| env: | ||
| STEPS_MATRIX_INFO_OUTPUTS_INFO: ${{ steps.matrix_info.outputs.info }} | ||
|
|
||
| - name: Upload test results artifact | ||
| if: ${{ !cancelled() }} | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: build_and_test_results-test-${{ steps.matrix_info.outputs.info }} | ||
| path: testapps/test-results-${{ steps.matrix_info.outputs.info }}* | ||
| retention-days: 2 | ||
|
|
||
| - name: Summarize test results | ||
| if: ${{ !cancelled() }} | ||
| shell: bash | ||
| run: | | ||
| cat testapps/test-results-${STEPS_MATRIX_INFO_OUTPUTS_INFO}.log | ||
| if [[ "${JOB_STATUS}" != "success" ]]; then | ||
| exit 1 | ||
| fi | ||
| env: | ||
| STEPS_MATRIX_INFO_OUTPUTS_INFO: ${{ steps.matrix_info.outputs.info }} | ||
| JOB_STATUS: ${{ job.status }} |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
The pull request introduces a new API filtering mechanism across several testing scripts (desktop_tester.py, print_matrix_configuration.py, test_simulator.py, read_ftl_test_result.py), enhancing the ability to target specific test applications. It also adds a new utility script, report_to_jules.py, for automated root cause analysis of failed GitHub Actions jobs, and improves retry_test_failures.py with more flexible job name matching and retry reporting. However, the new report_to_jules.py script contains potential security vulnerabilities related to Regular Expression Denial of Service (ReDoS) and Prompt Injection, which should be addressed to ensure the reliability and integrity of the automated reporting system.
| match = group_start_re.match(line) | ||
| if match: | ||
| step_name = match.group(1).strip() | ||
| if re.search(pattern, step_name): |
There was a problem hiding this comment.
The script uses user-provided regex patterns (include_step_pattern and include_job_pattern) in re.search against untrusted input (step names and job names). A malicious regex can cause catastrophic backtracking, leading to a Regular Expression Denial of Service (ReDoS) of the CI job. Consider sanitizing the user-provided regex patterns or using a regex engine that is not vulnerable to backtracking, such as google/re2.
|
|
||
| message = f"Logs for Test: {job['name']}\n{'-'*40}\n{truncated_log}\n" | ||
|
|
||
| send_message(FLAGS.jules_token, session_id, message) |
There was a problem hiding this comment.
Untrusted log content is directly concatenated into the prompt sent to the Jules LLM. An attacker who can control the logs (e.g., by causing a test failure with a specific error message) can inject instructions to manipulate the LLM's behavior and output, potentially leading to misleading root cause analysis reports. This is a classic Prompt Injection vulnerability. To mitigate this, use clear delimiters (e.g., XML-like tags) to separate the untrusted log content from the system instructions and explicitly instruct the LLM to ignore any instructions contained within the log content.
1 participant
Description
Testing
Type of Change
Place an
xthe applicable box: