Skip to content

Commit dfc95c1

Browse files
jketemambaluda
jketema
authored and
mbaluda
committed
Modernize qlpacks and update to CodeQL 2.8.5
1 parent e451f27 commit dfc95c1

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+409
-59
lines changed

‎.codeqlmanifest.json

+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
{ "provide": [ "codeql_modules/*/.codeqlmanifest.json", "cpp/.codeqlmanifest.json", "c/.codeqlmanifest.json"] }
1+
{ "provide": [ "cpp/*/src/qlpack.yml", "cpp/*/test/qlpack.yml", "c/*/src/qlpack.yml", "c/*/test/qlpack.yml" ] }

‎.github/actions/action.yml

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
name: Install CodeQL library pack dependencies
2+
description: |
3+
Downloads any necessary CodeQL library packs needed by packs in the repo.
4+
inputs:
5+
cli_path:
6+
description: |
7+
The path to the CodeQL CLI directory.
8+
required: false
9+
10+
mode:
11+
description: |
12+
The `--mode` option to `codeql pack install`.
13+
required: true
14+
default: verify
15+
16+
runs:
17+
using: composite
18+
steps:
19+
- name: Install CodeQL library packs
20+
shell: bash
21+
env:
22+
CODEQL_CLI: ${{ inputs.cli_path }}
23+
run: |
24+
PATH=$PATH:$CODEQL_CLI
25+
python scripts/install-packs.py --mode ${{ inputs.mode }}

‎.github/actions/install-codeql-packs/action.yml

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
name: Install CodeQL library pack dependencies
2+
description: |
3+
Downloads any necessary CodeQL library packs needed by packs in the repo.
4+
inputs:
5+
cli_path:
6+
description: |
7+
The path to the CodeQL CLI directory.
8+
required: false
9+
10+
mode:
11+
description: |
12+
The `--mode` option to `codeql pack install`.
13+
required: true
14+
default: verify
15+
16+
runs:
17+
using: composite
18+
steps:
19+
- name: Install CodeQL library packs
20+
shell: bash
21+
env:
22+
CODEQL_CLI: ${{ inputs.cli_path }}
23+
run: |
24+
PATH=$PATH:$CODEQL_CLI
25+
python scripts/install-packs.py --mode ${{ inputs.mode }}

‎.github/workflows/code-scanning-pack-gen.yml

+7-2
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,11 @@ jobs:
5959
codeql-home: ${{ github.workspace }}/codeql_home
6060
add-to-path: false
6161

62+
- name: Install CodeQL packs
63+
uses: ./.github/actions/install-codeql-packs
64+
with:
65+
cli_path: ${{ github.workspace }}/codeql_home/codeql
66+
6267
- name: Checkout external help files
6368
continue-on-error: true
6469
id: checkout-external-help-files
@@ -82,8 +87,8 @@ jobs:
8287
run: |
8388
PATH=$PATH:$CODEQL_HOME/codeql
8489
85-
codeql query compile --search-path cpp --threads 0 cpp
86-
codeql query compile --search-path c --search-path cpp --threads 0 c
90+
codeql query compile --threads 0 cpp
91+
codeql query compile --threads 0 c
8792
8893
cd ..
8994
zip -r codeql-coding-standards/code-scanning-cpp-query-pack.zip codeql-coding-standards/c/ codeql-coding-standards/cpp/ codeql-coding-standards/.codeqlmanifest.json codeql-coding-standards/supported_codeql_configs.json codeql-coding-standards/scripts/deviations codeql-coding-standards/scripts/reports

‎.github/workflows/codeql_unit_tests.yml

+9-11
Original file line numberDiff line numberDiff line change
@@ -66,11 +66,16 @@ jobs:
6666
codeql-home: ${{ github.workspace }}/codeql_home
6767
add-to-path: false
6868

69+
- name: Install CodeQL packs
70+
uses: ./.github/actions/install-codeql-packs
71+
with:
72+
cli_path: ${{ github.workspace }}/codeql_home/codeql
73+
6974
- name: Pre-Compile Queries
7075
id: pre-compile-queries
7176
run: |
72-
${{ github.workspace }}/codeql_home/codeql/codeql query compile --search-path cpp --threads 0 cpp
73-
${{ github.workspace }}/codeql_home/codeql/codeql query compile --search-path c --search-path cpp --threads 0 c
77+
${{ github.workspace }}/codeql_home/codeql/codeql query compile --threads 0 cpp
78+
${{ github.workspace }}/codeql_home/codeql/codeql query compile --threads 0 c
7479
7580
7681
- name: Run test suites
@@ -122,18 +127,11 @@ jobs:
122127
os.makedirs(os.path.dirname(test_report_path), exist_ok=True)
123128
test_report_file = open(test_report_path, 'w')
124129
files_to_close.append(test_report_file)
125-
if "${{ matrix.language }}".casefold() == "c".casefold():
126-
# c tests require cpp -- but we don't want c things on the cpp
127-
# path in case of design errors.
128-
cpp_language_root = Path(workspace, 'cpp')
129-
procs.append(subprocess.Popen([codeql_bin, "test", "run", "--failing-exitcode=122", f"--slice={slice}/{num_slices}", "--ram=2048", "--format=json", f'--search-path={cpp_language_root}', f'--search-path={language_root}', *test_roots], stdout=test_report_file, stderr=subprocess.PIPE))
130-
else:
131-
procs.append(subprocess.Popen([codeql_bin, "test", "run", "--failing-exitcode=122", f"--slice={slice}/{num_slices}", "--ram=2048", "--format=json", f'--search-path={language_root}', f'--search-path={language_root}', *test_roots], stdout=test_report_file, stderr=subprocess.PIPE))
130+
procs.append(subprocess.Popen([codeql_bin, "test", "run", "--failing-exitcode=122", f"--slice={slice}/{num_slices}", "--ram=2048", "--format=json", *test_roots], stdout=test_report_file, stderr=subprocess.PIPE))
132131
133132
for p in procs:
134-
p.wait()
133+
_, err = p.communicate()
135134
if p.returncode != 0:
136-
_, err = p.communicate()
137135
if p.returncode == 122:
138136
# Failed because a test case failed, so just print the regular output.
139137
# This will allow us to proceed to validate-test-results, which will fail if

‎.github/workflows/standard_library_upgrade_tests.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,7 @@ jobs:
116116
stdlib_path = os.path.join(codeql_home, 'codeql-stdlib')
117117
cpp_test_root = Path(stdlib_path, 'cpp/ql/test')
118118
print(f"Executing tests found (recursively) in the directory '{cpp_test_root}'")
119-
cp = subprocess.run([codeql_bin, "test", "run", "--format=json", f'--search-path={stdlib_path}', cpp_test_root], stdout=test_report_file, stderr=subprocess.PIPE)
119+
cp = subprocess.run([codeql_bin, "test", "run", "--format=json", cpp_test_root], stdout=test_report_file, stderr=subprocess.PIPE)
120120
if cp.returncode != 0:
121121
print_error_and_fail(f"Failed to run tests with return code {cp.returncode} and error {cp.stderr}")
122122

‎.github/workflows/verify-standard-library-dependencies.yml

+79
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
name: Verify Standard Library Dependencies
2+
3+
# Run this workflow every time the "supported_codeql_configs.json" file or a "qlpack.yml" file is changed
4+
on:
5+
pull_request:
6+
branches:
7+
- main
8+
- "rc/**"
9+
- next
10+
paths:
11+
- "supported_codeql_configs.json"
12+
- "qlpack.yml"
13+
workflow_dispatch:
14+
15+
jobs:
16+
prepare-matrix:
17+
name: Prepare CodeQL configuration matrix
18+
runs-on: ubuntu-latest
19+
outputs:
20+
matrix: ${{ steps.export-matrix.outputs.matrix }}
21+
steps:
22+
- name: Checkout repository
23+
uses: actions/checkout@v2
24+
25+
- name: Export unit test matrix
26+
id: export-matrix
27+
run: |
28+
echo "::set-output name=matrix::$(
29+
jq --compact-output \
30+
'.supported_environment | map([.+{os: "ubuntu-20.04-xl", codeql_standard_library_ident : .codeql_standard_library | sub("\/"; "_")}]) | flatten | {include: .}' \
31+
supported_codeql_configs.json
32+
)"
33+
34+
verify-dependencies:
35+
name: Verify dependencies
36+
needs: prepare-matrix
37+
38+
runs-on: ${{ matrix.os }}
39+
strategy:
40+
fail-fast: false
41+
matrix: ${{fromJSON(needs.prepare-matrix.outputs.matrix)}}
42+
43+
steps:
44+
- name: Checkout repository
45+
uses: actions/checkout@v2
46+
47+
- name: Setup Python 3
48+
uses: actions/setup-python@v2
49+
with:
50+
python-version: "3.x"
51+
52+
- name: Cache CodeQL
53+
id: cache-codeql
54+
uses: actions/[email protected]
55+
with:
56+
# A list of files, directories, and wildcard patterns to cache and restore
57+
path: ${{github.workspace}}/codeql_home
58+
# An explicit key for restoring and saving the cache
59+
key: codeql-home-${{matrix.os}}-${{matrix.codeql_cli}}-${{matrix.codeql_standard_library}}
60+
61+
- name: Install CodeQL
62+
if: steps.cache-codeql.outputs.cache-hit != 'true'
63+
uses: ./.github/actions/install-codeql
64+
with:
65+
codeql-cli-version: ${{matrix.codeql_cli}}
66+
codeql-stdlib-version: ${{matrix.codeql_standard_library}}
67+
codeql-home: ${{ github.workspace }}/codeql_home
68+
69+
- name: Verify dependencies
70+
shell: bash
71+
env:
72+
CLI_PATH: ${{ github.workspace }}/codeql_home/codeql
73+
STDLIB_PATH: ${{ github.workspace }}/codeql_home/codeql-stdlib
74+
run: |
75+
PATH=$PATH:$CLI_PATH
76+
ls $STDLIB_PATH
77+
pip install -r scripts/requirements.txt
78+
python3 scripts/verify-standard-library-version.py --codeql-repo $STDLIB_PATH --mode verify
79+

‎.gitignore

+3
Original file line numberDiff line numberDiff line change
@@ -20,3 +20,6 @@
2020
# C/C++ build artifacts
2121
*.o
2222
/databases/
23+
24+
# CodeQL build artifacts
25+
**/.codeql/**

‎c/.codeqlmanifest.json

-3
This file was deleted.

‎c/cert/src/codeql-pack.lock.yml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
dependencies:
3+
codeql/cpp-all:
4+
version: 0.0.13
5+
compiled: false
6+
lockVersion: 1.0.0

‎c/cert/src/qlpack.yml

+3-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
name: cert-c-coding-standards
22
version: 2.8.0-dev
33
suites: codeql-suites
4-
libraryPathDependencies: common-c-coding-standards
4+
dependencies:
5+
codeql/common-c-coding-standards: '*'
6+
codeql/cpp-all: 0.0.13

‎c/cert/test/codeql-pack.lock.yml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
dependencies:
3+
codeql/cpp-all:
4+
version: 0.0.13
5+
compiled: false
6+
lockVersion: 1.0.0

‎c/cert/test/qlpack.yml

+4-3
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
name: cert-c-coding-standards-tests
1+
name: codeql/cert-c-coding-standards-tests
22
version: 2.8.0-dev
3-
libraryPathDependencies: cert-c-coding-standards
4-
extractor: cpp
3+
extractor: cpp
4+
dependencies:
5+
codeql/cert-c-coding-standards: '*'

‎c/common/src/codeql-pack.lock.yml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
dependencies:
3+
codeql/cpp-all:
4+
version: 0.0.13
5+
compiled: false
6+
lockVersion: 1.0.0

‎c/common/src/qlpack.yml

+4-2
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1-
name: common-c-coding-standards
1+
name: codeql/common-c-coding-standards
22
version: 2.8.0-dev
3-
libraryPathDependencies: common-cpp-coding-standards
3+
dependencies:
4+
codeql/common-cpp-coding-standards: '*'
5+
codeql/cpp-all: 0.0.13

‎c/common/test/codeql-pack.lock.yml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
dependencies:
3+
codeql/cpp-all:
4+
version: 0.0.13
5+
compiled: false
6+
lockVersion: 1.0.0

‎c/common/test/qlpack.yml

+3-2
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
name: common-c-coding-standards-tests
1+
name: codeql/common-c-coding-standards-tests
22
version: 2.8.0-dev
3-
libraryPathDependencies: common-c-coding-standards
43
extractor: cpp
4+
dependencies:
5+
codeql/common-c-coding-standards: '*'

‎c/misra/src/codeql-pack.lock.yml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
dependencies:
3+
codeql/cpp-all:
4+
version: 0.0.13
5+
compiled: false
6+
lockVersion: 1.0.0

‎c/misra/src/qlpack.yml

+4-2
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
1-
name: misra-c-coding-standards
1+
name: codeql/misra-c-coding-standards
22
version: 2.8.0-dev
33
suites: codeql-suites
4-
libraryPathDependencies: common-c-coding-standards
4+
dependencies:
5+
codeql/common-c-coding-standards: '*'
6+
codeql/cpp-all: 0.0.13

‎c/misra/test/codeql-pack.lock.yml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
dependencies:
3+
codeql/cpp-all:
4+
version: 0.0.13
5+
compiled: false
6+
lockVersion: 1.0.0

‎c/misra/test/qlpack.yml

+4-3
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
name: misra-c-coding-standards-tests
1+
name: codeql/misra-c-coding-standards-tests
22
version: 2.8.0-dev
3-
libraryPathDependencies: misra-c-coding-standards
4-
extractor: cpp
3+
extractor: cpp
4+
dependencies:
5+
codeql/misra-c-coding-standards: '*'

‎cpp/.codeqlmanifest.json

-3
This file was deleted.

‎cpp/autosar/src/codeql-pack.lock.yml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
dependencies:
3+
codeql/cpp-all:
4+
version: 0.0.13
5+
compiled: false
6+
lockVersion: 1.0.0

‎cpp/autosar/src/qlpack.yml

+4-2
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
1-
name: autosar-cpp-coding-standards
1+
name: codeql/autosar-cpp-coding-standards
22
version: 2.8.0-dev
33
suites: codeql-suites
4-
libraryPathDependencies: common-cpp-coding-standards
4+
dependencies:
5+
codeql/common-cpp-coding-standards: '*'
6+
codeql/cpp-all: 0.0.13

‎cpp/autosar/test/codeql-pack.lock.yml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
dependencies:
3+
codeql/cpp-all:
4+
version: 0.0.13
5+
compiled: false
6+
lockVersion: 1.0.0

‎cpp/autosar/test/qlpack.yml

+3-2
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
name: autosar-cpp-coding-standards-tests
1+
name: codeql/autosar-cpp-coding-standards-tests
22
version: 2.8.0-dev
3-
libraryPathDependencies: autosar-cpp-coding-standards
43
extractor: cpp
4+
dependencies:
5+
codeql/autosar-cpp-coding-standards: '*'

‎cpp/cert/src/codeql-pack.lock.yml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
dependencies:
3+
codeql/cpp-all:
4+
version: 0.0.13
5+
compiled: false
6+
lockVersion: 1.0.0

‎cpp/cert/src/qlpack.yml

+4-2
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
1-
name: cert-cpp-coding-standards
1+
name: codeql/cert-cpp-coding-standards
22
version: 2.8.0-dev
33
suites: codeql-suites
4-
libraryPathDependencies: common-cpp-coding-standards
4+
dependencies:
5+
codeql/cpp-all: 0.0.13
6+
codeql/common-cpp-coding-standards: '*'

‎cpp/cert/test/codeql-pack.lock.yml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
dependencies:
3+
codeql/cpp-all:
4+
version: 0.0.13
5+
compiled: false
6+
lockVersion: 1.0.0

‎cpp/cert/test/qlpack.yml

+3-2
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
name: cert-cpp-coding-standards-tests
1+
name: codeql/cert-cpp-coding-standards-tests
22
version: 2.8.0-dev
3-
libraryPathDependencies: cert-cpp-coding-standards
43
extractor: cpp
4+
dependencies:
5+
codeql/cert-cpp-coding-standards: '*'

‎cpp/common/src/codeql-pack.lock.yml

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
dependencies:
3+
codeql/cpp-all:
4+
version: 0.0.13
5+
compiled: false
6+
lockVersion: 1.0.0

‎cpp/common/src/qlpack.yml

+3-2
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1-
name: common-cpp-coding-standards
1+
name: codeql/common-cpp-coding-standards
22
version: 2.8.0-dev
3-
libraryPathDependencies: codeql-cpp
3+
dependencies:
4+
codeql/cpp-all: 0.0.13

0 commit comments

Comments
 (0)