Skip to content

Commit b94e2b8

Browse files
samblessambles
committed
Hotfix/update piwind tests fixed (#109)
* Placeholder * Add docker-compose file for plat testing * Add build workflow for piwind image * test docker push + add piwind branch opt * fix docker login * Ready script for remote testing * disable on push * Extent dockerfile with package/branch install * booleans broken * Fix branch clone * Try single line? * Note for fix later * switch to docker build n push * Add missing dockerfile * Switch to single from ARG * Switch to single from ARG * fix * Fix push arg * Fix output * Fix defaullt docker push arg * Fix image link * Fix image url * Fix image url * Fix from image * fix * test that from image is set correctly * enough.. require an input image or fail * Why is the build arg ignored??! * Drop build args.. just use sed * try pytest-docker-compose * Work in progress * Fix url * touch up image summary * WIP - next step, convert to class and scope at class lvl * Update test to use class * Add all test classes * Fix paths and clean up * Fix test layout * Add DEBUG env * Switch to api client only + add tearDown method * try unit tests in gh actions * Fix requ * Run everything * Store results & update inputs * update name * this will prob fail * fix * try workaround * fix syntax * sans one s * rm stray input * fix exitode * fix secrets * Fix input image name * Add docker logging * check "master" branch testing * Add manual trigger to piwind test * Add script from build repo * remove build repo from mdk testing * Update branch selection * Update piwind test for calls from ext repos * fix inputs * replace checkout with script pull from url * Fix path * Fix inputs * Fix inputs * Fix inputs * fix param setting * fix param setting * fix missing from * update * fix piwind test branch * fix branch * fix branch * Move image build into test workflow * Add GH context for degugging * fix job needs * Fix check for package and branch install request * Fix missing dockerfile arg * fix inputs to piwind test * fix branch selection for fixes * Remove testing values * Fix build args * must load buildx image to docker local image list * switch to ref for PR * Update PR fetch * Test CI from external PR (#108) * Add PR triggers * Fix clone * Fix checkout Pull request from ref tag * Revert "Fix checkout Pull request from ref tag" This reverts commit d296175. * Add back older check-results script - prevent breakage * ready for merge * Update triggers * Update workflow notes * Fix branch trigger * Set branches based on PR target * fix * fix * Update readme * Fix for branch selection for push actions * Fix on push * Revert "Add all test classes" This reverts commit 3dc05f8. * Revert expected dir name * fix branch name on push * Fix script location * typo * fix * test PR merge vs develop
1 parent 53cd44b commit b94e2b8

12 files changed

+1356
-120
lines changed

Diff for: ‎.github/workflows/build.yml

+133
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
name: Build worker
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
docker_push:
7+
description: 'Push image to dockerhub and outout reference [true, false]'
8+
required: false
9+
type: boolean
10+
from_image:
11+
description: 'The worker image to build piwind [img:tag]'
12+
required: false
13+
default: 'coreoasis/model_worker:latest'
14+
piwind_image:
15+
description: 'Output image name and tag [img:tag]'
16+
default: 'coreoasis/github-actions:piwind_worker-<sha>'
17+
required: false
18+
oasislmf_branch:
19+
description: 'If set, pip install oasislmf branch [git ref]'
20+
required: false
21+
type: string
22+
oasislmf_package:
23+
description: 'If set, pip install local oasislmf package [pkg-filename.whl]'
24+
required: false
25+
type: string
26+
27+
workflow_call:
28+
inputs:
29+
docker_push:
30+
description: 'Push image to dockerhub and outout reference [true, false]'
31+
required: false
32+
type: string
33+
from_image:
34+
description: 'The worker image to build piwind [img:tag]'
35+
required: true
36+
type: string
37+
piwind_image:
38+
description: 'Output image name and tag [img:tag]'
39+
required: false
40+
type: string
41+
piwind_branch:
42+
description: '(Remote call) branch to build piwind worker from'
43+
required: false
44+
type: string
45+
oasislmf_branch:
46+
description: 'If set, pip install oasislmf branch [git ref]'
47+
required: false
48+
type: string
49+
oasislmf_package:
50+
description: 'If set, pip install local oasislmf package [pkg-filename.whl]'
51+
required: false
52+
type: string
53+
54+
outputs:
55+
piwind_image:
56+
description:
57+
value: ${{ jobs.build_piwind.outputs.image }}
58+
59+
jobs:
60+
build_piwind:
61+
name: Build PiWind Worker
62+
env:
63+
PIWIND_IMAGE: 'coreoasis/github-actions:piwind_worker-${{ github.sha }}'
64+
PIWIND_BRANCH: ${{ github.ref_name }}
65+
DOCKERFILE: 'docker/Dockerfile.piwind_worker'
66+
DOCKER_PUSH: 'false'
67+
BUILD_PACKAGE: ''
68+
BUILD_BRANCH: ''
69+
70+
runs-on: ubuntu-latest
71+
outputs:
72+
image: ${{ steps.docker_ref.outputs.image }}
73+
steps:
74+
- name: Set inputs
75+
run: |
76+
[[ -z "${{ inputs.docker_push }}" ]] || echo "DOCKER_PUSH=${{ inputs.docker_push }}" >> $GITHUB_ENV
77+
[[ -z "${{ inputs.piwind_image }}" ]] || echo "PIWIND_IMAGE=${{ inputs.piwind_image }}" >> $GITHUB_ENV
78+
[[ -z "${{ inputs.piwind_branch }}" ]] || echo "PIWIND_BRANCH=${{ inputs.piwind_branch }}" >> $GITHUB_ENV
79+
80+
- name: Set build arg
81+
if: inputs.oasislmf_package != '' || inputs.oasislmf_branch != ''
82+
run: |
83+
if [[ -n "${{ inputs.oasislmf_package }}" && -n "${{ inputs.oasislmf_branch }}" ]]; then
84+
echo "set either 'oasislmf_package' or 'oasislmf_branch' not both"
85+
exit 1
86+
fi
87+
[[ -z "${{ inputs.oasislmf_package }}" ]] || echo "BUILD_PACKAGE=${{ inputs.oasislmf_package }}" >> $GITHUB_ENV
88+
[[ -z "${{ inputs.oasislmf_branch }}" ]] || echo "BUILD_BRANCH=${{ inputs.oasislmf_branch }}" >> $GITHUB_ENV
89+
90+
- name: Github context
91+
run: echo "$GITHUB_CONTEXT"
92+
shell: bash
93+
env:
94+
GITHUB_CONTEXT: ${{ toJson(github) }}
95+
96+
- uses: actions/checkout@v3
97+
with:
98+
repository: OasisLMF/OasisPiWind
99+
ref: ${{ env.PIWIND_BRANCH }}
100+
101+
- name: Login to Docker Hub
102+
if: inputs.docker_push == 'true'
103+
uses: docker/login-action@v2
104+
with:
105+
username: ${{ secrets.DOCKERHUB_USERNAME }}
106+
password: ${{ secrets.DOCKERHUB_TOKEN }}
107+
108+
- name: Set up Docker Buildx
109+
uses: docker/setup-buildx-action@v2
110+
111+
- name: replace default image with input
112+
run: sed -i "/FROM /c\FROM ${{ inputs.from_image }}" docker/Dockerfile.piwind_worker
113+
114+
- name: Build and push
115+
uses: docker/build-push-action@v3
116+
with:
117+
file: ${{ env.DOCKERFILE }}
118+
context: .
119+
push: ${{ env.DOCKER_PUSH }}
120+
tags: ${{ env.PIWIND_IMAGE }}
121+
build-args: |
122+
oasislmf_package=${{ env.BUILD_PACKAGE }}
123+
oasislmf_branch=${{ env.BUILD_BRANCH }}
124+
125+
- name: Store image ref
126+
if: inputs.docker_push == 'true'
127+
id: docker_ref
128+
run: |
129+
echo "image=${{ env.PIWIND_IMAGE }}" >> $GITHUB_OUTPUT
130+
# link image url to run summary
131+
REPO=$(echo ${{ env.PIWIND_IMAGE }} | cut -d: -f1)
132+
TAG=$(echo ${{ env.PIWIND_IMAGE }} | cut -d: -f2)
133+
echo '#### Docker Image: [${{ env.PIWIND_IMAGE }}](https://hub.docker.com/r/'$REPO'/tags?name='$TAG')' >> $GITHUB_STEP_SUMMARY

Diff for: ‎.github/workflows/check_results.yml

+3-64
Original file line numberDiff line numberDiff line change
@@ -1,73 +1,12 @@
11
# =============================================================================
2-
# Workflow description:
3-
# Runs the quantitative tests on the PiWind model and checks for differences in output vs
4-
# https://github.com/OasisLMF/OasisPiWind/tree/master/tests/ci/expected
5-
#
6-
# If the generated output differs from the expected results folder then that step is failed.
7-
# Each test 'case' is executed via the OasisPlatform by running docker-compose
8-
#
9-
# Note: there are two test modes set by the 'build_worker' option
10-
# 1) build_worker=false - Test piwind on the most recent docker image release of 'coreoasis/model_worker'
11-
# 2) build_worker=true - Build a new worker image by git installing an oasislmf branch (set by 'mdk_branch')
12-
#
13-
#
14-
# Stored artifacts:
15-
# 'docker-logs': A zip file containing the docker logs from each container running the oasis stack.
16-
# Any execution errors will generally be stored in 'worker.log' file.
17-
#
18-
# docker-logs
19-
# ├── celery-db.log
20-
# ├── rabbit.log
21-
# ├── server-db.log
22-
# ├── server.log
23-
# ├── worker.log
24-
# └── worker-monitor.log
25-
#
26-
# 'loss-outputs': A zip file containing the generated 'oasis-files' and 'losses'.
27-
# Each test is stored within a sub-dir following the same structure
28-
# as the dir 'OasisPiWind/tests/ci/expected' from the piwind repo.
29-
# These results and logs intended to help with debugging, so are only stored for 5 days.
30-
#
31-
# loss-outputs
32-
# ├── 0_case
33-
# │   ├── input # (Generated ktools files)
34-
# ...
35-
# │   └── output # (output csv files)
36-
# │   ├── gul_S1_eltcalc.csv
37-
# │   └── gul_S1_summary-info.csv
38-
# ...
39-
# ├── 8_case
40-
# │   ├── input
41-
# │   └── output
42-
# └── control_set
43-
# ├── input
44-
# └── output
2+
# Workflow description: THIS WORKFLOW IS REPLACED WITH 'test.yml'
3+
# keep the file here until other oasislmf branch have been
4+
# updated to call the newer script
455
# ==============================================================================
466

477
name: check_results
488

499
on:
50-
push:
51-
workflow_dispatch:
52-
inputs:
53-
build_branch:
54-
description: 'Branch of build script repo - https://github.com/OasisLMF/build'
55-
required: true
56-
default: master
57-
platform_version:
58-
description: 'OasisPlatform Version to test using'
59-
required: true
60-
default: latest
61-
mdk_branch:
62-
description: 'Branch to test from Oasislmf repo - https://github.com/OasisLMF/OasisLMF'
63-
required: true
64-
default: develop
65-
build_worker:
66-
description: 'Build new worker based on "mdk_branch" value'
67-
required: true
68-
default: true
69-
70-
7110
workflow_call:
7211
inputs:
7312
build_branch:

Diff for: ‎.github/workflows/integration.yml

+180
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,180 @@
1+
# This workflow tests the PiWind expected results from 'tests/ci/expected'
2+
# its tested using 'tests/test_piwind_integration.py' which runs a set of
3+
# checks using the OasisLMF API client
4+
# vs the docker-compose file `docker-compose.yml` and ENV variables
5+
# SERVER_IMG: server image name "coreoasis/api_server"
6+
# SERVER_TAG: server image tag "latest"
7+
# WORKER_IMG: worker image name "coreoasis/model_worker"
8+
# WORKER_TAG: worker image tag ""latest
9+
#
10+
11+
name: PiWind Integration
12+
13+
on:
14+
workflow_dispatch:
15+
inputs:
16+
server_image:
17+
description: 'Server image [coreoasis/api_server]'
18+
required: false
19+
default: 'coreoasis/model_worker'
20+
server_tag:
21+
description: 'Server tag [semvar]'
22+
required: false
23+
default: 'latest'
24+
worker_image:
25+
description: 'Worker image [coreoasis/model_worker]'
26+
required: false
27+
default: 'coreoasis/model_worker'
28+
worker_tag:
29+
description: 'Worker image [semvar]'
30+
required: false
31+
default: 'latest'
32+
debug_mode:
33+
description: 'Enable verbose logging for containers [0, 1]'
34+
required: false
35+
default: '0'
36+
37+
# If set build a piwind worker with / branch package
38+
# otherwise use the default worker image
39+
oasislmf_branch:
40+
description: "Branch from oasislmf to test piwind with [git ref]"
41+
required: false
42+
oasislmf_package:
43+
description: "Oasislmf package build to test [pkg_filename]"
44+
required: false
45+
46+
workflow_call:
47+
inputs:
48+
piwind_branch:
49+
description: 'PiWind branch to test'
50+
type: string
51+
server_image:
52+
description: 'Server image [coreoasis/api_server]'
53+
required: false
54+
type: string
55+
server_tag:
56+
description: 'Server tag [semvar]'
57+
required: false
58+
type: string
59+
worker_image:
60+
description: 'Worker image [coreoasis/model_worker]'
61+
required: false
62+
type: string
63+
worker_tag:
64+
description: 'Worker image [semvar]'
65+
required: false
66+
type: string
67+
debug_mode:
68+
description: 'Enable verbose logging for containers [0, 1]'
69+
required: false
70+
type: string
71+
72+
# If set build a piwind worker with / branch package
73+
# otherwise use the default worker image
74+
oasislmf_branch:
75+
description: "Branch from oasislmf to test piwind with [git ref]"
76+
required: false
77+
type: string
78+
oasislmf_package:
79+
description: "Oasislmf package build to test [pkg_filename]"
80+
required: false
81+
type: string
82+
83+
jobs:
84+
check_results:
85+
runs-on: ubuntu-latest
86+
env:
87+
DOCKERFILE: 'docker/Dockerfile.piwind_worker'
88+
PIWIND_BRANCH: ${{ github.ref_name }}
89+
SERVER_IMG: 'coreoasis/api_server'
90+
SERVER_TAG: 'latest'
91+
WORKER_IMG: 'coreoasis/model_worker'
92+
WORKER_TAG: 'latest'
93+
DEBUG: 0
94+
95+
steps:
96+
- name: Set inputs
97+
run: |
98+
[[ -z "${{ inputs.piwind_branch }}" ]] || echo "PIWIND_BRANCH=${{ inputs.piwind_branch }}" >> $GITHUB_ENV
99+
[[ -z "${{ inputs.server_image }}" ]] || echo "SERVER_IMG=${{ inputs.server_image }}" >> $GITHUB_ENV
100+
[[ -z "${{ inputs.server_tag }}" ]] || echo "SERVER_TAG=${{ inputs.server_tag }}" >> $GITHUB_ENV
101+
[[ -z "${{ inputs.worker_image }}" ]] || echo "WORKER_IMG=${{ inputs.worker_image }}" >> $GITHUB_ENV
102+
[[ -z "${{ inputs.worker_tag }}" ]] || echo "WORKER_TAG=${{ inputs.worker_tag }}" >> $GITHUB_ENV
103+
[[ -z "${{ inputs.debug_mode }}" ]] || echo "DEBUG=${{ inputs.debug_mode }}" >> $GITHUB_ENV
104+
105+
- name: clone PiWind repository
106+
uses: actions/checkout@v3
107+
with:
108+
repository: OasisLMF/OasisPiWind
109+
ref: ${{ env.PIWIND_BRANCH }}
110+
111+
- name: Set build arg
112+
if: inputs.oasislmf_package != '' || inputs.oasislmf_branch != ''
113+
run: |
114+
if [[ -n "${{ inputs.oasislmf_package }}" && -n "${{ inputs.oasislmf_branch }}" ]]; then
115+
echo "set either 'oasislmf_package' or 'oasislmf_branch' not both"
116+
exit 1
117+
fi
118+
119+
- name: Set up Docker Buildx
120+
if: inputs.oasislmf_package != '' || inputs.oasislmf_branch != ''
121+
uses: docker/setup-buildx-action@v2
122+
123+
- name: replace default image with input
124+
if: inputs.oasislmf_package != '' || inputs.oasislmf_branch != ''
125+
run: sed -i "/FROM /c\FROM ${{ env.WORKER_IMG }}:${{ env.WORKER_TAG }}" ${{ env.DOCKERFILE }}
126+
127+
- name: Build and push
128+
if: inputs.oasislmf_package != '' || inputs.oasislmf_branch != ''
129+
uses: docker/build-push-action@v3
130+
with:
131+
file: ${{ env.DOCKERFILE }}
132+
context: .
133+
load: true # needed for local testing
134+
push: false
135+
tags: "${{ env.WORKER_IMG }}:piwind-test-${{ github.sha }}"
136+
build-args: |
137+
oasislmf_package=${{ inputs.oasislmf_package }}
138+
oasislmf_branch=${{ inputs.oasislmf_branch }}
139+
140+
- name: Override Worker image env
141+
if: inputs.oasislmf_package != '' || inputs.oasislmf_branch != ''
142+
run: echo "WORKER_TAG=piwind-test-${{ github.sha }}" >> $GITHUB_ENV
143+
144+
- name: Setup Python
145+
uses: actions/setup-python@v4
146+
with:
147+
python-version: '3.10'
148+
149+
- name: install test deps
150+
run: |
151+
python -m pip install --upgrade pip
152+
pip install pip-tools
153+
pip-compile tests/requirements.in -o tests/requirements.txt
154+
pip install -r tests/requirements.txt
155+
156+
- name: Run Checks
157+
run: pytest -v --use-running-containers
158+
159+
- name: Create docker logs
160+
if: always()
161+
run: |
162+
mkdir -p docker_logs
163+
docker-compose ps | tail -n +3 | awk '{print $1}' | xargs -I CONTAINER bash -c "docker logs CONTAINER > docker_logs/CONTAINER.log"
164+
docker-compose down
165+
166+
- name: Store docker logs
167+
if: always()
168+
uses: actions/upload-artifact@v3
169+
with:
170+
name: docker-logs
171+
path: ./docker_logs
172+
retention-days: 3
173+
174+
- name: Store result data
175+
if: always()
176+
uses: actions/upload-artifact@v3
177+
with:
178+
name: output-tars
179+
path: ./tests/result/
180+
retention-days: 3

0 commit comments

Comments
 (0)