Skip to content

Update LCFS, WAS, and ETB prerequisites #413

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MaxGhenis merged 4 commits into from
May 24, 2026
Merged

Update LCFS, WAS, and ETB prerequisites #413

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
53c75eb
Update private UK survey prerequisites
MaxGhenis May 24, 2026
d5d95b7
Use current UKDS tab zip layouts
MaxGhenis May 24, 2026
89fa2e7
Handle refreshed LCFS and ETB aggregate shifts
MaxGhenis May 24, 2026
5c3d0bc
Use locked uv environment in data CI
MaxGhenis May 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/pull_request.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,14 +64,14 @@ jobs:
uses: actions/setup-python@v6
with:
python-version: 3.13
- name: Install package
run: uv pip install -e ".[dev]" --system
- name: Sync locked environment
run: uv sync --frozen --all-extras
- name: Download data inputs
run: make download
run: uv run --frozen make download
env:
HUGGING_FACE_TOKEN: ${{ secrets.HUGGING_FACE_TOKEN }}
- name: Build datasets
run: make data
run: uv run --frozen make data
env:
TESTING: "1"
- name: Save calibration log (constituencies)
Expand All @@ -87,4 +87,4 @@ jobs:
path: la_calibration_log.csv

- name: Run tests
run: make test
run: uv run --frozen make test
12 changes: 6 additions & 6 deletions .github/workflows/push.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,14 +52,14 @@ jobs:
with:
workload_identity_provider: "projects/322898545428/locations/global/workloadIdentityPools/policyengine-research-id-pool/providers/prod-github-provider"
service_account: "policyengine-research@policyengine-research.iam.gserviceaccount.com"
- name: Install package
run: uv pip install -e ".[dev]" --system
- name: Sync locked environment
run: uv sync --frozen --all-extras
- name: Download data inputs
run: make download
run: uv run --frozen make download
env:
HUGGING_FACE_TOKEN: ${{ secrets.HUGGING_FACE_TOKEN }}
- name: Build datasets
run: make data
run: uv run --frozen make data
- name: Save calibration log (constituencies)
uses: actions/upload-artifact@v7
with:
Expand All @@ -72,9 +72,9 @@ jobs:
name: la_calibration_log.csv
path: la_calibration_log.csv
- name: Run tests
run: make test
run: uv run --frozen make test
- name: Upload data
run: make upload
run: uv run --frozen make upload
env:
HUGGING_FACE_TOKEN: ${{ secrets.HUGGING_FACE_TOKEN }}
- name: Publish a git tag
Expand Down
12 changes: 12 additions & 0 deletions policyengine_uk_data/datasets/create_datasets.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ def main():
"Clone and assign OA geography",
"Calibrate constituency weights",
"Calibrate local authority weights",
"Calibrate public service aggregates",
"Calibrate fuel litres",
"Save final dataset",
"Create tiny datasets",
Expand Down Expand Up @@ -288,6 +289,17 @@ def main():
)
update_dataset(materialize_step, "completed")

update_dataset("Calibrate public service aggregates", "processing")
from policyengine_uk_data.datasets.imputations.services.services import (
calibrate_rail_subsidy_spending,
)

calibrate_rail_subsidy_spending(
frs_calibrated,
frs_release.calibration_year,
)
update_dataset("Calibrate public service aggregates", "completed")

update_dataset("Calibrate fuel litres", "processing")
from policyengine_uk_data.datasets.imputations.consumption import (
calibrate_dataset_fuel_litre_proxies_to_road_fuel,
Expand Down
Loading