Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

backport_records for multiple collections is verbose #733

Closed
leplatrem opened this issue Jan 14, 2025 · 1 comment
Closed

backport_records for multiple collections is verbose #733

leplatrem opened this issue Jan 14, 2025 · 1 comment

Comments

@leplatrem
Copy link
Contributor

leplatrem commented Jan 14, 2025

Currently, in order to setup the backport_record cronjob that will copy records from one collection to another using filters, it takes all these config lines:

  backport_records_normandy:
    hasExternalSecrets: true
    schedule: "*/10 * * * *"
    suspend: false
    image:
      repository: us-docker.pkg.dev/moz-fx-remote-settings-prod/remote-settings-prod/remote-settings-lambdas
      tag: latest
      pullPolicy: Always

    env:
      - name: "BACKPORT_RECORDS_SOURCE_BUCKET"
        value: "main"

      - name: "BACKPORT_RECORDS_DEST_BUCKET"
        value: "main-workspace"

      - name: "BACKPORT_RECORDS_SOURCE_COLLECTION"
        value: "normandy-recipes-capabilities"

      - name: "BACKPORT_RECORDS_DEST_COLLECTION"
        value: "normandy-recipes"

      - name: "BACKPORT_RECORDS_SOURCE_FILTERS"
        value: "{\"recipe.uses_only_baseline_capabilities\": true}"

      - name: "SENTRY_ENV"
        value: "prod"

    args:
      - backport_records

If we want to onboard for examples 10 cronjobs that will breakdown records by country/regions. This is going to be extremely verbose.

I propose that we introduce a new input var that can combine them all:

BACKPORT_RECORDS_MAPPINGS="main/normandy-recipes-capabilities?recipe.uses_only_baseline_capabilities=true -> main-workspace/normandy-recipes

and one per line for multiple values:

env:
  - name: BACKPORT_RECORDS_MAPPINGS
    value: |
      main/shops?contains_regions=["asia"] -> main-workspace/shops-asia
      main/shops?contains_regions=["europe"] -> main-workspace/shops-europe
      main/shops?contains_regions=["north-america", "central-america", "south-america"] -> main-workspace/shops-america
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant