Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Populate dev with an updated LungMAP snapshots from prod #6841

Open
dsotirho-ucsc opened this issue Jan 22, 2025 · 4 comments
Open

Populate dev with an updated LungMAP snapshots from prod #6841

dsotirho-ucsc opened this issue Jan 22, 2025 · 4 comments
Assignees
Labels
- [priority] Medium data [subject] Data or metadata [use of this label is uncommon] debt [type] A defect incurring continued engineering cost enh [type] New feature or request orange [process] Done by the Azul team spike:2 [process] Spike estimate of two points

Comments

@dsotirho-ucsc
Copy link
Contributor

dsotirho-ucsc commented Jan 22, 2025

The following LungMAP snapshots will be removed from dev as part of #6516 (PR #6747) due to invalid metadata schema locations:

  • lungmap_dev_20037472ea1d4ddb9cd356a11a6f0f76__20220401_20220404_lm2
  • lungmap_dev_f899709cae2c4bb988f0131142e6c7ec__20220401_20220629_lm2

These snapshots should be replaced with updated snapshots from prod.

@dsotirho-ucsc dsotirho-ucsc added the orange [process] Done by the Azul team label Jan 22, 2025
@dsotirho-ucsc
Copy link
Contributor Author

dsotirho-ucsc commented Jan 22, 2025

Spike to ask the Broad if they can copy all current snapshots from prod to dev and remove the ones we currently use on dev.

@dsotirho-ucsc dsotirho-ucsc self-assigned this Jan 22, 2025
@dsotirho-ucsc dsotirho-ucsc added enh [type] New feature or request data [subject] Data or metadata [use of this label is uncommon] debt [type] A defect incurring continued engineering cost - [priority] Medium spike:2 [process] Spike estimate of two points labels Jan 22, 2025
@dsotirho-ucsc
Copy link
Contributor Author

Spike to ask the Broad if they can copy all current snapshots from prod to dev and remove the ones we currently use on dev.

https://humancellatlas.slack.com/archives/C01DQFA5F5Y/p1737665603936329

@dsotirho-ucsc
Copy link
Contributor Author

Reply from Nathan Calvanese:

I think we’ve discussed this one a little bit in the past. We don’t have the capability currently to mirror data across environments, nor do we have the ability to move data from anywhere in the production pipeline to the development pipeline. Our only option to accomplish this would be to have the wranglers re-state all of the datasets and process them through dev

@dsotirho-ucsc
Copy link
Contributor Author

@hannes-ucsc: "The Broad indicates that they cannot copy snapshots from prod to dev, and that the only recourse is to provide staging areas for them to import into TDR dev. Assignee to coordinate with the involved parties (Broad & LungMAP) in order to create a mirror of catalog lm8 in dev. This will require creating or reactivating the staging areas that were imported into TDR prod for lm8."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
- [priority] Medium data [subject] Data or metadata [use of this label is uncommon] debt [type] A defect incurring continued engineering cost enh [type] New feature or request orange [process] Done by the Azul team spike:2 [process] Spike estimate of two points
Projects
None yet
Development

No branches or pull requests

2 participants