3742 create data backend #64
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
add snapshot method clean up add create index add opensearch load add restore, fix health check fix parquet globbing, add load all step, make opensearch client based on host/port fix specs, move snapshot repo conf out of start task add scratchpad to config add dataset config added more indices, resolve _id from config workaround for coloc refactor _id field assignment add explode_datasets fix for create index race condition add variant mappings rearrange pipeline tidy up add evidence, slight improvement to load generator fix paths formatting allow path to be empty
add gcloud fix create disk image update dataset paths
fix storage location fix snapshot fix snapshot fix disk image fix disk image just do snapshot fix snapshot step in config add labels fix labels increase snapshot timeout Update README.md
add clickhouse stop step move clickhouse config add skeleton for loading to clickhouse
Update config.yaml add bigquery add bigquery dev step Delete bigquery.py update datatsets and add docstrings to bigquery.py fix bq access update bigquery datasets, enable hive partitioning update bigquery tables and fix hive partitioning
added clickhouse load, updated w2v sql to drop log table
add timeout reset to health check reorganise services, added run_container helper method
update doc strings rename step
… image if not exists increase default timeout for opensearch build the image only if the image is requested and not found
This was
linked to
issues
Apr 16, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added the steps to create the platform backend. Resolves opentargets/issues#3742 and opentargets/issues#3743
The following steps have been added: