DC1 West Round 1 Prerequisites

[lliming]

• DC1-Round1 event streams
• Publisher app publishes to West catalogue
• West Transaction API configured to use DC1-Round1 event streams
• West Event Consumer configured to use DC1-Round1 event streams
• Empty index
• West Event Consumer configured to use DC1-Round1 index
• Set of 1000 distinct CMIP6 datasets (no dups, including East's datasets)

[sashakames]

Selection query: https://esgf-node.llnl.gov/esg-search/search?project=CMIP6&limit=500&type=Dataset&format=application%2Fsolr%2Bjson&latest=true&activity_id=RFMIP,CMIP,C4MIP,LS3MIP,LUMIP&data_node=esgf-data2.llnl.gov&institution_id=E3SM-Project,RUBISCO,UCI,UCSB&variable_id=tas,ta,sftlf,rh,pr&fields=instance_id

[sturoscy-personal]

• West Transaction API configured to use DC1-Round1 event streams
  • Configured a west-playground and deployed to AWS ECS
  • POST to https://transaction-api.stac.esgf-west.org. Health check here
• West Event Consumer configured to use DC1-Round1 event streams
  • https://github.com/esgf2-us/west-consumer/tree/add-containers
  • Currently deployed to EC2, but will be moving to ECS with a similar build/deploy process as the Transaction API
• Empty index
  • Globus Search team has created several empty indices for us to use. We will be posting a list of indices with indications of how and where they are being used
• West Event Consumer configured to use DC1-Round1 index
  • Need to deploy to ECS, but the service is configured to use the DC1-Round1 index (more info to follow in the index list mentioned above)

[sturoscy-personal]

Selection query: https://esgf-node.llnl.gov/esg-search/search?project=CMIP6&limit=500&type=Dataset&format=application%2Fsolr%2Bjson&latest=true&activity_id=RFMIP,CMIP,C4MIP,LS3MIP,LUMIP&data_node=esgf-data2.llnl.gov&institution_id=E3SM-Project,RUBISCO,UCI,UCSB&variable_id=tas,ta,sftlf,rh,pr&field=instance_id

@sashakames The above link is giving a Bad Request? Invalid Parameter=field

[sashakames]

Fixed, above, had a typo. I ran a script to translate the query to paths. I'll upload that to the repo in a few.

[sashakames]

See: https://github.com/ESGF/esgf-roadmap/pull/47/files

[sturoscy-personal]

For the "Set of 1000 distinct CMIP6 datasets" please see Sasha's comment and pull request above. He identified 500 E3SM datasets that Lukasz replicated from LLNL to ALCF.