Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement catalogue-graph recorder service #5905

Closed
kenoir opened this issue Feb 5, 2025 · 0 comments
Closed

Implement catalogue-graph recorder service #5905

kenoir opened this issue Feb 5, 2025 · 0 comments

Comments

@kenoir
Copy link
Contributor

kenoir commented Feb 5, 2025

Moving towards replacing the existing concepts-pipeline we need a service that does the job of loading catalogue-pipeline identified concepts into an elasticsearch index in a way that supports re-implementing API responses like this: https://api.wellcomecollection.org/catalogue/v2/concepts/vk6nj4mt

We intend to avoid providing the same_as functionality in the existing API by removing all duplicates as described here: #5845

This functionality is currently provided by the recorder service in the concepts-pipeline repository. We should create a new service and ES index in the catalogue-pipeline repository intended to replace the existing service.

We will need a mechanism to keep the ES index up to date with the latest changes in the catalogue pipeline but this could be a regular batch load rather than an event based approach.

This may also be an opportunity to use the Serverless Elasticsearch offering to provide our ES index.

The scope of this ticket is to reproduce only existing functionality and not yet expose additional concept relationships present in the graph database.

@kenoir kenoir converted this from a draft issue Feb 5, 2025
@kenoir kenoir changed the title Implement catalogue-graph merger service Implement catalogue-graph recorder service Feb 5, 2025
@kenoir kenoir moved this from Sprint 5 February to Next in Digital platform Feb 5, 2025
@kenoir kenoir moved this from Next to Sprint 19 February in Digital platform Feb 18, 2025
@kenoir kenoir added the Size: 8 label Feb 19, 2025
@kenoir kenoir removed the Size: 8 label Feb 19, 2025
@kenoir kenoir moved this from Sprint 19 February to Next in Digital platform Feb 19, 2025
@kenoir kenoir closed this as completed Mar 11, 2025
@github-project-automation github-project-automation bot moved this from Next to Done in Digital platform Mar 11, 2025
@pollecuttn pollecuttn moved this from Done to Archive in Digital platform Mar 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Archive
Development

No branches or pull requests

2 participants