Skip to content

Import catalogus data into collection #141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
lukavdplas opened this issue May 11, 2023 · 1 comment
Open

Import catalogus data into collection #141

lukavdplas opened this issue May 11, 2023 · 1 comment
Assignees
@jgonggrijp

Comments

@lukavdplas
Copy link
Contributor

Users should be able to import results from a catalogus (CentreForDigitalHumanities/edpop-explorer#13) into a collection.

They can do this in two ways:

  • Manually select records -> the frontend sends a list of IDs
  • Select everything from a query (which may not all be loaded into the frontend) -> the frontend sends the query

This task is best handled asynchronously, i.e. through celery.
@lukavdplas lukavdplas mentioned this issue May 11, 2023
Open
@lukavdplas lukavdplas added this to EDPOP Feb 8, 2024
@lukavdplas lukavdplas moved this to Todo in EDPOP Feb 8, 2024
@lukavdplas lukavdplas self-assigned this Feb 8, 2024
@jgonggrijp
Copy link
Contributor

@lukavdplas via Teams:

ok, ik heb even gekeken en het laatste wat ik rondom de records in een collectie had gedaan stond in #195. Daar heb ik nu nog wat dingen afgewerkt, dus ik denk dat die gemerged kan worden. Dan is de data / api dus ook klaar om te worden uitgebreid met records toevoegen

@lukavdplas lukavdplas assigned jgonggrijp and unassigned lukavdplas Dec 11, 2024
@jgonggrijp jgonggrijp moved this from Todo to In Progress in EDPOP Dec 12, 2024
jgonggrijp added a commit that referenced this issue Jan 13, 2025
@jgonggrijp
Add the collect app to the INSTALLED_APPS (#141 #142)
9cb5ddf 
This appears to have been an omission from #181.
jgonggrijp added a commit that referenced this issue Jan 13, 2025
@jgonggrijp
Add an old-fashioned Django model for record garbage collection (#141 #…
0c9ba80 
…142)

Using the Postgres RDBMS for this as it is probably cleaner (and more
convenient) to keep this out of the triplestore.
jgonggrijp added a commit that referenced this issue Feb 14, 2025
@jgonggrijp
Revert "Add an old-fashioned Django model for record garbage collecti…
2095ca4 
…on (#141 #142)"

At second thought, I think I should store this information in the
triplestore. I discussed this with @tijmenbaarda and he agrees.

This reverts commit 0c9ba80.
jgonggrijp added a commit that referenced this issue Feb 14, 2025
@jgonggrijp
Track records for garbage collection when adding/removing to triplestore
68ebd36 
Ref #141
jgonggrijp added a commit that referenced this issue Feb 17, 2025
@jgonggrijp
Track records for garbage collection when adding/removing to triplestore
0425ee6 
Ref #141
jgonggrijp added a commit that referenced this issue Feb 17, 2025
@jgonggrijp
Implement garbage collection (#141)
2debe32
jgonggrijp added a commit that referenced this issue Mar 6, 2025
@jgonggrijp
Use rdfs:member instead of as:items in CollectionMembersField
58930be 
Necessary for #141 and #142. See also
CentreForDigitalHumanities/edpop-collection-ontology#4.
jgonggrijp added a commit that referenced this issue Mar 7, 2025
@jgonggrijp
Write add/remove SPARQL updates for CollectionMembersField (#141 #142)
555bdc0
jgonggrijp added a commit that referenced this issue Mar 12, 2025
@jgonggrijp
Write add/remove SPARQL updates for CollectionMembersField (#141 #142)
2ded36e
jgonggrijp added a commit that referenced this issue Mar 12, 2025
@jgonggrijp
Add sparql_multivalues utility function (#141 #142)
75664a9
jgonggrijp added a commit that referenced this issue Mar 12, 2025
@jgonggrijp
Make RDFField internal methods non-abstract (#141)
dbb2280 
If marked with @AbstractMethod, it is impossible to derive classes
that don't define these methods. The alternative, exception-raising
approach taken here ensures that the methods can be omitted if set and
clear are defined instead.
jgonggrijp added a commit that referenced this issue Mar 12, 2025
@jgonggrijp
Outfactor reusable bits from test_collection_model to fixtures (#141 #…
e77dd62 
…142)
jgonggrijp added a commit that referenced this issue Mar 12, 2025
@jgonggrijp
Rewrite set/clear/add/remove logic for collection records (#141 #142)
4ec5c9b
jgonggrijp added a commit that referenced this issue Mar 12, 2025
@jgonggrijp
Outfactor common logic in CollectionMembersField methods (#141 #142)
5437246
jgonggrijp added a commit that referenced this issue Mar 13, 2025
@jgonggrijp
Merge collection RDFModel test fixtures into conftest (#141 #142)
b227119
jgonggrijp added a commit that referenced this issue Mar 13, 2025
@jgonggrijp
Return number of added records from EDPOPCollection.add_records (#141)
dc2d9ec
jgonggrijp added a commit that referenced this issue Mar 13, 2025
@jgonggrijp
Define new record-adding viewset based on updated Collection model
add0eb6 
Ref #141 #142
jgonggrijp added a commit that referenced this issue Mar 13, 2025
@jgonggrijp
Delete the old record-adding viewset (#141)
c6bea51
jgonggrijp added a commit that referenced this issue Mar 13, 2025
@jgonggrijp
Use name rather than obsolete description in collection picker (#141 #…
bff8e72 
…142)
jgonggrijp added a commit that referenced this issue Mar 13, 2025
@jgonggrijp
Connect frontend to the new record addition API (#141 #142)
e584a9a
jgonggrijp added a commit that referenced this issue Mar 17, 2025
@jgonggrijp
Add the collect app to the INSTALLED_APPS (#141 #142)
9f48bce 
This appears to have been an omission from #181.
jgonggrijp added a commit that referenced this issue Mar 17, 2025
@jgonggrijp
Add an old-fashioned Django model for record garbage collection (#141 #…
ad0af7a 
…142)

Using the Postgres RDBMS for this as it is probably cleaner (and more
convenient) to keep this out of the triplestore.
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Use enforest instead of nestSubject, allow JsonLdNestedCollection wit…
6a626e2 
…hout targetClass

Ref #141 #142 and previous commit
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Drop nestSubject in favor of enforest (#141 #142)
86800e1
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Restore ability to list collection contents in frontend (#141 #195)
0b113b0
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Add the collect app to the INSTALLED_APPS (#141 #142)
4cd2521 
This appears to have been an omission from #181.
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Add an old-fashioned Django model for record garbage collection (#141 #…
9ffb302 
…142)

Using the Postgres RDBMS for this as it is probably cleaner (and more
convenient) to keep this out of the triplestore.
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Revert "Add an old-fashioned Django model for record garbage collecti…
daf6e10 
…on (#141 #142)"

At second thought, I think I should store this information in the
triplestore. I discussed this with @tijmenbaarda and he agrees.

This reverts commit 0c9ba80.
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Track records for garbage collection when adding/removing to triplestore
c1798e0 
Ref #141
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Implement garbage collection (#141)
9921002
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Use rdfs:member instead of as:items in CollectionMembersField
d7693bd 
Necessary for #141 and #142. See also
CentreForDigitalHumanities/edpop-collection-ontology#4.
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Write add/remove SPARQL updates for CollectionMembersField (#141 #142)
ff700f7
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Add sparql_multivalues utility function (#141 #142)
148095e
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Make RDFField internal methods non-abstract (#141)
ff3b334 
If marked with @AbstractMethod, it is impossible to derive classes
that don't define these methods. The alternative, exception-raising
approach taken here ensures that the methods can be omitted if set and
clear are defined instead.
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Outfactor reusable bits from test_collection_model to fixtures (#141 #…
ca7beea 
…142)
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Rewrite set/clear/add/remove logic for collection records (#141 #142)
9d2822d
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Outfactor common logic in CollectionMembersField methods (#141 #142)
d19e16e
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Merge collection RDFModel test fixtures into conftest (#141 #142)
f9ca65f
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Return number of added records from EDPOPCollection.add_records (#141)
ecb3b11
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Define new record-adding viewset based on updated Collection model
c46d870 
Ref #141 #142
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Delete the old record-adding viewset (#141)
ba43661
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Use updated field names in collection picker template (#141 #142)
e9a3300
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Connect frontend to the new record addition API (#141 #142)
21c68fe
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Reinstate RecordListView#currentSelection (#141 #142)
4157b06 
This method fell off in 90aaf9e.
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Reorganize how records are submitted to collections in frontend (#141 #…
581dc5e 
…142)

This restores the ability to add records to collections and also
improves the separation of concerns. The VRECollectionView is
responsible for selecting collections and for detecting when the user
wants to submit. Its parent views are responsible for passing the URIs
of the models that should be added to those collections.
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Adjust collection records endpoint (#141 #142 #195)
5ea7a00
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Render collection records in JSON-LD (#141 #195)
adbb1a9
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Add enforesting function for nesting a whole graph of JSON-LD data
14b6eff 
This supersedes the nestSubject function, which handled one subject at
a time. The enforest function can be more efficient because it caches
the previously obtained results for other subjects in the graph, and
it has the option to automatically detect top-level subjects.

Cf #141 #195 #242 #183
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Use enforest instead of nestSubject, allow JsonLdNestedCollection wit…
220f411 
…hout targetClass

Ref #141 #142 and previous commit
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Drop nestSubject in favor of enforest (#141 #142)
9139059
jgonggrijp added a commit that referenced this issue Mar 20, 2025
@jgonggrijp
Restore ability to list collection contents in frontend (#141 #195)
30cd7c5
@jgonggrijp jgonggrijp mentioned this issue Mar 20, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jgonggrijp jgonggrijp

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jgonggrijp @lukavdplas