Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ISSUE-208: Actions, new processing logic. EAD Sync Plugin and Ingest Sanity #213

Draft
wants to merge 54 commits into
base: 0.9.0
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
0dda72f
Adds Log dowload controller
DiegoPino Aug 8, 2024
f9a1bb0
Refactor of getProcessedAmiSetNodeUUids
DiegoPino Aug 8, 2024
70f3b3a
Report form now can use the download Controller
DiegoPino Aug 8, 2024
a2b1997
add `label` property to VBO actions (new in latest)
DiegoPino Aug 8, 2024
115f74f
First pass on using the CSV expander for deleting via an Action
DiegoPino Aug 8, 2024
a4cf1b1
First pass on the Action ADO queworker & and CSV to action
DiegoPino Aug 8, 2024
07c0fe0
small extra for the ADO queworker and coding standard
DiegoPino Aug 8, 2024
16a29de
Actually do some work
DiegoPino Aug 8, 2024
0e9ee47
reduce the batch size for now
DiegoPino Aug 8, 2024
ab1f5d5
VBO actions using AMI UUIDs. Almost there. Probably another 20 lines …
DiegoPino Oct 11, 2024
11ef13e
PHP doc type
DiegoPino Dec 9, 2024
37207b5
First pass on using the CSV expander for deleting via an Action
DiegoPino Aug 8, 2024
d432e70
First pass on the Action ADO queworker & and CSV to action
DiegoPino Aug 8, 2024
de9b3e0
Actually do some work
DiegoPino Aug 8, 2024
1e5ef7f
Prepping for deprecations and new support methods for ZIPs
DiegoPino Dec 10, 2024
26cae4e
Small deprecation/fixes on the Solr importer + PHP docs (copy pastas …
DiegoPino Dec 10, 2024
621e74f
The EAD Sync from XML plugin: Not usable yet. This code is WIP
DiegoPino Dec 10, 2024
d1ec87d
And i remembered why i set it up as public
DiegoPino Dec 10, 2024
dc65172
More code, less failures for EAD SYNC ... Still very much WIP
DiegoPino Dec 10, 2024
d0ae06f
More code cleanup for EADSync
DiegoPino Dec 10, 2024
090543a
Tiny wins, tiny wishes
DiegoPino Dec 11, 2024
04b299f
EAD Sync normalize based on accumulated headers before CSV output
DiegoPino Dec 12, 2024
d54e7e8
Allows ::csv_touch to also save on temporary://
DiegoPino Dec 12, 2024
7ce0260
More cleanup and fixes on EADSync getData() and Batch Finalizing method
DiegoPino Dec 12, 2024
5d02c03
EAD Sync works
DiegoPino Dec 12, 2024
3061d98
Pre-checks if every ADO is there/not and updates 'ami_sync_op' column
DiegoPino Dec 12, 2024
7e27d60
process form for Sync Op
DiegoPino Dec 13, 2024
b0fe7f3
Wrong array entry
DiegoPino Dec 13, 2024
746ebbc
Refactor CSV expander to deal with per-row secondary OPs based on Syn…
DiegoPino Dec 13, 2024
f4cc4a1
Queue worker aware now of sync.
DiegoPino Dec 13, 2024
a8fc047
adds 'sync' to valid ops
DiegoPino Dec 13, 2024
2aafe50
Sync requires even more tapping into processing sub logic
DiegoPino Dec 13, 2024
5edc0ba
Wrong header accumulation
DiegoPino Dec 13, 2024
95ca0ec
Match also on direct process the sync logic
DiegoPino Dec 13, 2024
fb911df
Small refine/better code for sync logic on the CSV expander
DiegoPino Dec 13, 2024
e41a2f1
More readable way of managing sync v/s secondary ops
DiegoPino Dec 13, 2024
e400e52
Gosh... ok, the XML goes into documents
DiegoPino Dec 13, 2024
10c96b1
Ask Allison about this
DiegoPino Dec 13, 2024
68cd305
Remove error logs
DiegoPino Dec 13, 2024
0371c64
For some reason ?? [] does not allow by reference (really makes sense)
DiegoPino Dec 15, 2024
f6c846b
Make failure of Action an error
DiegoPino Dec 15, 2024
7cf3fad
Remove error logs
DiegoPino Dec 15, 2024
41289c1
So messy. This should work better now
DiegoPino Dec 15, 2024
c4d1e32
Fixes NULLs coming in from Spreadsheets and empty columns
DiegoPino Feb 3, 2025
f5c8189
Fixes "process now" for nested/Sync/EAD sets
DiegoPino Feb 11, 2025
9fff6e0
make ZIP required for ead_sync, change the description too
DiegoPino Feb 11, 2025
ffd8d3b
remove checkbox for "unpublish" we don't do that anymore
DiegoPino Feb 11, 2025
08cf3d5
Error (Diego!). When new XML removes all (generates no containers)
DiegoPino Feb 11, 2025
36e3c9a
Filter out dot files. Mac dot files are also XML files!
DiegoPino Feb 11, 2025
1901f40
Move to 'document' instead of 'documents' bc we already have "history…
DiegoPino Feb 11, 2025
03913ce
Just a comma in a comment.
DiegoPino Feb 11, 2025
69ac9a0
Force all headers to exist, even if all are just delete/etc
DiegoPino Feb 12, 2025
67eafd3
better Zip extraction filter by extension
DiegoPino Feb 12, 2025
d83eee4
Context finished v/s Actual Files to process v/s no files was wrong
DiegoPino Feb 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions ami.links.task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,20 +39,26 @@ ami_set_entity.delete_process_form:
title: Delete Processed ADOs
weight: 12

ami_set_entity.action_process_form:
route_name: entity.ami_set_entity.action_process_form
base_route: entity.ami_set_entity.canonical
title: Run Action on Processed ADOs
weight: 13

ami_set_entity.reconcile_form:
route_name: entity.ami_set_entity.reconcile_form
base_route: entity.ami_set_entity.canonical
title: Reconcile LoD
weight: 13
weight: 14

entity.ami_set_entity.reconcileedit_form:
route_name: entity.ami_set_entity.reconcileedit_form
base_route: entity.ami_set_entity.canonical
title: Edit Reconciled LoD
weight: 14
weight: 15

entity.ami_set_entity.report_form:
route_name: entity.ami_set_entity.report_form
base_route: entity.ami_set_entity.canonical
title: Reports
weight: 15
weight: 16
5 changes: 5 additions & 0 deletions ami.permissions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,5 +37,10 @@ deleteados amiset entity:
restrict access: TRUE
deleteados own amiset entity:
title: 'Delete ADOs generated through own AMI Set Entities'
actionados amiset entity:
title: 'Run Action on ADOs referenced on any AMI Set Entities'
restrict access: TRUE
actionados own amiset entity:
title: 'Run Action on ADOs referenced on own AMI Set Entities'
override file destination ami entity:
title: 'Override default persistent file destination and naming for Files ingested via AMI Set Entities. This permission breaks how Archipelago conceives file preservation as a core concern so out of the box will not trigger any chance without enabled a Drupal settings.php global option. See Documentation for this'
21 changes: 21 additions & 0 deletions ami.routing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,15 @@ entity.ami_set_entity.delete_process_form:
requirements:
_entity_access: 'ami_set_entity.deleteados'

entity.ami_set_entity.action_process_form:
path: '/amiset/{ami_set_entity}/actionprocessed'
defaults:
_entity_form: ami_set_entity.actionprocessed
_title: 'Run Action on Ami Set'
requirements:
_entity_access: 'ami_set_entity.actionados'


entity.ami_set_entity.reconcile_form:
path: '/amiset/{ami_set_entity}/reconcile'
defaults:
Expand Down Expand Up @@ -123,3 +132,15 @@ ami.rowsbylabel.autocomplete:
_entity_access: 'ami_set_entity.view'
_permission: 'access content'
_csrf_token: 'TRUE'
ami.download_log:
path: '/amiset/{ami_set_entity}/report/download'
options:
parameters:
ami_set_entity:
type: 'entity:ami_set_entity'
defaults:
_controller: '\Drupal\ami\Entity\Controller\amiSetLogDownloadController::download'
requirements:
_entity_access: 'ami_set_entity.process'
_permission: 'access content'
_csrf_token: 'TRUE'
6 changes: 4 additions & 2 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
}
],
"require": {
"drupal/core": "^10",
"drupal/core": "^10.3",
"ext-zip": "*",
"ext-json": "*",
"drupal/views_bulk_operations": "^4.2",
Expand All @@ -24,7 +24,9 @@
"maennchen/zipstream-php": "^1.2 || ^2.1",
"drupal/google_api_client": "^3.2 || ^4.3",
"ramsey/uuid": "^4",
"monolog/monolog": "^2.9"
"monolog/monolog": "^2.9",
"ext-simplexml": "*",
"ext-libxml": "*"
},
"minimum-stability": "dev",
"prefer-stable": true,
Expand Down
Loading