Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QC-1075] Allow for multiple data sources in QC tasks #2098

Merged
merged 3 commits into from
Feb 7, 2024

Conversation

knopers8
Copy link
Collaborator

Which, effectively, allows to declary CCDB inputs as direct alongside of a data sampling policy.

To my impression, it seems to work with the example provided by Markus.

To be decided if allowing for multiple data sources is the best way forward. The users might try to combine direct data sources with sampled sources, which will normally leave plenty of partially filled timeslices which are never executed. However, this generalization is in line with multiple data sources for Checks and Aggregators.
A possible approach would be to declare a ccdb data source, which is interally treated as direct, but can be explicitely asserted that it is used in combination with a data sampling policy.

Which, effectively, allows to declary CCDB inputs as direct alongside of a data sampling policy
@knopers8 knopers8 requested a review from Barthelemy as a code owner January 19, 2024 16:45
@knopers8
Copy link
Collaborator Author

Hi @mfasDa , this PR, in combination with the attached config file, allows to put CCDB inputs as direct in combination with a sampling policy. On my setup, I was getting valid results, but perhaps I am missing something. Could you please give it a go as well?

The way to configure might change a bit, but the underlying behaviour should stay the same... that is assuming that you confirm this works as expected.

cells2.json

@knopers8 knopers8 changed the title [WIP][QC-1075] Allow for multiple data sources in QC tasks [QC-1075] Allow for multiple data sources in QC tasks Jan 31, 2024
@knopers8
Copy link
Collaborator Author

@mfasDa Since you might be busy with something else, I will propose to move on with this PR, then later we can address any issues you might encounter while trying it.

@Barthelemy Ready for review

@Barthelemy
Copy link
Collaborator

@knopers8 do I understand correctly that all the config files will have to be adapted ?

@knopers8
Copy link
Collaborator Author

No, it is backwards-compatible.

@Barthelemy
Copy link
Collaborator

indeed, sorry, I missed one line

@Barthelemy
Copy link
Collaborator

waiting for o2 to finish. I don't think that the failure of O2fst/o2 is related

@Barthelemy Barthelemy merged commit 7206976 into AliceO2Group:master Feb 7, 2024
7 checks passed
@knopers8 knopers8 deleted the task-datasources branch February 12, 2024 08:47
@mfasDa
Copy link
Collaborator

mfasDa commented Jun 12, 2024

Hi @knopers8 , Hi @Barthelemy ,

Sorry for the late reply! The solution works for us. I get both the CCDB objects and the downsampled data. Our PR can now also be merged.

Thanks a lot for your help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants