Speedup dag

[ggmarshall]

more dag efficencies

[codecov]

Codecov Report

Attention: Patch coverage is 60.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 42.01%. Comparing base (b1bbe44) to head (29fae09).
Report is 15 commits behind head on main.

Files with missing lines	Patch %	Lines
workflow/src/legenddataflow/pars_loading.py	57.14%	3 Missing ⚠️
workflow/src/legenddataflow/execenv.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #118      +/-   ##
==========================================
- Coverage   42.10%   42.01%   -0.09%     
==========================================
  Files          19       18       -1     
  Lines        1209     1202       -7     
==========================================
- Hits          509      505       -4     
+ Misses        700      697       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[gipert]

> snakemake --workflow-profile workflow/profiles/lngs -n 'all-l200-p13-r003-evt.gen'
Using workflow specific profile workflow/profiles/lngs for setting default command line arguments.
ValueError in file /data1/shared/l200-p13/prodenv/prod-orig/dev-v0.0.2/workflow/Snakefile, line 47:
Can't run Props.read_from on sources-value of type <class 'pathlib.PosixPath'>
  File "/data1/shared/l200-p13/prodenv/prod-orig/dev-v0.0.2/workflow/Snakefile", line 47, in <module>
  File "/data1/shared/l200-p13/prodenv/prod-orig/dev-v0.0.2/workflow/src/legenddataflow/cal_grouping.py", line 21, in __init__
  File "/data1/shared/l200-p13/prodenv/prod-orig/dev-v0.0.2/.venv/lib/python3.12/site-packages/dbetto/catalog.py", line 269, in read_from
  File "/data1/shared/l200-p13/prodenv/prod-orig/dev-v0.0.2/.venv/lib/python3.12/site-packages/dbetto/catalog.py", line 267, in read_impl

[gipert]

can we try to add something to the test suite to catch these problems?

[ggmarshall]

hmm i dont get this error

[ggmarshall]

its a dbetto version issue, we require >= 1.2.3 which is correct

[ggmarshall]

this needs testing that passing the table_maps as json strings works as intended

[gipert]

its a dbetto version issue, we require >= 1.2.3 which is correct

you mean i have an old one?

[ggmarshall]

i believe so

[gipert]

boh

InputFunctionException in rule build_dsp in file /data1/shared/l200-p13/prodenv/prod-orig/dev-v0.0.2/workflow/rules/dsp.smk, line 37:
Error:
  AttributeError: 'Catalog' object has no attribute 'channelmap'
Wildcards:
  datatype=anp
  period=p13
  run=r003
  experiment=l200
  timestamp=20241217T094846Z
Traceback:
  File "/data1/shared/l200-p13/prodenv/prod-orig/dev-v0.0.2/workflow/rules/dsp.smk", line 45, in <lambda>
  File "/data1/shared/l200-p13/prodenv/prod-orig/dev-v0.0.2/workflow/src/legenddataflow/scripts/table_name.py", line 34, in get_table_mapping (rule build_dsp, line 37, /data1/shared/l200-p13/prodenv/prod-orig/dev-v0.0.2/workflow/rules/dsp.smk)

[ggmarshall]

I refer you to my previous statement of this needs testing

[ggmarshall]

ok should be good now

[gipert]

still

InputFunctionException in rule build_evt in file /data1/shared/l200-p13/prodenv/prod-orig/dev-v0.0.2/workflow/rules/evt.smk, line 15:
Error:
  TypeError: 'NoneType' object is not iterable
Wildcards:
  datatype=anp
  period=p13
  run=r003
  experiment=l200
  timestamp=20241217T094846Z
Traceback:
  File "/data1/shared/l200-p13/prodenv/prod-orig/dev-v0.0.2/workflow/rules/evt.smk", line 25, in <lambda>
  File "/data1/shared/l200-p13/prodenv/prod-orig/dev-v0.0.2/workflow/src/legenddataflow/pars_loading.py", line 105, in get_par_file (rule build_evt, line 15, /data1/shared/l200-p13/prodenv/prod-orig/dev-v0.0.2/workflow/rules/evt.smk)

I don't have time now but we should figure out why the ci tests pass...