Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to V1 #437

Merged
merged 383 commits into from
Feb 5, 2025
Merged
Show file tree
Hide file tree
Changes from 250 commits
Commits
Show all changes
383 commits
Select commit Hold shift + click to select a range
39d83ae
enable energy offset in benchmark
rettigl Jun 22, 2024
ac1aa18
typing fixes
rettigl Jun 23, 2024
ec92432
apply bias offset for energy calibration in energy calibrator, and om…
rettigl Jun 25, 2024
7d60c92
fix test
rettigl Jun 25, 2024
ff134a5
fix spelling in all files using Code Spell Checker
rettigl Jun 25, 2024
57d6cd7
some additional spell fixes
rettigl Jun 26, 2024
06a2715
Merge pull request #449 from OpenCOMPES/spelling_fixes
rettigl Jun 26, 2024
ebd2b32
Merge remote-tracking branch 'origin/v1_feature_branch' into refactor…
rettigl Jun 26, 2024
d131fe4
remove nans from all electron channels
zain-sohail Jun 28, 2024
194c874
use pd import, load h5 file inside df creator
zain-sohail Jun 28, 2024
af33740
update comments to explain the code
zain-sohail Jun 28, 2024
50f7ee1
make review changes
zain-sohail Jun 28, 2024
65d909d
fix tests with review comments
zain-sohail Jun 28, 2024
b7537a8
fix dropna
zain-sohail Jun 28, 2024
b0b090d
fix minor stuff and add test to see if exception handling works in pa…
zain-sohail Jun 30, 2024
f5b9148
Merge pull request #329 from OpenCOMPES/refactor-flashloader
zain-sohail Jun 30, 2024
118084e
fix timestamp issue in flashloader
zain-sohail Jun 30, 2024
2a76734
Update benchmark targets
zain-sohail Jun 30, 2024
2c82686
Merge remote-tracking branch 'origin/v1_feature_branch' into energy_c…
rettigl Jun 30, 2024
b9afad7
Merge pull request #455 from OpenCOMPES/update_benchmark_targets
rettigl Jun 30, 2024
bf9d624
Merge remote-tracking branch 'origin/v1_feature_branch' into energy_c…
rettigl Jun 30, 2024
0fdff42
Merge remote-tracking branch 'origin/main' into v1_feature_branch
rettigl Jul 2, 2024
7f9422c
additional spell fixes
rettigl Jul 2, 2024
b90b7f3
change behavior of tof_binning and adc_binning to reflect actual binn…
rettigl Jun 27, 2024
84421ea
add additional ignore entries
rettigl Jun 27, 2024
0da2561
move num_cores to "core" and use it globally
rettigl Jun 27, 2024
6c642b4
Merge branch 'v1_feature_branch' into energy_calibration_bias_shift
rettigl Jul 2, 2024
232c6cc
fix spelling
rettigl Jul 2, 2024
78b5fbf
bump version to 0.1.10a6
Jul 2, 2024
cc8145e
Update dependencies
rettigl Jul 2, 2024
7687fd1
fix flash tests to also rune fine with defined system config
rettigl Jul 3, 2024
6e04835
add typing to tests, and fix issues
rettigl Jul 3, 2024
6b5ff8a
detect illegal keyword arguments
rettigl Jul 3, 2024
05b9a20
Merge remote-tracking branch 'origin/flash_test_fixes' into catch_ill…
rettigl Jul 4, 2024
77692d8
add tests for illegal keyword errors
rettigl Jul 4, 2024
11d4d74
Merge pull request #465 from OpenCOMPES/flash_test_fixes
zain-sohail Jul 6, 2024
f6c33e6
fix group_name error
zain-sohail Jul 2, 2024
30e7447
bring back old behavior
zain-sohail Jul 2, 2024
49d93e9
minor changes
zain-sohail Jul 2, 2024
d830e8c
use concat for faster join
zain-sohail Jul 3, 2024
c48a6ec
try saving both
zain-sohail Jul 3, 2024
4d11d7a
change fill section
zain-sohail Jul 4, 2024
88f0e9c
parallel processing
zain-sohail Jul 4, 2024
f6a45e5
change paths config to hold raw and processed keys
zain-sohail Jul 4, 2024
a8632ab
change filehandling of buffer files as it's more complicated now, and…
zain-sohail Jul 4, 2024
53ea8fd
add dtype handler, make more changes to filepath handler
zain-sohail Jul 6, 2024
e436397
update methods and tests
zain-sohail Jul 7, 2024
1b2e788
sxp test fixes
zain-sohail Jul 7, 2024
7ed0962
processor test fixes
zain-sohail Jul 7, 2024
7d1262b
sxp test fixes
zain-sohail Jul 7, 2024
0a677df
Merge remote-tracking branch 'origin/main' into v1_feature_branch
rettigl Jul 8, 2024
989f438
Merge branch 'v1_feature_branch' into energy_calibration_bias_shift
rettigl Jul 8, 2024
8774b8f
Merge branch 'v1_feature_branch' into tof_binning-as-actual-binning-v…
rettigl Jul 8, 2024
245762a
Merge branch 'energy_calibration_bias_shift' into catch_illegal_kwds
rettigl Jul 8, 2024
638ed68
add further tests
rettigl Jul 8, 2024
84f99ef
fix typing
rettigl Jul 8, 2024
5b6f940
remove 0 vals from all pulse channels
zain-sohail Jul 8, 2024
ecd517d
Merge pull request #451 from OpenCOMPES/tof_binning-as-actual-binning…
rettigl Jul 8, 2024
748cb11
Merge branch 'v1_feature_branch' into energy_calibration_bias_shift
rettigl Jul 8, 2024
1ef47b1
update energy calibration description
rettigl Jul 8, 2024
a017fe4
fix Energy(TOF) view for binding energy scale
rettigl Jul 8, 2024
c0d61ea
add some more tests for mpes loader
rettigl Jul 8, 2024
4b682a5
Merge branch 'v1_feature_branch' into fix-459
rettigl Jul 9, 2024
b325ba6
update paths
rettigl Jul 9, 2024
852ce8e
revert the benchmarking
zain-sohail Jul 10, 2024
2de5243
update aux channel handling
zain-sohail Jul 15, 2024
ae4a2c0
Merge remote-tracking branch 'origin/main' into v1_feature_branch
rettigl Jul 16, 2024
3eb5dea
Merge remote-tracking branch 'origin/main' into v1_feature_branch
rettigl Jul 16, 2024
c717114
Merge branch 'v1_feature_branch' into energy_calibration_bias_shift
rettigl Jul 16, 2024
cb7c97a
Merge pull request #411 from OpenCOMPES/energy_calibration_bias_shift
rettigl Jul 16, 2024
1e2ea1b
Merge pull request #466 from OpenCOMPES/catch_illegal_kwds
rettigl Jul 16, 2024
cce0ba7
limit matplotlib to fix typing issue
rettigl Jul 16, 2024
770af0d
index sorting
zain-sohail Jul 17, 2024
7944043
roll back the buffer_handler with small changes
zain-sohail Jul 17, 2024
adfd335
fix some test issues
zain-sohail Jul 17, 2024
fbd5cbf
fix energy calibration and delay range
rettigl Jul 18, 2024
dc74566
Merge remote-tracking branch 'origin/v1_feature_branch' into fix-459
rettigl Jul 18, 2024
4d35e54
some metadata changes
zain-sohail Jul 18, 2024
b9bef4f
Merge branch 'fix-459' into flash-minor-changes
zain-sohail Jul 18, 2024
0aab5c4
use correct lock
zain-sohail Jul 18, 2024
1966ac4
roll back to iterations
zain-sohail Jul 18, 2024
d09e715
add aux alias and subchannels argument
zain-sohail Jul 23, 2024
b0471f2
change name and return of the run method
zain-sohail Jul 23, 2024
85c1315
Merge pull request #479 from OpenCOMPES/flash-minor-changes
zain-sohail Jul 23, 2024
89ba09e
Merge pull request #469 from OpenCOMPES/fix-459
zain-sohail Jul 23, 2024
ddfb16c
Update benchmark targets
zain-sohail Jul 23, 2024
019bf40
remove invalid channels
zain-sohail Jul 23, 2024
80da01d
create exception and use it to find invalid files
zain-sohail Jul 25, 2024
e8b8a4d
Merge pull request #483 from OpenCOMPES/update_benchmark_targets
rettigl Jul 25, 2024
337e6f8
Merge remote-tracking branch 'origin/main' into v1_feature_branch
rettigl Jul 25, 2024
4d1ce1e
add exception when no valid files are available and fix testing
zain-sohail Jul 26, 2024
f8aab53
implement suggested changes
zain-sohail Jul 31, 2024
d305f19
let code run even if no train channels exist
zain-sohail Aug 11, 2024
69521c9
Merge remote-tracking branch 'origin/main' into v1_feature_branch
rettigl Aug 14, 2024
779b594
limit pynxtools
rettigl Aug 14, 2024
36acf44
Merge branch 'v1_feature_branch' into remove-invalid-channels
rettigl Aug 16, 2024
2b6f96e
filter out all negative pulse values as they are invalid
zain-sohail Aug 21, 2024
a890d5d
filter out in final df
zain-sohail Aug 21, 2024
60e43e7
Merge pull request #484 from OpenCOMPES/remove-invalid-channels
zain-sohail Aug 21, 2024
83fe946
Merge remote-tracking branch 'origin/main' into v1_feature_branch
rettigl Sep 19, 2024
bda9e10
replace prints by logging
rettigl Jul 18, 2024
f824bad
fixes for new matplotlib
rettigl Jul 18, 2024
fbb0af2
use base sed logger
rettigl Aug 11, 2024
469236d
add logging to mpes loader
rettigl Aug 11, 2024
37feca1
add verbosity to loader interface
rettigl Aug 12, 2024
2fbae5d
add logging to delay calibrator
rettigl Aug 12, 2024
3292658
use logging in momentum calibrator
rettigl Aug 13, 2024
e3af246
use logging in energy calibrator, and further fixes
rettigl Aug 13, 2024
2a69836
some further fixes
rettigl Aug 13, 2024
79a181d
add function to set verbosity, and use in each class
rettigl Sep 19, 2024
8716dc0
Make verbose a private property, and add getter and setters for it, t…
rettigl Sep 19, 2024
c90e839
Merge pull request #490 from OpenCOMPES/logging
rettigl Sep 19, 2024
64848b9
adding this flag due to https://github.com/astral-sh/ruff/issues/5434
zain-sohail Jul 31, 2024
eb5ee3f
first pydantic model for config
zain-sohail Jul 31, 2024
600ef55
without typeddict
zain-sohail Sep 14, 2024
24998df
remove defaults
zain-sohail Sep 14, 2024
bea6079
update lock file with pydantic
zain-sohail Sep 14, 2024
c7aac14
nest the models
zain-sohail Sep 15, 2024
c179d9a
add copytool in configmodel and other attrs
zain-sohail Sep 15, 2024
d7cab5f
update config files to conform to model
zain-sohail Sep 15, 2024
75f1200
use configmodel in processor class
zain-sohail Sep 16, 2024
8ef5807
update modules to new config
zain-sohail Sep 16, 2024
02aea0a
fix some config problems
zain-sohail Sep 16, 2024
d84f6e4
update lockfile
rettigl Oct 7, 2024
715c42a
update poetry, limit numpy
rettigl Jun 24, 2024
a70529a
update lockfile
rettigl Sep 11, 2024
daf5c2d
limit dask version to <2024.8 due to bug where computation takes forw…
rettigl Oct 11, 2024
1aa4a83
disable dask query-planning as workaround for bugs in dask-expr
rettigl Oct 11, 2024
41f05ef
remove limits on python versions for tests
rettigl Oct 11, 2024
d56d6d9
Merge pull request #448 from OpenCOMPES/update_dask_version
rettigl Oct 12, 2024
5c0f75e
Merge remote-tracking branch 'origin/v1_feature_branch' into pydantic…
rettigl Oct 12, 2024
1146b4f
fix tests for calibrators
rettigl Oct 12, 2024
a1a9b27
make model fail on extra parameters
rettigl Oct 12, 2024
e3577bb
fix flash loader
rettigl Oct 12, 2024
8054bfa
fix calibrator tests again
rettigl Oct 12, 2024
35dcd11
fix processor tests
rettigl Oct 12, 2024
738cd85
fix sxp loader
rettigl Oct 12, 2024
bc6f457
update notebooks
rettigl Oct 12, 2024
bc40fea
fix remaining tests
rettigl Oct 12, 2024
b6db85f
add config model for copy tool
rettigl Oct 13, 2024
bdc5bac
Add further type refinements to config model
rettigl Oct 13, 2024
022dc69
add tests for config model
zain-sohail Oct 14, 2024
52a11dc
fix remaining tutorials
rettigl Oct 14, 2024
9edbea2
fix config model tests
rettigl Oct 15, 2024
ad8705d
add review suggestions
rettigl Oct 15, 2024
7aa7231
fix reporting of energy/delay offsets
rettigl Oct 15, 2024
ec20e03
fix handling of nexus input files and tests
rettigl Oct 15, 2024
4e535cf
fix error reporting
rettigl Oct 15, 2024
7a7441c
fix sxp notebook
rettigl Oct 17, 2024
fb04ce6
changes from review
rettigl Oct 21, 2024
fea015a
Move static (#511)
zain-sohail Oct 22, 2024
68b2eaf
Merge pull request #487 from OpenCOMPES/pydantic-model
rettigl Oct 22, 2024
6d5c55c
fix benchmarks
rettigl Oct 22, 2024
1de860a
Merge remote-tracking branch 'origin/main' into v1_feature_branch
rettigl Nov 12, 2024
47458dd
update merged changes
rettigl Nov 12, 2024
d26654b
update build scripts to new config model
rettigl Nov 12, 2024
d18934c
move code to src folder
rettigl Oct 14, 2024
4375692
fix tests
rettigl Oct 14, 2024
f2931ef
update workflows
rettigl Oct 14, 2024
35e4966
update tutorials
rettigl Oct 14, 2024
07cbc89
add config to build specs
rettigl Oct 14, 2024
757e19e
fix flash build script again
rettigl Oct 15, 2024
1499550
update path to Nexus config file
rettigl Oct 15, 2024
a101031
fix tests
rettigl Oct 15, 2024
0d1b805
fix benchmarks
rettigl Oct 22, 2024
1b7b926
Merge pull request #508 from OpenCOMPES/src_layout
rettigl Nov 12, 2024
df8d89b
move Flash Nxmpes config
rettigl Nov 12, 2024
49b4b02
update build script
rettigl Nov 12, 2024
f3da086
update jupyter to >v4
rettigl Nov 12, 2024
1cbfd36
adopt figure sizes
rettigl Oct 15, 2024
dd91313
change color map to default plt cycle
rettigl Oct 15, 2024
6f5f033
use matplotlib also for momentum correction
rettigl Oct 15, 2024
366de39
fix type error
rettigl Oct 15, 2024
6365b93
fix figure sizes and font sizes
rettigl Oct 16, 2024
fb0b025
remove redundant toctree entry
rettigl Nov 13, 2024
7ff8fc2
fix config docs
rettigl Nov 13, 2024
8ecde26
Merge pull request #509 from OpenCOMPES/style_fixes
rettigl Nov 13, 2024
8f4b6bf
Merge pull request #516 from OpenCOMPES/update_jupyter
rettigl Nov 21, 2024
1a28294
Merge remote-tracking branch 'origin/main' into v1_feature_branch
rettigl Nov 30, 2024
f4350d8
fix config paths
rettigl Nov 30, 2024
02d8a71
fix new notebooks and merging bugs
rettigl Nov 30, 2024
787447d
fix changed variable name
rettigl Nov 30, 2024
9be909b
Merge remote-tracking branch 'origin/main' into v1_feature_branch
rettigl Dec 3, 2024
2d0b2fd
Merge remote-tracking branch 'origin/main' into v1_feature_branch
rettigl Dec 19, 2024
637504d
fix merge errors
rettigl Dec 19, 2024
0f39523
move config into sed
rettigl Dec 21, 2024
d76deab
fix config references
rettigl Dec 21, 2024
5f2f179
fix build scripts
rettigl Dec 21, 2024
a21a4cb
Merge remote-tracking branch 'origin/main' into fix_src_layout
rettigl Dec 30, 2024
2b162ad
Merge pull request #537 from OpenCOMPES/fix_src_layout
rettigl Dec 30, 2024
afd4984
update pyproject and workflows
rettigl Jan 1, 2025
1118d54
use dynamic versioning
rettigl Jan 1, 2025
e07c99e
fix spelling
rettigl Jan 1, 2025
a50e9b8
add v1 branch
rettigl Jan 1, 2025
4049c1e
update tests
rettigl Jan 2, 2025
208beee
limit photutils
rettigl Jan 2, 2025
422a7f3
remove 3.8 again
rettigl Jan 2, 2025
fd63650
fix coverage report
rettigl Jan 2, 2025
d55e176
add build step
rettigl Jan 2, 2025
3c241c8
try fix release
rettigl Jan 2, 2025
dee2501
fix benchmark
rettigl Jan 2, 2025
bfca5bd
fix url
rettigl Jan 2, 2025
d868238
add pypi-test url and credentials
rettigl Jan 2, 2025
469b7b7
fix scripts
rettigl Jan 2, 2025
1ca5acf
remove password
rettigl Jan 2, 2025
ead1256
add verbose output
rettigl Jan 2, 2025
6452e2b
fix docs config
rettigl Jan 2, 2025
e30c65c
publish to main pypi, and remove trigger
rettigl Jan 3, 2025
f831898
update docs
rettigl Jan 3, 2025
64b07c1
further docs fixes
rettigl Jan 4, 2025
178c6ea
Merge pull request #540 from OpenCOMPES/move_to_uv
rettigl Jan 5, 2025
8d9bcd5
remove empty pulses from timed dataframe, and bring back old behavior
rettigl Jan 6, 2025
39c2e52
add further exceptions for completely empty files, and exceptions
Jan 7, 2025
b4ddac4
allow both timed dataframe formats
zain-sohail Jan 10, 2025
73269e5
add back docstring
zain-sohail Jan 10, 2025
87fa1fa
reformat
zain-sohail Jan 10, 2025
b3e79c3
Merge pull request #546 from OpenCOMPES/flash_normalization_fixes-zain
zain-sohail Jan 10, 2025
299d04b
fix metadata issues, and add more debug logs
zain-sohail Jan 10, 2025
e2dfcc5
fix lint errors
zain-sohail Jan 10, 2025
24df1f0
remove repitition
zain-sohail Jan 10, 2025
5eba2e3
remove warnings
zain-sohail Jan 10, 2025
ccd5a8c
add documentation about feature
zain-sohail Jan 12, 2025
2dca036
Merge pull request #541 from OpenCOMPES/flash_normalization_fixes
zain-sohail Jan 12, 2025
620f4c4
Merge branch 'v1_feature_branch' into flash-metadata-fixes
zain-sohail Jan 12, 2025
d66d34d
update token handling with env variables
zain-sohail Jan 12, 2025
9a7a8b0
update test
zain-sohail Jan 12, 2025
9d22be7
add the dotenv package
zain-sohail Jan 12, 2025
a99b945
Merge pull request #542 from OpenCOMPES/more-broken-file-fixes
rettigl Jan 12, 2025
80ef9c1
read write env variables without extra package, tests added
zain-sohail Jan 12, 2025
a251f61
Merge branch 'v1_feature_branch' into flash-metadata-fixes
zain-sohail Jan 12, 2025
0adb961
config error
zain-sohail Jan 13, 2025
b60a891
Merge remote-tracking branch 'origin/main' into v1_feature_branch
rettigl Jan 13, 2025
5084bb7
use value error
zain-sohail Jan 14, 2025
9b13c99
search for .env in cwd and os env variables
zain-sohail Jan 14, 2025
977223d
Merge branch 'flash-metadata-fixes' of github.com:OpenCOMPES/sed into…
zain-sohail Jan 14, 2025
25a935b
bring back comments
zain-sohail Jan 14, 2025
82fc11f
Merge pull request #547 from OpenCOMPES/flash-metadata-fixes
zain-sohail Jan 15, 2025
2c2a9ae
Merge branch 'v1_feature_branch' into pydantic-error-handling
zain-sohail Jan 15, 2025
cd27f4a
Merge branch 'v1_feature_branch' of github.com:OpenCOMPES/sed into v1…
rettigl Jan 16, 2025
c333747
Merge remote-tracking branch 'origin/main' into v1_feature_branch
rettigl Jan 16, 2025
1752d34
Merge pull request #549 from OpenCOMPES/pydantic-error-handling
rettigl Jan 16, 2025
b3ad646
update tests
zain-sohail Jan 16, 2025
27234e0
Fix release and update documentation
rettigl Jan 19, 2025
321f2fe
use user platformdir also for user config
rettigl Jan 20, 2025
ab5adac
Merge pull request #553 from OpenCOMPES/fix-552
zain-sohail Jan 20, 2025
fb71ec3
Merge remote-tracking branch 'origin/v1_feature_branch' into config_r…
rettigl Jan 20, 2025
d283e93
add tests for system config dir
rettigl Jan 20, 2025
975a506
Merge pull request #554 from OpenCOMPES/fix-release-and-docs
rettigl Jan 20, 2025
8ff0040
Merge pull request #555 from OpenCOMPES/config_renaming
rettigl Jan 20, 2025
cf9f594
update workflows
rettigl Feb 5, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .cspell/custom-dictionary.txt
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ caldir
calib
calibdict
caplog
capsys
cdeform
cdeformfield
cdisp
Expand Down Expand Up @@ -77,6 +78,8 @@ datastreams
datestring
ddir
delaxes
delayeds
delenv
Desy
Deutsches
dfield
Expand All @@ -85,12 +88,14 @@ dfpart
dfpid
dictionarized
dictmerge
DLDAUX
DOOCS
dpkg
dropna
dset
dsets
dtype
dtypes
easimon
ecalibdict
electronanalyser
Expand Down Expand Up @@ -125,6 +130,7 @@ ftype
fwhm
genindex
getgid
getgrgid
getmtime
gpfs
griddata
Expand Down Expand Up @@ -290,6 +296,7 @@ ptargs
pullrequest
pval
pyarrow
pydantic
pydata
pyenv
pygments
Expand Down Expand Up @@ -330,6 +337,7 @@ scipy
SDIAG
sdir
segs
setp
sfile
shutil
Sixten
Expand All @@ -340,6 +348,8 @@ splinewarp
stackax
stackaxis
stepsize
subchannel
subchannels
subdir
subdirs
subfolders
Expand Down Expand Up @@ -401,6 +411,8 @@ xpos
xratio
xrng
xscale
xticklabels
xticks
xtrans
Xuser
xval
Expand All @@ -411,6 +423,8 @@ ylabel
ypos
yratio
yscale
yticklabels
yticks
ytrans
zain
Zenodo
Expand Down
29 changes: 18 additions & 11 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,13 @@ name: benchmark
on:
workflow_dispatch:
push:
branches: [ main, create-pull-request/patch ]
branches: [ main, v1_feature_branch, create-pull-request/patch ]
paths-ignore:
pyproject.toml

env:
UV_SYSTEM_PYTHON: true

jobs:
benchmark:
runs-on: ubuntu-latest
Expand All @@ -16,28 +19,32 @@ jobs:
- name: Check out the repository
uses: actions/checkout@v4
with:
lfs: true
fetch-depth: 0

- uses: tibdex/github-app-token@v1
id: generate-token
with:
app_id: ${{ secrets.APP_ID }}
private_key: ${{ secrets.APP_PRIVATE_KEY }}

# Use cached python and dependencies, install poetry
- name: "Setup Python, Poetry and Dependencies"
uses: packetcoders/action-setup-cache-python-poetry@main
# Setup python
- name: Set up Python 3.10
uses: actions/setup-python@v5
with:
python-version: 3.8
poetry-version: 1.2.2
python-version: "3.10"

- name: Install dependencies
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh

- name: Install project dependencies
run: poetry install
- name: Install package
run: |
uv pip install ".[dev]"

# Run benchmarks
- name: Run benchmarks on python 3.8
- name: Run benchmarks on python 3.10
run: |
poetry run pytest --full-trace --show-capture=no -sv benchmarks/benchmark_*.py
pytest --full-trace --show-capture=no -sv benchmarks/benchmark_*.py

- name: Obtain git status
id: status
Expand Down
85 changes: 34 additions & 51 deletions .github/workflows/documentation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,14 @@ on:
branches: [ main ]
tags: [ v* ]
paths:
- sed/**/*
- src/sed/**/*
- tutorial/**
- .github/workflows/documentation.yml
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

env:
UV_SYSTEM_PYTHON: true

jobs:
build:
Expand All @@ -26,69 +28,50 @@ jobs:
remove-android: 'true'
remove-docker-images: 'true'

# Check out repo and set up Python
- name: Check out the repository
uses: actions/checkout@v4
# Check out repo and set up Python
- uses: actions/checkout@v4
with:
lfs: true
fetch-depth: 0

# Use cached python and dependencies, install poetry
- name: "Setup Python, Poetry and Dependencies"
uses: packetcoders/action-setup-cache-python-poetry@main
# Setup python
- name: Set up Python 3.10
uses: actions/setup-python@v5
with:
python-version: 3.9
poetry-version: 1.8.3
python-version: "3.10"

- name: Install dependencies
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh

- name: Install notebook dependencies
run: poetry install -E notebook --with docs
- name: Install package
run: |
uv pip install ".[docs,notebook]"

- name: Install pandoc
run: |
sudo wget https://github.com/jgm/pandoc/releases/download/3.1.8/pandoc-3.1.8-1-amd64.deb
sudo dpkg -i pandoc-3.1.8-1-amd64.deb

# rm because hextof_workflow notebook can not run outside maxwell
- name: copy tutorial files to docs
run: |
cp -r $GITHUB_WORKSPACE/tutorial $GITHUB_WORKSPACE/docs/
cp -r $GITHUB_WORKSPACE/sed/config $GITHUB_WORKSPACE/docs/sed

mkdir -p $GITHUB_WORKSPACE/docs/src/sed
cp -r $GITHUB_WORKSPACE/src/sed/config $GITHUB_WORKSPACE/docs/src/sed/

- name: download RAW data
# if: steps.cache-primes.outputs.cache-hit != 'true'
run: |
cd $GITHUB_WORKSPACE/docs
poetry run python scripts/download_data.py
python scripts/download_data.py

- name: build parquet files
run: |
cd $GITHUB_WORKSPACE/docs
poetry run python scripts/build_flash_parquets.py
poetry run python scripts/build_sxp_parquets.py

# to be removed later. This theme doesn't support <3.9 python and our lock file contains 3.8
- name: install pydata-sphinx-theme
run: |
poetry run pip install pydata-sphinx-theme

- name: Change version for develop build
if: startsWith(github.ref, 'refs/heads/') && github.ref != 'refs/heads/main'
run: |
VERSION=`sed -n 's/^version = "\(.*\)".*/\1/p' $GITHUB_WORKSPACE/pyproject.toml`
MOD_VERSION=$VERSION".dev0"
echo $MOD_VERSION
sed -i "s/^version = \"$VERSION\"/version = \"$MOD_VERSION\"/" $GITHUB_WORKSPACE/pyproject.toml

- name: Change version for release build
if: startsWith(github.ref, 'refs/tags/')
run: |
OLD_VERSION=`sed -n 's/^version = "\(.*\)".*/\1/p' $GITHUB_WORKSPACE/pyproject.toml`
NEW_VERSION=`echo ${GITHUB_REF#refs/tags/} | sed -n 's/^v\(.*\)/\1/p'`
echo $NEW_VERSION
sed -i "s/^version = \"$OLD_VERSION\"/version = \"$NEW_VERSION\"/" $GITHUB_WORKSPACE/pyproject.toml
python scripts/build_flash_parquets.py
python scripts/build_sxp_parquets.py

- name: build Sphinx docs
run: poetry run sphinx-build -b html $GITHUB_WORKSPACE/docs $GITHUB_WORKSPACE/_build
run: sphinx-build -b html $GITHUB_WORKSPACE/docs $GITHUB_WORKSPACE/_build

- name: Upload artifact
uses: actions/upload-artifact@v4
Expand All @@ -102,17 +85,12 @@ jobs:
needs: build
steps:
- name: Checkout docs repo
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
repository: ${{ github.repository_owner }}/docs
token: ${{ secrets.GITHUB_TOKEN }}
path: 'docs-repo'

- name: Set up Python 3.9
uses: actions/setup-python@v4
with:
python-version: 3.9

- name: Setup SSH
uses: webfactory/[email protected]
with:
Expand All @@ -129,11 +107,16 @@ jobs:
run: |
if [[ $GITHUB_REF == refs/tags/* ]]; then
VERSION=${GITHUB_REF#refs/tags/}
echo "folder=sed/$VERSION" >> $GITHUB_OUTPUT
rm docs-repo/sed/stable
rm -rf docs-repo/sed/latest
ln -s -r docs-repo/sed/$VERSION docs-repo/sed/stable
ln -s -r docs-repo/sed/$VERSION docs-repo/sed/latest
echo "folder=sed/$VERSION" >> $GITHUB_OUTPUT
if [[ $VERSION == *a* ]]; then
rm -rf docs-repo/sed/latest
ln -s -r docs-repo/sed/$VERSION docs-repo/sed/latest
else
rm -rf docs-repo/sed/stable
rm -rf docs-repo/sed/latest
ln -s -r docs-repo/sed/$VERSION docs-repo/sed/stable
ln -s -r docs-repo/sed/$VERSION docs-repo/sed/latest
fi
elif [[ $GITHUB_REF == refs/heads/main ]]; then
rm -rf docs-repo/sed/latest
echo "folder=sed/latest" >> $GITHUB_OUTPUT
Expand Down
28 changes: 19 additions & 9 deletions .github/workflows/linting.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,34 +6,44 @@ on:
paths-ignore:
pyproject.toml

env:
UV_SYSTEM_PYTHON: true

jobs:
lint:
runs-on: ubuntu-latest
steps:
# Check out repo and set up Python
- uses: actions/checkout@v4
with:
lfs: true
fetch-depth: 0

# Use cached python and dependencies, install poetry
- name: "Setup Python, Poetry and Dependencies"
uses: packetcoders/action-setup-cache-python-poetry@main
# Setup python
- name: Set up Python 3.10
uses: actions/setup-python@v5
with:
python-version: 3.8
poetry-version: 1.2.2
python-version: "3.10"

- name: Install dependencies
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh

- name: Install package
run: |
uv pip install ".[dev]"

# Linting steps, execute all linters even if one fails
- name: ruff
run:
poetry run ruff sed tests
ruff src/sed tests
- name: ruff formatting
if: ${{ always() }}
run:
poetry run ruff format --check sed tests
ruff format --check src/sed tests
- name: mypy
if: ${{ always() }}
run:
poetry run mypy sed tests
mypy src/sed tests
- name: spellcheck
if: ${{ always() }}
uses: streetsidesoftware/cspell-action@v6
Expand Down
Loading
Loading