Skip to content

Updated scripts and requirements for python3.12 upgrade #59

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 38 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
abfd5b1
updated FirstTimeSetup.py for python3.12 version, updated reqs file t…
gsun1729 Oct 2, 2024
a58c494
rm excess commenting on reqs.txt file
gsun1729 Oct 2, 2024
dd89c7c
mv recipes folder into PythonEnvForAivia
gsun1729 Oct 2, 2024
04886a8
moved tests into their own directory, and added tests for adjustGamma…
gsun1729 Oct 2, 2024
d17dd10
chagned kwargs from selem to footprint for updated skimage morphology…
gsun1729 Oct 4, 2024
52c6fd4
added testing inputs for labeled mask splitting recipes
gsun1729 Oct 4, 2024
b8520c6
added tests for ProcessImages recipes, along with configs and ground …
gsun1729 Oct 4, 2024
9a52f9a
mv'd tests for ProcessImages into their own ProcessImages dir, along …
gsun1729 Oct 4, 2024
f16f96f
added tests for TransformImages Recipes, recipe StackReg_ImageAlignme…
gsun1729 Oct 4, 2024
46a23db
changed imports for matplotlib colormaps, added logic for handling t …
gsun1729 Oct 4, 2024
0a01347
added logic for skipping calling on aivia.exe configurable via test c…
gsun1729 Oct 4, 2024
712e812
added updated pystackreg library compatible with py3.12
gsun1729 Oct 4, 2024
2aea44a
added tests for staackreg_imagealignment
gsun1729 Oct 4, 2024
102dde1
added handling for RGB images and multichannel images
gsun1729 Oct 7, 2024
46135b8
changed file separator and handling for input and output folders for …
gsun1729 Oct 7, 2024
2ec9fc4
changed output_name to output_path, usr provides complete path for ou…
gsun1729 Oct 7, 2024
80b4d36
added pydicom for dicom recipes
gsun1729 Oct 7, 2024
f7a93e0
added comparison func for json outputs
gsun1729 Oct 7, 2024
26dfabf
added tests w/ intputs and ground truths for convertimagesforaivia re…
gsun1729 Oct 7, 2024
2e2ace1
added tests for collectimagemetrics recipes
gsun1729 Oct 7, 2024
d45127b
updated readme for tests
gsun1729 Oct 10, 2024
6be461c
changed highlighting format for markdown for github in tests readme
gsun1729 Oct 10, 2024
96adea7
updated tests readme with working color
gsun1729 Oct 10, 2024
76aec36
updated color scheme
gsun1729 Oct 10, 2024
20a2d58
reduced dir depth, will later replace screenshots with updated screen…
gsun1729 Oct 10, 2024
ef9d346
updated readme structure with test list
gsun1729 Oct 10, 2024
c233ab9
corrected filepaths
gsun1729 Oct 10, 2024
42074be
updated title links to table of contents
gsun1729 Oct 10, 2024
d71caa1
corrected env name in links
gsun1729 Oct 10, 2024
a5e83a5
corrected indenting on code statement and updated python to 312 from 39
gsun1729 Oct 11, 2024
2365b94
updated packages to latest compatible version
gsun1729 Oct 11, 2024
46a9721
formatting and updating docs
gsun1729 Oct 11, 2024
7a3e2ac
updated environment for py3.12 with updated reqs and indentation on S…
gsun1729 Oct 11, 2024
c3c4769
updated zipped environments for ones compatible with py3.12
gsun1729 Oct 11, 2024
f00ccbf
updated docs with updated images
gsun1729 Oct 11, 2024
1376e66
updated readmes to reflect need to run env setup before running tests
gsun1729 Nov 13, 2024
8bf1125
shortened image filenames so there are no zip errors when extracting …
gsun1729 Dec 18, 2024
18d4e98
Update .gitignore
gsun1729 Dec 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
60 changes: 59 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,60 @@
/Recipes/__pycache__
/Utilities/__pycache__
/Utilities/__pycache__
**/outputs/*.tif
**/outputs/*.tiff
**/outputs/*.aiviaexperiment


# These are some examples of commonly ignored file patterns.
# You should customize this list as applicable to your project.
# Learn more about .gitignore:
# https://www.atlassian.com/git/tutorials/saving-changes/gitignore

# Node artifact files
node_modules/
dist/

# Compiled Java class files
*.class

# Compiled Python bytecode
*.py[cod]

# Log files
*.log

# Package files
*.jar

# Maven
target/
dist/

# JetBrains IDE
.idea/

# Unit test reports
TEST*.xml

# Generated by MacOS
.DS_Store

# Generated by Windows
Thumbs.db

# Applications
*.app
*.exe
*.war

# Large media files
*.mp4
*.avi
*.flv
*.mov
*.wmv
*.csv
*.db

# python files
*.pyc
Binary file removed DRVisionFiles/Snapshots/LogEverything.png
Binary file not shown.
Binary file removed DRVisionFiles/Snapshots/PythonFilenameInAivia.png
Binary file not shown.
8 changes: 5 additions & 3 deletions PythonEnvForAivia/FirstTimeSetup.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ def run(params):
env_dir.mkdir(parents=False, exist_ok=True)
subprocess.check_call([str(Path(sys.executable).parent / 'Scripts/virtualenv.exe'), f'{env_dir}'])

# copy essential python packages(python39.zip) to virtual environment
# copy essential python packages(python312.zip) to virtual environment
# see https://github.com/pypa/virtualenv/issues/1185
if not os.path.exists(env_dir/'Scripts/python39.zip'):
copyfile(Path(sys.executable).parent / 'python39.zip', env_dir/'Scripts/python39.zip')
if not os.path.exists(env_dir/'Scripts/python312.zip'):
copyfile(Path(sys.executable).parent / 'python312.zip', env_dir/'Scripts/python312.zip')

# install requirements
mess = 'Python packages will now be installed. An internet connection is needed.\n\n' \
Expand All @@ -41,6 +41,8 @@ def run(params):

pip_path = env_dir / 'Scripts' / 'pip.exe'
requirement_dir = pathlib.Path(os.path.dirname(os.path.realpath(__file__)))
# subprocess.check_call(
# [str(pip_path), 'install', 'setuptools==70.0.0'])
subprocess.check_call(
[str(pip_path), 'install', '-r', str(requirement_dir/'requirements.txt')])

Expand Down
13 changes: 6 additions & 7 deletions PythonEnvForAivia/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Python virtual Environment For Aivia
# Python Virtual Environment for Aivia

## General Information

The FirstTimeSetup.py script will ensure a virtual environment is created to run the python recipes for Aivia with the appropriate packages.
The [`FirstTimeSetup.py`](/PythonEnvForAivia/FirstTimeSetup.py) script will ensure a virtual environment is created to run the python recipes for Aivia with the appropriate packages.
No further install should be needed. Hence, the script is run once only.

The recipes are then containing some code to:
Expand All @@ -13,8 +13,7 @@ The recipes are then containing some code to:

## Requirements

* Python 3.9 - comes with Aivia

* Python 3.12 - comes with Aivia
* Accept the risks that come from running Python Scripts you download from the internet. These scripts are provided to you to use at your own risk.

## Installation
Expand All @@ -33,16 +32,16 @@ The tree structure of the folder is as follow
└───Recipes
└───[category subfolders]
└───... .py

```

## Execution

1. Load `FirstTimeSetup.py` in Aivia by "File>Open" or drag-and-drop

1. Load [`FirstTimeSetup.py`](/PythonEnvForAivia/FirstTimeSetup.py) in Aivia by "File>Open" or drag-and-drop
2. Load any 2D image in Aivia

3. Click on "Start" button and wait for "Process Completed" message at the bottom of the recipe console
1. Please make sure that you have the Internet connection and wait if bandwidth is slow.
2. At the end of this step, a new folder called `env` should be created in the `PythonEnvForAivia` folder. This will be the folder that contains a virtual environment identical to Aivia's embedded python that all future Recipes will run off of.


## Returns
Expand Down
Original file line number Diff line number Diff line change
@@ -1,98 +1,103 @@
import os.path
import numpy as np
from skimage.io import imread, imsave
from skimage.metrics import mean_squared_error, structural_similarity
from skimage.exposure import match_histograms, rescale_intensity
import ctypes

"""
Calculates SSIM map as a result of the comparison of 2 channels and metrics values (in the log file).

For the output image, it is highly recommended to use LUT color mapping to better see the variations in the SSIM values
All real SSIM values (ranging from 0 to 1) can be retrieved from the map doing the following: divide intensities by 255 if image is 8-bit, or by 65535 if 16-bit.

Side note: MSE and mean SSIM (and NRMSE, PSNR) values are output in the log
To be able to see the printed info in the log file, set:
File > Options > Logging > Verbosity = everything

Sources:
https://scikit-image.org/docs/dev/api/skimage.metrics.html?highlight=structural#skimage.metrics.structural_similarity
https://scikit-image.org/docs/dev/auto_examples/color_exposure/plot_histogram_matching.html#sphx-glr-auto-examples-color-exposure-plot-histogram-matching-py


Requirements
------------
numpy (comes with Aivia installer)
scikit-image (comes with Aivia installer)

Parameters
----------
First input: image to compare (e.g.Deep Learning restored image)
Second input: reference (e.g. Ground Truth image), the one adjusted by histogram matching.
IMPORTANT: Input channels need to have the same bit depth

Returns
-------
First output: calculated SSIM map
Second output: reference image transformed with histogram matching

"""

# [INPUT Name:inputGTImagePath Type:string DisplayName:'Input Ground Truth Image']
# [INPUT Name:inputRTImagePath Type:string DisplayName:'Input Restored Image']
# [OUTPUT Name:resultPathAdj Type:string DisplayName:'GT Hist match image']
# [OUTPUT Name:resultPath Type:string DisplayName:'SSIM image']
def run(params):
RTimageLocation = params['inputRTImagePath']
GTimageLocation = params['inputGTImagePath']
resultLocation = params['resultPath']
resultLocationAdj = params['resultPathAdj']

# Checking existence of temporary files (individual channels)
if not os.path.exists(RTimageLocation):
print(f'Error: {RTimageLocation} does not exist')
return;
if not os.path.exists(GTimageLocation):
print(f'Error: {GTimageLocation} does not exist')
return;

# Loading input images
RTData = imread(RTimageLocation)
GTData = imread(GTimageLocation)
print(f'Dimensions of Restored image: {RTData.shape}')
print(f'Dimensions of GT image: {GTData.shape}')

# Checking dtype is the same for both input channels
if GTData.dtype != RTData.dtype:
error_mes = "The bit depth of your input channels is not the same. Convert one of them and retry."
ctypes.windll.user32.MessageBoxW(0, error_mes, 'Error', 0)
sys.exit(error_mes)

# Histogram matching
matched_GTData = match_histograms(GTData, RTData).astype(RTData.dtype)

# MSE measurement
# valMSE = skimage.measure.compare_mse(RTData, GTData) # deprecated in scikit-image 0.18
valMSE = mean_squared_error(RTData, matched_GTData)
print(f'___ MSE = {valMSE} ___') # Value appears in the log if Verbosity option is set to 'Everything'

# SSIM measurement
outFullSSIM = structural_similarity(RTData, matched_GTData, full=True)

# Extracting mean value (first item)
outMeanSSIM = outFullSSIM[0]
print(f'___ Mean SSIM = {outMeanSSIM} ___')

# Extracting map (second item)
outSSIM = outFullSSIM[1]
print(f'Bit depth of SSIM array: {outSSIM.dtype}')

# Convert output array whose range is [0-1] to adjusted bit range (8- or 16-bit) if necessary
if RTData.dtype != np.dtype('float64') and RTData.dtype != np.dtype('float32'):
outputData = rescale_intensity(outSSIM, in_range=(0, 1), out_range=(0, np.iinfo(RTData.dtype).max))
outputData = outputData.astype(RTData.dtype)
else:
outputData = outSSIM

imsave(resultLocation, outputData)
imsave(resultLocationAdj, matched_GTData)
import os.path
import numpy as np
from skimage.io import imread, imsave
from skimage.metrics import mean_squared_error, structural_similarity
from skimage.exposure import match_histograms, rescale_intensity
import ctypes

"""
Calculates SSIM map as a result of the comparison of 2 channels and metrics values (in the log file).

For the output image, it is highly recommended to use LUT color mapping to better see the variations in the SSIM values
All real SSIM values (ranging from 0 to 1) can be retrieved from the map doing the following: divide intensities by 255 if image is 8-bit, or by 65535 if 16-bit.

Side note: MSE and mean SSIM (and NRMSE, PSNR) values are output in the log
To be able to see the printed info in the log file, set:
File > Options > Logging > Verbosity = everything

Sources:
https://scikit-image.org/docs/dev/api/skimage.metrics.html?highlight=structural#skimage.metrics.structural_similarity
https://scikit-image.org/docs/dev/auto_examples/color_exposure/plot_histogram_matching.html#sphx-glr-auto-examples-color-exposure-plot-histogram-matching-py


Requirements
------------
numpy (comes with Aivia installer)
scikit-image (comes with Aivia installer)

Parameters
----------
First input: image to compare (e.g.Deep Learning restored image)
Second input: reference (e.g. Ground Truth image), the one adjusted by histogram matching.
IMPORTANT: Input channels need to have the same bit depth

Returns
-------
First output: calculated SSIM map
Second output: reference image transformed with histogram matching

"""

# [INPUT Name:inputGTImagePath Type:string DisplayName:'Input Ground Truth Image']
# [INPUT Name:inputRTImagePath Type:string DisplayName:'Input Restored Image']
# [OUTPUT Name:resultPathAdj Type:string DisplayName:'GT Hist match image']
# [OUTPUT Name:resultPath Type:string DisplayName:'SSIM image']
def run(params):
RTimageLocation = params['inputRTImagePath']
GTimageLocation = params['inputGTImagePath']
resultLocation = params['resultPath']
resultLocationAdj = params['resultPathAdj']
channel_axis=params.get('channel_axis')
if channel_axis == "None":
channel_axis = None
else:
channel_axis = int(channel_axis)

# Checking existence of temporary files (individual channels)
if not os.path.exists(RTimageLocation):
print(f'Error: {RTimageLocation} does not exist')
return;
if not os.path.exists(GTimageLocation):
print(f'Error: {GTimageLocation} does not exist')
return;

# Loading input images
RTData = imread(RTimageLocation)
GTData = imread(GTimageLocation)
print(f'Dimensions of Restored image: {RTData.shape}')
print(f'Dimensions of GT image: {GTData.shape}')

# Checking dtype is the same for both input channels
if GTData.dtype != RTData.dtype:
error_mes = "The bit depth of your input channels is not the same. Convert one of them and retry."
ctypes.windll.user32.MessageBoxW(0, error_mes, 'Error', 0)
sys.exit(error_mes)

# Histogram matching
matched_GTData = match_histograms(GTData, RTData).astype(RTData.dtype)

# MSE measurement
# valMSE = skimage.measure.compare_mse(RTData, GTData) # deprecated in scikit-image 0.18
valMSE = mean_squared_error(RTData, matched_GTData)
print(f'___ MSE = {valMSE} ___') # Value appears in the log if Verbosity option is set to 'Everything'

# SSIM measurement
outFullSSIM = structural_similarity(RTData, matched_GTData, full=True, channel_axis=channel_axis)

# Extracting mean value (first item)
outMeanSSIM = outFullSSIM[0]
print(f'___ Mean SSIM = {outMeanSSIM} ___')

# Extracting map (second item)
outSSIM = outFullSSIM[1]
print(f'Bit depth of SSIM array: {outSSIM.dtype}')

# Convert output array whose range is [0-1] to adjusted bit range (8- or 16-bit) if necessary
if RTData.dtype != np.dtype('float64') and RTData.dtype != np.dtype('float32'):
outputData = rescale_intensity(outSSIM, in_range=(0, 1), out_range=(0, np.iinfo(RTData.dtype).max))
outputData = outputData.astype(RTData.dtype)
else:
outputData = outSSIM

imsave(resultLocation, outputData)
imsave(resultLocationAdj, matched_GTData)
Loading