Skip to content

DRAFT: Adding tutorial files for AWS #139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 16, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/apps/register.md
Original file line number Diff line number Diff line change
@@ -144,7 +144,7 @@ Finally, click `Submit`. Visit the Apps page to make sure everything looks OK.

## README / Description / Topics

Brainlife re-uses information stored in github repo.
Brainlife reuses information stored in github repo.

* App Description / Topics

65 changes: 65 additions & 0 deletions docs/tutorial/aws-brainlife.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Using Brainlife with Amazon Web Services

## Introduction

This tutorial guides you through accessing and using Brainlife datasets with various AWS services. Brainlife datasets are hosted on Amazon S3 and can be seamlessly integrated with SageMaker, Lambda, EC2, and Batch for neuroimaging analysis and processing.

## Accessing Brainlife Datasets

Brainlife datasets are stored in an S3 bucket at `s3://brainlife/`. Each dataset includes a `config.json` file containing essential metadata.

To list available datasets using the AWS CLI:

```
aws s3 ls s3://brainlife/
```

## Using Datasets with AWS Services

### SageMaker

In a SageMaker notebook, you can import Brainlife datasets using:

!!! info
Replace `project_id` and `dataset_id` with Brainlife-specific identifiers

```
!aws s3 cp s3://brainlife/project_id/dataset_id/ . --recursive
```

### Lambda

For serverless applications, reference Brainlife datasets in your Lambda function by specifying the S3 path and using the AWS SDK to access the data.

### EC2 & Batch

For compute-intensive tasks, you can use EC2 instances or AWS Batch with Brainlife datasets. Access the datasets directly from S3 using the AWS SDK or CLI commands in your processing scripts. For neuroimaging workflows, you can either develop custom applications or leverage Brainlife's pre-built pipelines that are compatible with cloud-based execution environments.

## OME-Zarr on S3

OME-Zarr is a format for storing bioimaging data, often used in microscopy workflows. If you’re storing an OME-Zarr dataset in S3 (including Brainlife’s S3 buckets), you’ll find a directory structure containing various `.z*` files that specify metadata, multiscale images, and associated attributes.

Here’s an example of what an OME-Zarr layout might look like:

```
OME-Zarr/
├── .zattrs # Global attributes (JSON)
├── .zgroup # Group metadata (JSON)
├── 0 # Multiscale level 0 (highest resolution)
│ ├── .zarray # Zarr array metadata (JSON)
│ ├── 0 # Chunk file(s) containing raw data
│ ├── 1
│ └── 2
└── 1 # Multiscale level 1
├── .zarray
├── 0
├── 1
└── 2
```

- **`.zattrs`** and **`.zgroup`** store high-level metadata about the dataset.
- The numbered directories (`0`, `1`, etc.) represent different resolutions or scales of the image data.
- Each `.zarray` contains metadata for that specific resolution level.
- The numbered files within each resolution directory (for example, `0`, `1`, `2`) contain the actual chunked pixel data.

When working with OME-Zarr data on S3, you can use tools such as [zarr-python](https://zarr.readthedocs.io/en/stable/) or [ome-zarr-py](https://github.com/ome/ome-zarr-py) to read and write image data directly from the bucket.
2 changes: 1 addition & 1 deletion docs/tutorial/diffusion-tractography.md
Original file line number Diff line number Diff line change
@@ -161,7 +161,7 @@ Once you're happy with the results, you can move onto fitting the CSD, DTI, and
* Hit 'Submit'
1. Once the app is finished running, view the results by clicking the 'eye' icon to the right of the dataset
* Choose 'mrview' as your viewer
* This will overlay the tractogram on the generated fractional anistropy (FA) image
* This will overlay the tractogram on the generated fractional anisotropy (FA) image
1. You can also generate a QA image of the whole-brain tractograms by running the 'Generate figures of whole-brain tractogram (tck)' using the whole-brain tractogram (tck) generated above! Archive the results and save with the tag 'qa whole brain tractogram'.

If you're happy with the results, you're ready to move onto segmenting major white matter tracts!
2 changes: 1 addition & 1 deletion docs/tutorial/networkneuroscience-structural.md
Original file line number Diff line number Diff line change
@@ -175,7 +175,7 @@ Once you're happy with the results, you can move on to fitting the CSD, DTI, and
* Hit 'Submit'
1. Once the app is finished running, view the results by clicking the 'eye' icon to the right of the dataset
* Choose 'mrview' as your viewer
* This will overlay the tractogram on the generated fractional anistropy (FA) image
* This will overlay the tractogram on the generated fractional anisotropy (FA) image
1. You can also generate a QA image of the whole-brain tractograms by running the 'Generate figures of whole-brain tractogram (tck)' using the whole-brain tractogram (tck) generated above! Archive the results and save with the tag 'qa whole brain tractogram'.

If you're happy with the results, you're ready to move on to structural connectivity generation!
2 changes: 1 addition & 1 deletion docs/tutorial/old/networkneuroscience.md
Original file line number Diff line number Diff line change
@@ -239,7 +239,7 @@ Once you're happy with the results, you can move onto fitting the CSD, DTI, and
* Hit 'Submit'
1. Once the app is finished running, view the results by clicking the 'eye' icon to the right of the dataset
* Choose 'mrview' as your viewer
* This will overlay the tractogram on the generated fractional anistropy (FA) image
* This will overlay the tractogram on the generated fractional anisotropy (FA) image
1. You can also generate a QA image of the whole-brain tractograms by running the 'Generate figures of whole-brain tractogram (tck)' using the whole-brain tractogram (tck) generated above! Archive the results and save with the tag 'qa whole brain tractogram'.

If you're happy with the results, you're ready to move on to structural connectivity generation!
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
@@ -38,6 +38,7 @@ nav:
- Introduction: tutorial/introduction-to-brainlife.md
- ezBIDS: tutorial/ezBIDS.md
- Anatomy: tutorial/t1w-preprocessing.md
- AWS: tutorial/aws-brainlife.md
- FMRI Preprocessing: tutorial/fmri-preprocessing-tutorial.md
- PRF Mapping: tutorial/prf-mapping.md
- DWI Preprocessing : tutorial/diffusion-preprocessing.md