BioImage.IO Meeting Minutes

[oeway]

Hi,

The BioImage.IO team will meet every Wednesday 4PM CET, check the next meeting in your time zone.

Meeting Format

As discussed during in the AI4Life general assembly meeting (Oct. 10th, 2023), we will make the meeting more structured. For the first 15 minutes, we will take turns and having one presentation every week from one of the AI4Life partners, or guest speakers from community partners, developer etc.

The aim of the first 15-minute presentation is to provide community-friendly (especial for users and developers outside the AI4Life consortium) updates on new features, changes, announcements etc.

And the later part, we will have regular internal discussions or other relevant topics. Everyone is welcome to join the meeting via zoom: https://kth-se.zoom.us/j/65777152331, feel free to add your name and topic to the meeting agenda.

[oeway]

June 26th, 2020

Attendees

Name	affiliation	username
Wei	KTH	@oeway
Esti	UC3M	@esgomezm
Hao	KTH	@haoxusci
Arrate	UC3M	@ArrateMunoz
Anna	EMBL	@akreshuk
Fynn	EMBL	@FynnBe
Maxim	EMBL	@m-novikov

Agenda + Minutes

• All: general intro and your plans for the zoo in the near future

• [name=Wei]:

  • updates on the new bioimage.io website
    • rebuild the website with a build system and CI
    • New UI design powered by Bulma
    • Support models, applications, notebooks, datasets
    • Support badges
    • Share the site template with other projects: covid19ai.io, shareloc.xyz etc.
    • Building backend (imjoy-ai-engine) for supporting uploading datasets and online inference service

• [name=Anna] Technical question: single model with multiple weights, trained in different ways

• [name=Anna] Other potential collaborators: ZeroCostDL4Mic

• Discussion

  • How to support linking datasets with model weights and describe the training
  • Extend the model description spec, also for describing datasets, notebooks, applications

Note

• How to group different cards
  • if only weights changed == same card
  • If postprocessing changed == different card

[oeway]

July 10th, 2020

Attendees

Name	affiliation	username
Wei	KTH	@oeway
Esti	UC3M	@esgomezm
Daniel	EPFL	@dasv74
Anna	EMBL	@akreshuk
Carlos	UC3M	@carlosuc3m
Maksim Novikov	EMBL	@m-novikov
Ricardo Henriques	IGC/UCL	@paxcalpt

Agenda + Minutes

• [name=Wei]

  • Updates for the website
    • resource description file
    • support links between items (apps to notebooks, dataset etc.)
    • support pagination
    • add ZeroCostDL4Mic and HPA as a community partner
    • comment feature powered by github
    • community partner
      • what's the criterion and procedure to join as a community partner?
        • Should we allow their own model zoo? (Think image.sc)
        • Open source? comercial partners?

          • Anna: accessible dataset, source of the model

          • Esti: published model without close dataset

          • Wei: sensitive medical data

          • Ricardo & Wei: Use badges to indicate and encourage

          • conclusion: give recommendation and show badges

        • Allow at least one admin of bioimage.io to access the repo?
          • conclusion: agreed
      • what kind of role and responsibility does the community partner have?
        • Contribute and maintain models/applications/notebooks
        • Accept third-party contribution if relevant? Say ilastik will in charge of all PRs for the ilastik models?
          • Anna: for long term, we should see more interoperatable models and that should go to BioImage.IO
          • for dedicated models that works for only one community partner, contribute to the partner instead of BioImage.IO

• [name=Anna]

  • What examples do we need to add to make it easy for people to understand how to run models?

  • We should add a general "About" page describing what and why

    • Wei: Will do, this already been added for the community partners

• [name=Daniel]

  • Demo of Model manager in DeepImageJ
  • Suggestions by Wei:
    • Add bug report button to link the same github issue as on BioImage.IO
    • Show linked items

• [name=Esti]

  • Good practices uploading models; i.e. authorship & responsabilities
    • Anna & Ricardo: cite and acknowledge properly, typically it's fine to upload models that are derived from others' work
    • Wei & Ricardo: follow the license, most license will include terms about attribution

• Discussion

  • What kind of users we are targeting with BioImage.IO

  • What's scope and focus of the bioimage.io platform? Should we only provide models, or including also the downstream/upstream utility tools for building workflow. The layout of the website.

  • How to upload a model compatible with different software? Same card / id besides proper reference?

    • Wei: use CI to automatically test the software compatibility, using BioEngine api
    • Anna: These tests should be done automatically

• Suggestions

  • Daniel: Allow users to post images to the github issue for each model
  • Esti: support linking to image.sc
  • Ricardo: Network view for navigation

[esgomezm]

July 24th, 2020

Attendees

Name	affiliation	username
Esti	UC3M	@esgomezm
Maksim Novikov	EMBL	@m-novikov
Arrate	UC3M	@amunoz
Constantin	EMBL	@constantinpape
Anna	EMBL	@akreshuk

Minutes

Discussion

• What’s the distinction between models and notebooks
  -- DeepImageJ: Notebooks and Models are actually separate
  -- Notebook is used for training the model
  -- models are then only the train
  -- more general, how do we link models, notebooks (e.g. used to train the model) and datasets (e.g. where model was trained on)

• datasets:
  -- how do we reflect “small” changes to the data-sets, e.g. fixing issues in the ground-truth
  -- public datasets can be “easily” added
  -- what about the private datasets?
  -- change the name? maybe "working data", "training data"

• easier way to explore the models apart from search, ideas:
  -- “structured tag cloud”: selecting multiple tags to find a model that fits the “segmentation”, “electron-microscopy”, “membranes”, …

• Discussion about the CZI Open Software call
  https://chanzuckerberg.com/rfa/essential-open-source-software-for-science/
  DeepImageJ is considering to apply; question is if/how to integrate the BioImage.io in there.
  Including Demo resources.
  Ilastik is also applying for the call, but for a different purpose?
  Are other community partners applying?
  If so, it would be nice if all could mention the model zoo.
  bioimage.io is not yet mature enough to apply on its own

• once the website is more stable, we could start to write user facing docs
  a good example would be integration of the different community partners, e.g. train model in a ZeroCost notebook and run it with another community partner software

• started discussion on interoperability between tools by community partners; big issue is (still :)) the different pre and post-processing steps that are implemented in a language specific manner. For now, porting between tools will probably be done manual by a developer of that tool.

• When to change bioimage.io as landing page / reference model page for all community partners (e.g. for DeepImageJ, CSBDeep, ZeroCost home pages)? Then we could announce on image.sc etc. E.g. First of November. Discuss with Wei. Need to discuss with other community partners.

• Citation discussion; how many citations to put in

Suggestions for the website

How do we make the website more intuitive for users?

Name-changes:

• Change the name "Datasets" to "Training Data"; this should be clear for users.
• Change the name "Models" to "Preconfigured Models"
• Change the name "Notebooks" to "Code"

Could also go with more actionable names:

• "Models" to "Find Models"
• "Applications" to "Preview" or "Try on your data"?
• "Notebook" to "Find Code"
• "Datasets" to "Find Data"
• Get rid of the "Application" button and just provide "Run" or "Demo" button on the model windows

Icons on the Model and Notebook Cards are different for the each Model / Network. This is confusing.

[oeway]

BioImage.IO Meeting Minutes

August 21st, 2020

Attendees

Name	affiliation	username
Wei Ouyang	KTH	@oeway
Emma Lundberg	KTH	emmalundberg
Ricardo Henriques	UCL / IGC	@paxcalpt
Guillaume Jacquemet	ÅAU/UTU	@guijacquemet
Anna	EMBL	@akreshuk

Minutes

• planning for the release of Bioimage.IO
  • release date: November 1st
  • features to offer by then:
    • display plain model cards for download
    • support interoperatability? @emma high priority
      • a list of useful/popular model, make sure it compatible with a few software. (show complementary with software rather than competition)
        • pytorch model (segmentation): ilastik, imjoy, zero
        • tensorflow model (denoising): zero, fiji, deepimagej, imjoy
        • classification: different types
        • starting from a training dataset? task?
    • use outline color encoding for types @anna
    • show text description for tags
    • online inference (single model, save it for the release)
    • model searching application (lower priority)
    • integration with colab, binder
    • community parties
      • guidelines for the community parter
      • short recommendation from all the community partner
      • feature models, sorting, popularity, model searching (lower priority)
    • Group the software by three types of tasks: 1) preprocessing/ annotation 2) training / transfer learning 3) prediction. Click a model, show a list of software grouped by tasks types
    • Improve 2h rendering interval @ricardo
    • parser script should validate the link, valide the fields, cover image size, add badges

[oeway]

September 4th, 2020

Attendees

Name	affiliation	username
Wei Ouyang	KTH	@oeway
Constantin Pape	EMBL	@constantinpape
Anna Kreshuk	EMBL	@akreshuk
Maksim Novikov	EMBL	@m-novikov
Fynn Beuttenmüller	EMBL	@FynnBe
Ricardo Henriques	IGC/UCL	@HenriquesLab
Estibaliz Gómez	UC3M	@esgomezm

Agenda

• Wei
  • Outline color encoding
  • Zarr dataset and Vizarr
  • ImageJ.JS
  • discussion
    • Run DeepImageJ models
    • Model conversion service
      • BioEngine Application running on Binder
      • Tensorflow to Tensorflow.JS
      • PyTorch to ONNX to Tensorflow?
    • RPC link between the website and software
      • From the website: Send model from the website to Ilastik/Fiji etc.
      • From the software: launch model browser
      • Example PycroManager
• Ilastik
  • Specification of training, Application specific?
  • Website header too large?

Minutes

• Add underline color LUT
• What models to use as show case
  • 2D Unet / Nuclei Broad
  • 3D Unet
• Token (url) for linking model across software
• Roadmap general issue
• Training:
  • descriptive training procedure is too ambitious
  • instead: partners can support their own training procedures; training APIs can be shared
  • example: ilastik will have a training API that can also be exposed through imjoy RPC so it could be used from the browser

[oeway]

September 18th, 2020

Attendees

Name	affiliation	username
Wei Ouyang	KTH	@oeway
Anna Kreshuk	EMBL	@akreshuk
Carlos García	UC3M	@carlosuc3m
Estibaliz Gómez	UC3M	@esgomezm
Novikov Maksim	EMBL	@m-novikov
Florian Jug	CSBD + HT	@fjug
Fynn Beuttenmüller	EMBL	@FynnBe
Maksim Novikov	EMBL	@m-novikov

Agenda

• Roadmap general issue
  • Model runners
  • Improve Landing
  • Documentation
  • animal icon?
  • Run button for each model
  • (Simple) models which can be run in multiple applications. E.g. train a model in a notebook in ZeroCost, run in both DeepImageJ and CSBDeep (or ilastik) (note: deal with output error produced by different framework)

• random thoughts from the ilastik side: https://docs.google.com/document/d/1FUqEzEuwEi8Z_XULl4MgisDl-eX0OV6QKBQwxEcBQbk/edit?usp=sharing

• Cross-compatibility of models - how much can we guarantee for a model following the specs?
• Generic model runner (two runners, PyTorch and Tensorflow) to have the same as HPA model runner for all models?
• What should really be in the "Applications" part?
• New applications: CellPose, ImageJ.JS, CellPose-ImageJ.

[oeway]

October 2nd, 2020

Attendees

Name	affiliation	username (github)
Wei Ouyang	KTH	@oeway
Florian Jug	CSBD + HT	@fjug
Dominik Kutra	EMBL	@k-dominik
Jan Funke	Janelia	@funkey
Carlos García	UC3M	@carlosuc3m
Estibaliz Gómez	UC3M	@esgomezm
Arrate Muñoz	UC3M	@arratemunoz

Agenda

• Florian

  • Pitch the model zoo project

• Wei

  • Updates on the bioimage.io website

  • BioEngine API for model runners

    api.registerService({
       "type": "model-runner",
       "init": init, // initialize the runner
       "predict": predict, // perform prediction
       "fit": fit // perform training (optional)
    })

    Alternatively, we can use scikit-learn style with fit and transform function.

    • ilastik
    • deepimagej
      • support via pyimagej
      • support via imagej.js

  • Demo for a new model: Learn2Sofie

• Jan

  • Introduction about the live supervised learning platform at Janelia

• Anna and Wei

  • plan for the November release, implementation of the model runners.

[oeway]

October 30th, 2020

Attendees

Name	affiliation	username (github)
Wei Ouyang	KTH	@oeway
Anna Kreshuk	EMBL	@akreshuk
Tom Burke	CSBD	@tomburke-rse
Florian Jug	CSBD + HT	@fjug
Deborah Schmidt	CSBD	@frauzufall
Arrate Muñoz	UC3M	@arratemunoz
Anna Kreshuk	EMBL	@akreshuk
Novikov Maksim	EMBL	@m-novikov
Fynn Beuttenmüller	EMBL	@FynnBe
Dominik Kutra	EMBL	@k-dominik
Estibaliz Gómez	UC3M	@esgomezm

Agenda

• Discuss the test_inputs and test_outputs format: will be feeded as a list and use npy as the only formats, this will be specifically used for CI testing, but combined with converters, for the users, we can use another field, e.g. test_samples
• Change normalization to preprocessing, and agreed on several types of preprocessing functions
• The model file can contain multiple formats of the same weights, the weights is a dictionary which uses a unique format string as the key

See also in the project: https://github.com/orgs/bioimage-io/projects/1

[oeway]

November 13th, 2020

Attendees

Name	affiliation	username (github)
Wei Ouyang	KTH	@oeway
Anna Kreshuk	EMBL	@akreshuk
Tom Burke	CSBD	@tomburke-rse
Florian Jug	CSBD + HT	@fjug
Deborah Schmidt	CSBD	@frauzufall
Arrate Muñoz	UC3M	@arratemunoz
Anna Kreshuk	EMBL	@akreshuk
Novikov Maksim	EMBL	@m-novikov
Fynn Beuttenmüller	EMBL	@FynnBe
Dominik Kutra	EMBL	@k-dominik
Estibaliz Gómez	UC3M	@esgomezm
Constantin Pape	EMBL	@constantinpape
Carlos García	UC3M	@carlosuc3m

Agenda

• 0.3.0 project: https://github.com/orgs/bioimage-io/projects/1

• Authors in weights vs. authors in model

  • 3 parties to give credit to:
    • Who is the creator of the bioimage.io.model.zip?
      The creator in the yaml points that out. The top-level description describes the training setup, purpose, etc.
    • Who is the creator of the type of model?
      The existing list of citations is good enough!
    • Who trained the linked weights?
      If not the same as the creator of the model, names can be given here. Description field is superfluous and needs to be removed here!

• Weights-formats we support:

  • tensorflow_saved_model_bundle: Zip the weights and the folder with the variables and link to this zip from the yaml-file. (File name: *.zip)
  • keras_hdf5:
  • tensorflow_js: a json file + binary weights files
  • pytorch_state_dict:
  • pytorch_script:

• Naming convention for the model bundle: *.bioimage.io.model.zip

• show a table with features (weights format, pre/post processing types) that are implemented by consumers