|
| 1 | +--- |
| 2 | +blogpost: true |
| 3 | +date: Jan 20, 2025 |
| 4 | +author: Grzegorz Bokota |
| 5 | +location: World |
| 6 | +category: news, help-wanted |
| 7 | +language: English |
| 8 | +--- |
| 9 | + |
| 10 | +# Triangles Speedup – call for beta testers |
| 11 | + |
| 12 | +We are excited to announce that significant performance improvements are coming to napari Shapes layers. |
| 13 | + |
| 14 | +Shapes layers in napari represent 2D geometric objects, — rectangles, circles, |
| 15 | +polygons, paths… — possibly embedded in a higher-dimensional space, for |
| 16 | +example, 2D polygons of cell outlines within a 3D image stack. |
| 17 | +[Vispy](https://vispy.org), which powers napari's graphics, uses OpenGL to draw |
| 18 | +on the screen. The fundamental unit of OpenGL graphics is *triangles*, which |
| 19 | +can be put together to draw more complex shapes such as polygons. This means |
| 20 | +that we have a preproprocessing step in napari to break down input shapes into |
| 21 | +triangles. This step is called *triangulation*. |
| 22 | + |
| 23 | +Until now, we have been using an algorithm called Constrained Delaunay |
| 24 | +Triangulation |
| 25 | +([1](https://doi.org/10.1007/BF01553881),[2](https://www.cs.jhu.edu/~misha/Spring16/Chew87.pdf)), |
| 26 | +implemented in pure Python in Vispy. This has been a performance bottleneck |
| 27 | +when creating Shapes layers with thousands to hundreds of thousands of shapes. |
| 28 | +(An optional dependency [`triangle`](http://www.cs.cmu.edu/~quake/triangle.html) |
| 29 | +can be used to speed things up, but because it uses a proprietary license, we |
| 30 | +cannot ship it by default and we cannot use it in the napari bundled app.) |
| 31 | + |
| 32 | +Thanks to the [SpatialData](https://spatialdata.scverse.org/) community, which |
| 33 | +decided to sponsor this work, we were able to implement a faster algorithm for |
| 34 | +rendering triangles used for rendering geometries in a napari shapes layer. |
| 35 | +(`SpatialData` is a framework for the representation of spatial multimodal omics |
| 36 | +data and part of the single-cell omics analysis suite |
| 37 | +[scverse](https://scverse.org/).) |
| 38 | + |
| 39 | +We have tested the new implementation on a few example datasets of SpatialData, |
| 40 | +and we see a significant speedup. For example, in this [Xenium Human Lung |
| 41 | +Cancer dataset from 10x |
| 42 | +Genomics](https://www.10xgenomics.com/datasets/preview-data-ffpe-human-lung-cancer-with-xenium-multimodal-cell-segmentation-1-standard) |
| 43 | +(available in the SpatialData Zarr format using [these |
| 44 | +scripts](https://github.com/giovp/spatialdata-sandbox/tree/main/xenium_2.0.0_io)), |
| 45 | +the cell boundaries are stored as 162,000 polygons. When visualizing these |
| 46 | +polygons in napari, creation of the shapes layer drops from almost 4 minutes |
| 47 | +(napari 0.4.19) to just 20 seconds! (Ubuntu 20.04 with Intel Core i7-8700 CPU @ |
| 48 | +3.20GHz) |
| 49 | +Most of the time creating a Shapes layer with so many shapes was spent on |
| 50 | +triangulation, which takes just 2.5s with our changes. |
| 51 | + |
| 52 | +We obtain this dramatic speedup through a combination of algorithmic changes |
| 53 | +(using the sweeping line triangulation algorithm from |
| 54 | +[3](https://doi.org/10.1007/978-3-540-77974-2)) and code changes — |
| 55 | +implementing the algorithm in C++ instead of pure Python. |
| 56 | + |
| 57 | +For now, our fast triangulation is implemented in |
| 58 | +`PartSegCore-compiled-backend`, a compiled Python package I was already |
| 59 | +distributing for [PartSeg](https://partseg.github.io), which is the main |
| 60 | +software output of my PhD and the work that brought me to napari and eventually |
| 61 | +the napari core team. |
| 62 | +In the near future we plan to create the `bermuda` package (yes, because of the |
| 63 | +Bermuda triangle — and with thanks to [Aperio's](https://aperiosoftware.com) |
| 64 | +Thomas Robitaille a.k.a. astrofrog for the PyPI package name!), which will |
| 65 | +contain fast spatial algorithms for the napari Shapes layer and beyond. |
| 66 | +We plan to develop in Rust which [seems to give us even faster |
| 67 | +performance](https://github.com/napari/bermuda/pull/1) on top of better memory |
| 68 | +safety guarantees. Plus all those shipwrecks are probably pretty rusty! |
| 69 | + |
| 70 | +In the meantime, this is brand-new work and the first foray into compiled code |
| 71 | +directly for napari, so we are looking for beta testers. If you use the Shapes |
| 72 | +layer, please read on for how you can help, while also benefit from the |
| 73 | +increased performance that this work brings! |
| 74 | + |
| 75 | +## How to use it *today* |
| 76 | + |
| 77 | +This feature requires napari in version at least 0.5.6. As of this writing, a |
| 78 | +pre-release version of napari 0.5.6 is available on PyPI. |
| 79 | + |
| 80 | +You can install the pre-release using `pip` by specifying a version: |
| 81 | + |
| 82 | +```bash |
| 83 | +pip install "napari[optional,pyqt6]>=0.5.6rc0" |
| 84 | +``` |
| 85 | + |
| 86 | +The "optional" flag ensures that you get the compiled extensions mentioned in |
| 87 | +this post. |
| 88 | + |
| 89 | +Since this code is experimental, we have made it opt-in in napari preferences |
| 90 | +to enable using it. To enable it, open napari, and open the menu File → |
| 91 | +Preferences, then open the Experimental tab and enable the checkbox for "Use |
| 92 | +C++ code to speed up creation and updates of Shapes layers". |
| 93 | + |
| 94 | + |
| 95 | + |
| 96 | +You can also toggle it using the `COMPILED_TRIANGULATION` environment variable, |
| 97 | +for example launching `napari` using the terminal command |
| 98 | +`COMPILED_TRIANGULATION=1 napari`, or `COMPILED_TRIANGULATION=1 jupyter lab` |
| 99 | +and then using napari within Jupyter. |
| 100 | + |
| 101 | +If you spot any issues (visual or functional), please let us know in our [Zulip |
| 102 | +chat room](https://napari.zulipchat.com/), or create an issue on [our GitHub |
| 103 | +repo](https:/github.com/napari/napari/issues). You are also welcome to come ask |
| 104 | +for help in our [community |
| 105 | +meetings](https://napari.org/dev/community/meeting_schedule.html). |
| 106 | + |
| 107 | +## Ongoing work |
| 108 | + |
| 109 | +During our own testing, we sometimes observed crashes because of |
| 110 | +[floating point precision](https://0.30000000000000004.com) issues. |
| 111 | +Although we've worked around those issues for now, it's possible more issues |
| 112 | +are lurking. Ultimately, we may need to move to algorithms resistant to |
| 113 | +floating point approximation errors, which are known as *numerically stable* |
| 114 | +methods. |
| 115 | + |
| 116 | +This work has focused on the initial triangulation step in the construction of |
| 117 | +napari Shapes layers. We haven't (yet) updated the triangulation code when |
| 118 | +*editing* layers with hundreds of thousands of shapes. Currently, all triangles |
| 119 | +in a Shapes layer live in a single big NumPy array — which means that if we |
| 120 | +update a shape, the whole array needs to be recomputed! 😵 This is fine for |
| 121 | +small datasets but can result in freezes of several seconds when editing or |
| 122 | +adding shapes to a list of 100,000+. There's lots of ways around this but we |
| 123 | +have yet to implement any — so, unfortunately, performance issues while editing |
| 124 | +*may* not be improved by this work. |
| 125 | + |
| 126 | +napari has longstanding issues displaying polygons with holes in them. This |
| 127 | +work does not address them but opens the door to doing so very efficiently. If |
| 128 | +you've been waiting for that, stay tuned! |
| 129 | + |
| 130 | +Finally, as mentioned above, our immediate next step is to move the code to |
| 131 | +Rust in the [bermuda](https://github.com/napari/bermuda) package. We hope to |
| 132 | +make it a core dependency of napari sooner rather than later, which will open |
| 133 | +the door to *many* speedups in napari! 🚀 So look forward to that. |
| 134 | + |
| 135 | +## Call for help 2 |
| 136 | + |
| 137 | +All this work is happening because Python (and Pythonic) libraries for spatial |
| 138 | +algorithms are pretty thin on the ground. We're also mostly noobs at Rust so if |
| 139 | +you are interested in fast spatial algorithms, Rust, and n-dimensional data |
| 140 | +visualization, we could use your help! Please have a look at our [Community |
| 141 | +page](https://napari.org/stable/community/index.html) and join us in our [Zulip |
| 142 | +chat room](https://napari.zulipchat.com) or come to a [meeting suitable for |
| 143 | +your time zone!](https://napari.org/stable/community/meeting_schedule.html#meeting-schedule) |
| 144 | +We'd love to hear from you. |
| 145 | + |
| 146 | +## Acknowledgments |
| 147 | + |
| 148 | +This work is made possible by a [Data |
| 149 | +Insights](https://chanzuckerberg.com/science/programs-resources/cell-science/data-insights/) grant from the Chan Zuckerberg Initiative to |
| 150 | +the SpatialData team. We want to acknowledge the team's support for napari |
| 151 | +upstream, and highlight how individual grants can be used to support collective |
| 152 | +software efforts that help thousands of users. We will have more to say on this |
| 153 | +in an upcoming blog post. Finally, we also want to thank the [scverse |
| 154 | +team](https://scverse.org/), a consortium of open-source developers dedicated |
| 155 | +to single-cell omics. |
| 156 | + |
| 157 | +Luca Marconato, Juan Nunez-Iglesias, Peter Sobolewski, and Wouter-Michiel |
| 158 | +Vierdag helped with the writing of this post. |
0 commit comments