Text rendering or collision detection performance issues

[Buthrakaur]

We just migrated from Mapbox to Maplibre and discovered degraded performance especially on iOS (React Native app for both platforms) when zooming in/out or quickly panning the map. We have 3 symbol layers with small/medium/large variants of POI markers to let collision use the largest possible icon available for a nice map UX with hundreds of POIs in a city. I discovered the rendering performance goes massively down when I add a text in the style and especially when I set text halo (textHaloWidth, textHaloColor) so I had to turn off text halo effects to keep at least some acceptable performance. The Mapbox component seems to have some more performance optimization around text collision detection or text rendering in general. It's surprising the Android performance is still acceptable (even though worse than Mapbox) and it's sluggish on iPhones.

I'd appreciate any recommendations on how to improve the performance by changing the style - I ended up with this:

const useTextHalo = Platform.OS === 'android';

const baseStyle: SymbolLayerStyle = {
  textHaloWidth: 2,
  textHaloBlur: 0,
  textAnchor: 'top',
  textMaxWidth: 10,
  textFont: ['get', 'textFont'],
  iconAllowOverlap: false,
  textAllowOverlap: false,
  iconOptional: false,
  iconAnchor: ['get', 'iconAnchor'],
  symbolSortKey: ['get', 'priority'],
  textOffset: [0, 0.1],
};

export const pinLargeStyle: SymbolLayerStyle = {
  ...baseStyle,
  textField: ['get', 'title'],
  textSize: SIZE_L,
  textColor: ['get', 'textColor'],
  textHaloColor: ['get', 'textHaloColor'],
  iconImage: ['get', 'large'],
  iconSize: ICON_SIZE_MULTIPLIER,
};

export const pinMediumStyle: SymbolLayerStyle = {
  ...baseStyle,
  textField: ['get', 'title'],
  iconImage: ['get', 'medium'],
  textSize: ['case', ['==', ['get', 'type'], 'tour'], SIZE_L, SIZE_M],
  textColor: ['get', 'textColor'],
  iconColor: ['get', 'iconColor'],
  textHaloColor: ['get', 'textHaloColor'],
  iconSize: ICON_SIZE_MULTIPLIER,
};

export const pinSmallStyle: SymbolLayerStyle = {
  iconImage: ['get', 'small'],
  iconColor: ['get', 'iconColor'],
  symbolSortKey: ['get', 'priority'],
  iconAnchor: 'center',
  iconSize: ICON_SIZE_MULTIPLIER,
};

if (!useTextHalo) {
  [pinLargeStyle, pinMediumStyle, pinSmallStyle].forEach((style) => {
    delete style.textHaloWidth;
    delete style.textHaloBlur;
    delete style.textHaloColor;
    delete style.textColor;
  });
}


...

  return (
    <>
      <Images images={images} />
      <ShapeSource id={props.layerId} shape={featuresCollection} onPress={onFeaturePress} hitbox={useMemo(() => ({ width: 0, height: 0 }), [])}>
        <SymbolLayer
          id={`SymbolLayer${props.layerId}Small`}
          minZoomLevel={props.minZoomLevel}
          maxZoomLevel={props.maxZoomLevel}
          style={pinSmallStyle}
        />
        <SymbolLayer
          id={`SymbolLayer${props.layerId}Medium`}
          minZoomLevel={props.minZoomLevel}
          maxZoomLevel={props.maxZoomLevel}
          style={pinMediumStyle}
        />
        <SymbolLayer
          id={`SymbolLayer${props.layerId}Large`}
          minZoomLevel={props.minZoomLevel}
          maxZoomLevel={props.maxZoomLevel}
          style={pinLargeStyle}
        />
      </ShapeSource>
    </>
  );

This is the intended result which worked well with Mapbox in terms of performance but is not acceptable with MapLibre on iOS:

[Buthrakaur]

I'm not sure if #2382 could resolve at least part of the problem but it sounds relevant.

[louwers]

Hi @Buthrakaur thanks for sharing your performance-related issue!

Looping in @sjg-wdw. Maybe the rendering team can have a look at it. Do you have the style available as a JSON? Would you be willing to (privately) share your style with MapLibre developers? We're always interested in heavy styles to help improve performance. We have some tracing integrated which can be helpful to study what in particular is slowing things down.

[sjg-wdw]

Could this be a problem with sorting? @TimSylvester

[TimSylvester]

I wouldn't think so, but it doesn't seem like layout or rendering should be significant in that case.

[jakub-oone]

Hi @louwers,

we would really appreciate that your rendering team could take a look at it.

how should I get the style as JSON?? I have just this:

`pinSmallStyle:

{"iconImage":["get","small"],"iconColor":["get","iconColor"],"symbolSortKey":["get","priority"],"iconAnchor":"center","iconSize":0.5}

pinMediumStyle:

{"textHaloWidth":2,"textHaloBlur":0,"textAnchor":"top","textMaxWidth":10,"textFont":["get","textFont"],"iconAllowOverlap":false,"textAllowOverlap":false,"iconOptional":false,"iconAnchor":["get","iconAnchor"],"symbolSortKey":["get","priority"],"textOffset":[0,0.1],"textField":["get","title"],"iconImage":["get","medium"],"textSize":["case",["==",["get","type"],"tour"],14,12],"textColor":["get","textColor"],"iconColor":["get","iconColor"],"textHaloColor":["get","textHaloColor"],"iconSize":0.5}

pinLargeStyle:

{"textHaloWidth":2,"textHaloBlur":0,"textAnchor":"top","textMaxWidth":10,"textFont":["get","textFont"],"iconAllowOverlap":false,"textAllowOverlap":false,"iconOptional":false,"iconAnchor":["get","iconAnchor"],"symbolSortKey":["get","priority"],"textOffset":[0,0.1],"textField":["get","title"],"textSize":14,"textColor":["get","textColor"],"textHaloColor":["get","textHaloColor"],"iconImage":["get","large"],"iconSize":0.5}`

THX!

[louwers]

@jakub-oone We would need the sources as well in order to debug it.

You could use https://maplibre.org/maputnik to create it.

[jakub-oone]

@louwers you mean ShapeSource with shapes? Like this? {"type":"FeatureCollection","features":[{"type":"Feature","id":"627603bd-439a-4573-82cd-459238d07f67","properties":{"id":"627603bd-439a-4573-82cd-459238d07f67","small":"standard_minimised","medium":"see_and_do","large":"map_placeholder_poi_see","title":"Prague Castle","textColor":"rgba(48, 48, 48, 1)","textHaloColor":"rgb(255, 255, 255)","textFont":["Roboto Medium"],"type":"sight","iconAnchor":"bottom","priority":8000000,"isLocked":false,"isVisited":false},"geometry":{"type":"Point","coordinates":[14.4012459,50.0908412]}},{"type":"Feature","id":"dc3e6f6c-19c5-4a92-9c0a-983f009d7469","properties":{"id":"dc3e6f6c-19c5-4a92-9c0a-983f009d7469","small":"standard_minimised","medium":"see_and_do","large":"map_placeholder_poi_see","title":"Charles Bridge","textColor":"rgba(48, 48, 48, 1)","textHaloColor":"rgb(255, 255, 255)","textFont":["Roboto Medium"],"type":"sight","iconAnchor":"bottom","priority":8000001,"isLocked":false,"isVisited":false},"geometry":{"type":"Point","coordinates":[14.411229548783695,50.08621680257775]}}]}

of course there are much more features...

[Buthrakaur]

We were finally able to isolate the issue and prepared a minimum sample repo here: https://github.com/SmartGuideApp/maplibre-test - it's a React Native app so for running it on iOS following sequence needs to be done:

npm i
cd ios
pod install
cd ..
npm run ios

The root cause of the performance problem is style having symbolSortKey: ['get', 'priority'] https://github.com/SmartGuideApp/maplibre-test/blob/main/App.tsx#L6 as suggested by @sjg-wdw . When it's present the map rendering is slowed down when moving the map and especially when zooming in and out. The rendering is back to normal when this line is commented out. The map style doesn't seem to matter - https://demotiles.maplibre.org/style.json is used in the repo. The performance seems to be impacted more significantly on iOS but some smaller performance impact is visible even on Android phones.

[sjg-wdw]

Sort keys are very slow. Do you need them here?

[Buthrakaur]

We are going to stop using them of course as we can sort the features array instead but I believe it would still be great if the performance get fixed at some point of time or at least include a warning in the docs or make this style property obsolete. Mapbox component handles this without any issues - they probably implemented some optimization after the fork was made.

[TimSylvester]

It's expected to be slower because things that can normally be grouped together can't be when sorted, but there may still be some problem making it slower than it should be. From just the screenshot above, it doesn't seem like there are enough instances that it should be noticeable.

[sjg-wdw]

I wonder if there are a lot of symbols not appearing.

[Buthrakaur]

yes - there's over thousand symbols with 3 layers (small/medium/large icons)

[sjg-wdw]

This is starting to make more sense.

Icon and text deconfliction is done in real time in the renderer. It's a legacy of the original design and it works well if most of the things you give it are going to appear on the screen.

At a certain point, when most of the things you give that deconfliction engine are invisible, you're using it wrong. It's only meant to make a few very quick decisions.

MapLibre could help fix this by moving deconfliction out of the main rendering loop so it could take as long as it needs. Working through 1k or even 100k items isn't all that bad if you do it right, just not at 30fps. We did this in WhirlyGlobe and it was just a matter of memory.

But for the short term, you can avoid sorting or just feed it fewer things to deconflict.

[TimSylvester]

I could be wrong, but I don't think sorting directly affects the placement pass. I'd expect it to only apply to things that end up being selected for rendering. Possibly we're sorting the entire list of things and only rendering some, but the sort itself should be cheap, even for thousands of items, it's creating drawables and rendering them that seems most likely to be a problem.

[Buthrakaur]

@TimSylvester I would expect exactly what you are saying (sorting array of ~thousand items being cheap operation => no perceivable effect of using symbolSortKey) but there seems to be some very inefficient code around the sorting or the dynamic data driven sort key retrieval may have some performance penalty?

I recorded a screen recording from our real app if you want to see the difference - I was quickly zooming in and out. This is app version without symbolSortKey where zooming is completely smooth: https://photos.app.goo.gl/QBjdVt4ZbGpH8DqZ8 vs this is app version with symbolSortKey where you can see how the zooming is stuttered: https://photos.app.goo.gl/ek57EdQqCnP2JFWo7 . The only difference is the symbolSortKey presence in the symbol style.

[TimSylvester]

The actual sort is very cheap, it's actually an ordered insert into a list based on a float key.

The other odd part is it being worse on iOS. If that's not a red herring, it implies it h as something to do with rendering itself, pretty much everything else is platform-neutral.

Are you able to get a Metal frame capture with profiling?

[Buthrakaur]

@TimSylvester I'm not fluent in iOS native development so that would be hard for me honestly but if it's easy for you the sample repository #3223 (comment) has exactly the same perf issue - would that be possible, please?

[louwers]

@Buthrakaur Could you try these steps? https://developer.apple.com/documentation/xcode/capturing-a-metal-workload-in-xcode

[TimSylvester]

$ nvm current
v22.14.0
$ npm -v
10.9.2
$ npm -i
npm <command>

Usage:

npm install        install all the dependencies in your project
npm install <foo>  add the <foo> dependency to your project
npm test           run this project's tests
...
$ npm install
up to date, audited 983 packages in 1s
$ cd ios
$ gem install cocoapods
Successfully installed cocoapods-1.16.2
$ pod install
.../ruby-2.7.0/lib/ruby/2.7.0/rubygems.rb:275:in `find_spec_for_exe': can't find gem cocoapods (>= 0.a) with executable pod (Gem::GemNotFoundException)
$ rvm use macruby
Using .../.rvm/gems/macruby-0.12
$ pod install
.../Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems.rb:283:in `find_spec_for_exe': can't find gem cocoapods (>= 0.a) with executable pod (Gem::GemNotFoundException)
   from .../Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems.rb:302:in `activate_bin_path'
   from /usr/local/bin/pod:23:in `<main>'

[Buthrakaur]

@TimSylvester thanks for giving it a try. I use Mac only seldomly so I'm not sure what can be wrong with CocoaPods on your computer but there's some docs here: https://reactnative.dev/docs/set-up-your-environment?os=macos&platform=ios#cocoapods or here: https://guides.cocoapods.org/using/getting-started.html

[Buthrakaur]

I can see in the manual sudo may be required sudo gem install cocoapods

[TimSylvester]

sudo didn't help, but bundle install got me past that error. The xcode build fails with Library 'DoubleConversion' not found though.

[Buthrakaur]

@TimSylvester bundle install is not the same as pod install so I believe that's why you're getting the build error as CocoaPods are still not installed or did you run successfully pod install after bundle install? After pods are installed please make sure you open the xcworkspace file and not xcodeproj. We actually just pushed a fix to the repo to the Podfile where $MLRN.post_install(installer) was missing so please pull the changes. I'm not sure if that may caused the error too.

[TimSylvester]

I mean that after running bundle install, then pod install succeeded.

Now I get Dependencies could not be resolved because root depends on 'maplibre-gl-native-distribution' 6.11.0 and root depends on 'maplibre-gl-native-distribution' 6.10.0.