Super slow rendering on very wide datasets - display columns only when necessary

[Tagar]

Apache Zeppelin recently upgraded to ui-grid - that looks awesome btw!

What we're facing though is on wider datasets Chrome sometimes spends minutes to render a table.
The table doesn't have a lot of data (it's very sparsely populated), so it seems just the number of columns is what important.

Maybe more of an extreme case - 5000 columns x 1000 rows recent Chrome spent 20-30 minutes to render on a decent PC. That's not a make-up case and we actually had to display such a wide sparsely populated table. (Our company normally deals with wider datasets)

Even on not so wide datasets we found browser rendering could be lagging..

What is a recommended size of dataset (in terms of number of columns, and number rows) that is know where ui-grid works well?

As an example, Cloudera HUE worked around displaying such wide datasets by adding "virtual" horizontal scrolling - basically columns are pushed to an html table only when they have to be visualized (if a user has pushed horizontal scrolling to the right .. and efficiently removing columns from html when they're not necessary for visualization). Cloudera Hue can render such wide datasets very quickly because browser has to render only a tiny subset of columns that it actually needs.

[kpats]

Same issue.. I think ui-grid is providing vertical(row) virtualization but not horizontal. It'll be good if that can be done.

[kpats]

Also the performance is horrible on IE even for smaller number of columns(say 200 columns) than chrome as there is no horizontal virtualization.

[JLuitjens]

The row virtualization settings have power in cases like this, should be a way to dynamic determine that and a threshold limit because it definitely does improve the performance of grids when you work with arbitrary grid lengths obviously the performance depends on the amount of grid data available

These options tend to help in performance for vertical, but needs to be wide handled as well, perhaps too many wide columns over rows is the wrong way of thinking the data, should be more rows vs columns?

fastWatch: true,
flatEntityAccess: true,
virtualizationThreshold: 150,

[hanishw]

When wider data is used (Columns >= 100), grid becomes sluggish in scrolling and rendering.
On increasing columnVirtualizationThreshold to say 100, horizontal scrolling smoothens but vertical scrolling degrades.

I guess that's due to more watchers being fired.

Setting:

fastWatch: true and
flatEntityAccess: true,

makes no difference.

[hanishw]

I could fix the issue by letting watchers fire individually for rows and columns.
I was earlier updating gridOptions.colDefs and gridOptions.data together. Then changed it to:

1. gridOptions.colDefs = new settings...
2. $timeout(function() { gridOptions.data = new data... },500, true);

And it works like charm. Obviously, one can adjust timeout values by hitting and trialing out on data and columns length.