[css-color] [filter-effects] Should no-op filters produce different output from no filter?

[smfr]

The filter property states that "Functions must operate in the sRGB color space". This implies that all the input colors to filters, like display-p3 colors and images with P3 color profiles get flattened to sRGB before filter processing.

This has the unfortunate side effect that the results of filter: none and filter: saturate(1) are different when the filtered content contains P3 colors, and that authors can't apply nice filter effects to P3 images while retaining out-of-sRGB colors.

Chrome appears to not follow this spec text precisely, and seems (on macOS at least) to apply filters in some extended sRGB colorspace (tested on iMac Pro with DisplayP3 display).

Testcase: https://codepen.io/smfr/pen/PoOVmPX

cc @svgeesus

[svgeesus]

Certainly in 1998-2000 the SVG spec was created as sRGB-only. The color-interpolation-filters property has the values auto | sRGB | linearRGB where linearRGB is the initial value, and corresponds to the CSS Color 4 srgb-linear colorspace which (like all the predefined RGB spaces there) is unbounded.

There is an open issue SVG color-interpolation-filters to use more color spaces which could be solved by adding additional values to color-interpolation-filters. As you say, another option is to allow extended sRGB as a way to access WCG colors, which would then not require new values on that property but would require careful review of wording about clipping out-of-range values. Such as this one:

The RGBA result from each filter primitive will be clamped into the allowable ranges for colors and opacity values. Thus, for example, the result from a given filter primitive will have any negative color values or opacity values adjusted up to color/opacity of zero.
https://drafts.fxtf.org/filter-effects/#FilterPrimitivesOverviewIntro

Note that processing in unbounded srgb-linear should give the same result as processing in CIE XYZ, both are linear-light.

Notice too that feColorMatrix has the sRGB-linear to XYZ conversion matrices baked in for saturate, hue rotate and luminance. So using unbounded srgb-linear as the filter working space does seem a promising approach.

The only worry is whether existing content is depending on the clamping behavior; if so, some form of opt-in might be needed. Perhaps a linearRGBextended value, or some such?

[svgeesus]

Related: Impact of colorspace on filters

[svgeesus]

So: given that

• SVG filters already operate in linear-light sRGB, and the sRGB to XYZ conversion matrix is baked into the spec with the saturation and luminance operators
  • but they assume 8-bit and hard clamp at 0 and 255 on each stage in the filter chain so can't do WCG
• extended, linear-light sRGB is already defined in CSS color 4 as srgb-linear]() and can represent wide color gamut colors, such as P3 colors
  • plus it is already proposed for use in Canvas for WCG
  • plus there is existing GPU hardware support for extended, linear-light sRGB
• color-interpolation filters exists in Filter Effects 1 with the values auto | sRGB | linearRGB and the initial value is linearRGB

Then I propose to extend color-interpolation filters to take an additional value with a definition link to CSS color 4, and that value is explicitly an extended, WCG space and has a 16-bit precision requirement.

Extended means that negative values, and values greater than 1.0, are allowed. For example color(display-p3 0 1 0), which is outside the sRGB gamut, is color(srgb-linear -0.225 1.0421 -0.079)

This will add WCG support to SVG filters, is opt-in so existing content continues to render the same, and can also be opted into to get the improved precision (8 bits linear light has visible banding) even for sRGB-only content.

The new value could be sRGB-linear for compatibility with Canvas and with CSS Color 4, although above I suggested linearRGBextended which I could also live with.

I'm guessing this would go in Filter Effects 2 ?

[smfr]

A new value for color-interpolation-filters seems OK, but because it would be opt-in for web developers, doesn't address my initial concerns. I'd prefer that no-op CSS filters (like blur(0), saturate(1) etc) act like no filter is applied, which means they can't use sRGB interpolation by default. I'd prefer this for two reasons:

1. Web developers working on sRGB displays don't unintentionally clamp filtered Display-P3 assets to sRGB inside no-op filters
2. User agents can optimize away no-op filters from their rendering pipeline

Doing this would require a spec change along the lines of "CSS filters operate in the current compositing colorspace" (with some handwaving because we specify that the working colorspace is sRGB but that's not what browsers do now).

[svgeesus]

OK so this would be additional wording on no-op filter in particular (and means that saturate(1) and saturate(0.99) work differently) but is certainly doable.

Doing this would require a spec change along the lines of "CSS filters operate in the current compositing colorspace" (with some handwaving because we specify that the working colorspace is sRGB but that's not what browsers do now).

What do they do now? Do they all do the same thing?

[smfr]

What do they do now? Do they all do the same thing?

WebKit does compositing in the colorspace of the display. I don't know what Firefox and Chrome do.

[css-meeting-bot]

The CSS Working Group just discussed [css-color] [filter-effects] Should no-op filters produce different output from no filter?.

The full IRC log of that discussion

<fantasai> topic: [css-color] [filter-effects] Should no-op filters produce different output from no filter?
<fantasai> github: [css-color] [filter-effects] Should no-op filters produce different output from no filter?
<fantasai> github: https://github.com//issues/7100
<fantasai> smfr: Issue here is that CSS filters are specified to use sRGB as their color interpolation model
<chris> they use linear-light sRGB
<fantasai> smfr: This is a problem when authors apply filters to colors outside of sRGB gamut
<fantasai> smfr: e.g. display-p3 or color specified with LCH color space
<chris> q+
<fantasai> smfr: This is important because often authors will apply no-op filters, with intent to animate later
<fantasai> smfr: This means such filters will flatten the colors
<fantasai> smfr: In testing, looked like Chrome isn't doing this: they allow out-of-sRGB colors through filters
<fantasai> smfr: WebKit does limit colors to sRGB
<fantasai> smfr: Don't think this is desriable
<fantasai> smfr: One, I think authors expect no-op filters to not have an effect
<fantasai> smfr: And second as an implementer, I'd like to optimize away such filters by not doing anything special
<fantasai> smfr: Since CSS specifies sRGB compositing and filters are specified to do this also
<fantasai> smfr: CSS uses non-linear sRGB
<fantasai> smfr: so I'm not sure how to resolve this, because I don't think we specify our color compositing model in a way that matches browsers
<Rossen_> q
<fantasai> smfr: maybe Chris has an opinion
<fantasai> chris: Correct that filters use sRGB and clamp at each stage in the pipeline
<Rossen_> ack chris
<fantasai> chris: They do use linear-light sRGB
<fantasai> chris: They don't use gamut-?? values
<fantasai> chris: Very obvious when you're doing it wrong
<smfr> s/gamut-??/gamut-encoded/
<fantasai> chris: I suggested a way forward for this, beefing up filters to have a wider color gamut
<fantasai> chris: Simon said very nice, but doesn't solve my issue which is no-op filter
<fantasai> chris: My concern about is that as soon as someone starts to animate, from 0 to 0.01, then there's a sudden change
<fantasai> chris: I can see that it would help with no-op filters, but not with an animation case
<fantasai> smfr: That's very true, and that's why I think we need to specify that CSS filters interpolate in the working color space, so that they don't truncate colors?
<fantasai> chris: The concept of a working color space isn't well-defined
<fantasai> chris: we have different ones for different operations
<fantasai> chris: Talkinga bout compositing, but not just compositing, it's also the convesion from luminance or conversion from saturation, all use constants from sRGB and not display-p3
<fantasai> chris: need to fix that, and the way to fix that is to continue using sRGB ones, and because in linear-light make them extended, and in that way can represent all other colors as well
<fantasai> chris: I think that's the best minimal change, otherwise you have [problems]
<fantasai> smfr: That would be fine...
<Rossen_> q?
<fantasai> chris: If we can bash this out, would be great
<fantasai> smfr: Would be interested in hearing from Chromium and Gecko, what their filter color space is
<fantasai> chrishtr: I'll have to check
<fantasai> emilio: same here
<fantasai> emilio: but pretty sure we have code to optimize away no-op filters
<fantasai> Rossen_: so, do more investigation on Chrome and Gecko side and then come back to this?
<fantasai> chris, smfr: sounds fine
<chrishtr> will investigate

[ccameron-chromium]

What do they do now? Do they all do the same thing?

WebKit does compositing in the colorspace of the display. I don't know what Firefox and Chrome do.

Chrome does compositing in the colorspace of the display. We also do rasterization in the colorspace of the display.

There are some exceptions. If the display has a color space that is too far from sRGB-like interoplationbehavior (e.g, has a linear or PQ transfer function), then we do composite and raster in extended-sRGB, and do a final blit converting to linear or PQ.

We haven't carefully quantified exactly what "too far from sRGB-like interpolation behavior" means (you can examine the current logic here).

[LeaVerou]

Then I propose to extend color-interpolation filters to take an additional value with a definition link to CSS color 4, and that value is explicitly an extended, WCG space and has a 16-bit precision requirement.

P3 can get away with 8 bit, so I'm worried that requiring 16 bit will discourage implementations.

[svgeesus]

P3 can get away with 8 bit, so I'm worried that requiring 16 bit will discourage implementations.

No. Gamma-encoded P3 can get away with 8 bits per component, agreed. Linear-light needs considerably more bits as the space is not perceptually even, so avoiding banding requires more bits.

To see this, iterate from 0 to 255 and at each stage

• convert to float 0..1
• convert to linear-light
• multiply by 2^n, round (to simulate using n bits)

Then count how many unique values there are. I wrote code to do this and 12 bits precision was the minimum to retain 256 unique values. (That code is for srgb-linear, but display-p3 and srgb use the same transfer function).

[svgeesus]

In addition, covering the full gamut of display-p3 in srgb-linear requires more than 8 bits of gamma-encoded values because the range is a bit greater than 0.0 - 1.0. For example, color(display-p3 1 0 0) is color(srgb-linear 1.22494 -0.0421 -0.0196)

[css-meeting-bot]

The CSS Working Group just discussed [css-color] [filter-effects] Should no-op filters produce different output from no filter?.

The full IRC log of that discussion

<dael> Topic: [css-color] [filter-effects] Should no-op filters produce different output from no filter?
<dael> github: https://github.com//issues/7100
<dael> smfr: This is the problem where filter like blur-0 flattens p3 colors to srgb per spec but impl, chrome always and some webkit will propegate p3 colors through filters
<Rossen_> q?
<dael> smfr: chris suggested new value for color interpolation, but doesn't solve problem i'd like. Spec doesn't match impl and I'd like them not to flatten P3 colors
<dael> chris: suggestion seemed to be copositing is in device color space. All filter operations or just compositing?
<dael> smfr: I believe impl are running filter operations in display color space which is P3 and close to srgb
<dael> chris: Looks okay but is wrong. What I suggested would fix that but doesn't fix existing issue
<dael> chris: Slightly concerned about making it do what you want b/c makes it hard to test if it's behaving correct. not completely opposed but want ot get it right
<dael> smfr: svg filters are spec in srgb color space and all constants work in srgb. One approach would be how to spec those in lab color space so impl can filter in that
<dael> chris: lab not good choice. svg are linear srgb not gamma encoded. I suggested an extended linear srgb. outside P3 clor would be fine. if want to upgrade whole thing to another color space then xyz would be a better choice then lab.
<dael> chris: doing in extended linear srgb would give you same result if not clipping
<dael> smfr: Might be possible
<dael> chris: You and I are attaching different bits of problema nd maybe can do both
<dael> smfr: You mean add new color interopolation and change default?
<dael> chris: yeah
<dael> Rossen_: Possible in current impl or compat issue?
<dael> smfr: Won't match what impl do. Would be signifinct work in one webkit codepath
<dael> Rossen_: Academically good, but would be avoided by impl
<dael> smfr: Our slow path, CPU based which is 8-bit unsigned. We'd have to change that to floating point or non-8bit
<dael> smfr: Our codepath with GPU is assuming srgb so applying that to values in P3 which happens to work most of the time. Probably what Chrome does too
<dael> Rossen_: chrishtr do you know if that's what youre doing?
<dael> smfr: For GPU accelerated you're feeding through P3 color values when using srgb matrix?
<dael> chrishtr: not sure, need to talk to Chris Cameron (?)
<dael> smfr: Would be good to see if willing to change to chris suggestion
<dael> chrishtr: Is that in the issue?
<dael> smfr: Yes, thing so
<dael> chrishtr: I'll re-read and ask Chris
<dael> chris: Please also get back to me in issue if something not clear. linear srgb extended should be GPU friendly
<dael> smfr: This should come up with canvas filters too so someone should think about that case too
<dael> Rossen_: Sounds to me like we need to have more eyes on the problem
<dael> chris: Agree
<dael> Rossen_: Want to capture proposed combined solution
<chrishtr> +1 to issue
<dael> chris: I'd rather write it on the issue instead of try and word on the fly
<dael> Rossen_: Alright, please add to issue
<dael> Rossen_: Anything else on this?

[heycam]

I'd prefer that no-op CSS filters (like blur(0), saturate(1) etc) act like no filter is applied, which means they can't use sRGB interpolation by default. I'd prefer this for two reasons:

1. Web developers working on sRGB displays don't unintentionally clamp filtered Display-P3 assets to sRGB inside no-op filters
2. User agents can optimize away no-op filters from their rendering pipeline

One argument against this is that we generally want to avoid discontinuities at special values. If we're animating between filter: blur(1) and filter: blur(0) for example, it would look odd if at the blur(0) state the colors changed because we're no longer flattening to sRGB.

[ccameron-chromium]

At the risk of derailing this discussion, my feeling on this issue is that we should add a feature to the web where the "working color space" of an element may be specified. This would be similar to a Canvas2D's color space. The behavior is equivalent to: all inputs are converted to that space, and then all interpolation (blending, filters, etc) is done in that space.

The default working color space today is "something not too perceptibly different from sRGB". The wiggle room there is so that compositors, etc, can make concessions to efficiency (e.g, working directly in the output device's color space).

With the ability to specify a working color space, the page's author can decide if they prefer "srgb-linear" or if they want "lab", or any of the other options we are exposing. There is no "right" space for interpolation for all applications (the main dividing line is content to have luminance that is physically linear versus perceptually uniform).

An issue that came up here is "what precision should we provide?". When exposing "display-p3" to Canvas2D, this was also a concern. The phrasing there came out as: Should the selection of a buffer's color space imply a certain precision for that buffer? That was a hard question, and we punted by only exposing "display-p3", where, like "srgb", the most reasonable default would be 8-bit-per-color (we excluded "rec2020" both because the transfer function for it had some contention, but also because 8-bit-per-color is not a reasonable default for that space). For a space like "srgb-linear", 8-bit-per-color is very-not-good. When a developer selects that, should we have a table of "minimum precision for each color space"? Or should the developer have to specify a minimum precision (and default to 8-bit if unspecified, which will look bad, but, if all browsers make it look bad, developers will not accidentally do it).

Re this question:

For GPU accelerated you're feeding through P3 color values when using srgb matrix?

To clarify, this is: Suppose the user specifies a color filter matrix M. Suppose we have an input color x in P3 space. What is the output in P3 space?

Option A: (1) let y= x converted to sRGB, (2) let z=My, (3) let w= z converted to to P3, write w to the P3 output buffer
Option B: write Mx to the P3 output buffer

My understanding is that we do Option B in all paths (GPU and CPU).

[svgeesus]

my feeling on this issue is that we should add a feature to the web where the "working color space" of an element may be specified.

Already considered:

• [css-color] Working Color Space #300
• [css-color] What happens to rgb() etc with Working Color Spaces? #481
• [css-color-4] Effect of working color space on computed & used values #3844

and rejected because, as you said:

There is no "right" space for interpolation for all applications (the main dividing line is content to have luminance that is physically linear versus perceptually uniform).

So yes, the appropriate colorspace depends on the operation being done so will vary for different properties on a given element as one size does not fit all. Basically the choices come down to:

1. Linear-light for simulating physical properties (CIE XYZ is a good choice here, linear-light sRGB also gives the same result as long as extended values (outside 0-1) are handled without clipping). This is what compositing should use, but doesn't; it is what svg filters use, except clipped to the sRGB gamut due to the age of the spec.
2. Perceptually linear, either rectangular or polar, where midpoints of an interpolation need to be visually in the middle of the two colors being interpolated. OKLab is the best choice here although CIE Lab is also acceptable. This is what CSS Gradients use, for non-legacy color formats or by opt-in.
3. Gamma-encoded sRGB or indeed gamma-encoded display RGB, if backwards compatibility is a more import a concern than either linear-light or perceptual linearity.

An issue that came up here is "what precision should we provide?". When exposing "display-p3" to Canvas2D, this was also a concern. The phrasing there came out as: Should the selection of a buffer's color space imply a certain precision for that buffer? That was a hard question, and we punted by only exposing "display-p3", where, like "srgb", the most reasonable default would be 8-bit-per-color

That was a good choice for Canvas, given that dealing with greater than 8 bits per component was deferred until later. For the color() syntax in CSS Color 4, the required precision per colorspace is listed in the spec. Note that compared to Canvas, which needs to store color values per-pixel, in CSS there is a need to only store color values per property declaration, greatly reducing the memory impact.

(we excluded "rec2020" both because the transfer function for it had some contention,

that issue was solved a while ago

but also because 8-bit-per-color is not a reasonable default for that space).

Right - it needs minimum 10, preferably 12 bits and Rec BT.2020 gives values for 10bit, 12bit and float representations.

For a space like "srgb-linear", 8-bit-per-color is very-not-good.

Right. 12 is the minimum here, to round-trip all 265 gamma-encoded 8 bit values.

[LeaVerou]

Also, CSS is not like Photoshop where at all times you are one person working on one document. In CSS you may have effects coming in from different sources, so having a single working space doesn't make sense.

[chrishtr]

@svgeesus and @smfr, I think @ccameron-chromium 's comment above answers the question you wanted answered (with the "option B" sentence), let me know if you need more info.

[ccameron-chromium]

my feeling on this issue is that we should add a feature to the web where the "working color space" of an element may be specified.

Already considered:

• [css-color] Working Color Space #300

My thought on this was to have it be something not for the whole page, but could be set to apply at a lower level (e.g, one set of contents wants physical linear blending, but then some sub-content inside that wants perceptual linear blending within itself, etc). We can iterate and discuss at a later date (this thread was only slightly related to that feature).