Screen orientation change events as a special kind of user activation

[marcoscaceres]

A common use case on video sites is:

1. A user starts playing a video on their phone in portrait mode.
2. After a few seconds, the user decides they are enjoying the video... so they rotate their phone into landscape mode.
3. Rotating the device triggers a "user initiated" orientation change event.
4. The site can then call .requestFullscreen() and have such orientation changes be classified as "triggered by user activation".

HTML currently specifies "activation triggering input events", but excludes orientation change events as counting as having "transient activation".

The Screen Orientation spec had some hacky text that would magically label these specific orientation events as having been initiated by the user. And the Fullscreen API was depending on that. However, it would be good to have a generic mechanism for labelling these Events properly and having them participate in the user activation model (i.e., they are classified as being part of "activation triggering input events").

[marcoscaceres]

Perhaps a simple solution might be to add a isUserInitiated (bool) either as IDL attribute or internal slot on the Event interface. Then HTML can check for that being true to count as transient activation.

@annevk, @domenic, @mustaqahmed, @michaelwasserman thoughts?

[domenic]

How would that be different from isTrusted?

[marcoscaceres]

The problem is calling screen.orientation.lock(x) can potentially trigger an orientation change.

So, if you have a PWA that is presented in full screen via web manifest (not via .requestFullscreen()), the user agent can't distinguish between a user-initiated orientation change and a screen orientation API initiated orientation change.

(I know you are not proposing this, but) making such events not trusted wouldn't be right, as they are user agent generated events.

[domenic]

I'm hesitant about exposing this as an API until we've done more of a survey of use cases and other places where it might apply, but exposing it as an internal slot seems reasonable. Then we only have to worry about getting it right on specific places where callers need to check it.

Some interesting related notes:

• element.click() already breaks the contract of "if it's user-agent generated, we set isTrusted to true". Although this case is borderline (is it really trusted, if the "user agent code" is basically a wrapper function for dispatchEvent()??), we generally regret it. So indeed, probably following that precedent is not good. Maybe if we'd had something like what you're proposing here, we could have avoided that.

• The navigation API (being merged into HTML soon, I promise!) has a somewhat-similar concept, but I guess it is not overlapping in the end. For its use cases, isTrusted suffices: https://wicg.github.io/navigation-api/#new-user-navigation-involvement

[domenic]

For the internal slot route, we could even be very specific for now, and just have something like "is a user-initiated orientation change event" or "is a special activation triggering event".

[marcoscaceres]

Actually, .isTrusted could work if screen.orientation.lock() consumes the user activation, because then there would be no opportunity to call .requestFullscreen() or any other API gated on user activation.

However, there is some concern that changing the Screen Orientation spec might not be web compatible: w3c/screen-orientation#218

But .isTrusted does provide a really elegant solution if we can change the orientation spec.

[marcoscaceres]

Actually, yeah, thinking about what you said above @domenic... a very specific slot might make most sense because we only have one very specific use case: changing orientation should allow entering/exiting full screen.

It would be bad if changing the orientation would allow, for instance, a popup window, initiate a payment request, initiate a web share, etc.

[michaelwasserman]

A specific internal slot seems like the most appropriate option presented (but I'm still a relative novice here).

[smaug----]

element.click() is a horrible special case (like is dispatching click events from JS), because of legacy reasons. Effectively click handling doesn't care about isTrusted.

It would be quite confusing if UA dispatched event had isTrusted sometimes true, sometimes false. isTrusted is supposed to mean whether UA dispatched it or it was dispatched from JS. That behavior is even documented in the DOM spec.

[marcoscaceres]

Agree, let's put the .isTrusted aside and figure out where to label this.

The other problem is that even though it originates from an event, this needs to available at the window-wide (same as transient activation).

That is, the UA should be to check:

if "is a user-initiated orientation change event" OR "has transient activation" then ... enter fullscreen; otherwise reject.

But "is a user-initiated orientation change event" has to expire with the same timing model as transient activation (and probably shouldn't count as sticky activation).

[domenic]

Although I can see how orientation change allowing window.open() or PaymentRequest is weird, I will say that this will all be a lot easier if you reuse the existing user activation model. If you're willing to accept that consequence, then the spec modification is simply:

• Add the internal slot to such orientation change events
• Add a bullet point to https://html.spec.whatwg.org/#activation-triggering-input-event making such events activation-triggering.

If you want to create a second thing, which is like user activation but not really, that's much more work. You would need to define something like:

• Each window has a last orientation change timestamp
• A new algorithm similar to https://html.spec.whatwg.org/#activation-notification which updates the last orientation change timestamp whenever the user changes orientation
• A new predicate, e.g. "has recently changed orientation", which compares the Window's last orientation change timestamp with the current time and the transient activation duration. (Or with a separate duration, if we want to allow them to vary independently?)
• Maybe a separate consumption algorithm, similar to https://html.spec.whatwg.org/#consume-user-activation, that resets the last orientation change timestamp for a window and its descendants. (This would be necessary if you want to prevent repeated fullscreening and un-fullscreening and then fullscreening again, in response to a single orientation change.)