Proposal: a DocumentFragment whose nodes do not get removed once inserted

[WebReflection]

edit I wouldn't mind Node.DOCUMENT_PERSISTENT_FRAGMENT_NODE name/kind neither

TL;DR

The document fragment is a great primitive to wrap together numerous nodes and append these directly as batch, however a fragment loses all its children as soon as appended, making it's one-off usage limited in those cases where a list of nodes, at a certain position, is meant.

This proposal would like to explore the possibility of a live document fragment that:

• does not lose its childNodes once appended
• it's still transparent / unaddressable from CSS
• it's also transparent for any other node so that it's still possible, as example, to append many <TD> or <TR> through such fragment, and keep a reference for future updates

Why

Both virtual DOM based libraries, such React, as well as direct DOM based one, such as hyperHTML, or lit-html, have been implementing their own version of a persistent fragment in a way or another.

If there was a primitive to directly reference more than a node, through a fragment with a well known position on the DOM, I am pretty sure all libraries would eventually move to adopt such primitive, so that a tag function could handle both <p>1</p> and <p>1</p><p>2</p> without needing to re-invent a similar wheel every single time, and making portability between libraries and frameworks easier than ever: it's just a DOM node!

Example

// either persistent or live
const pf = document.createLiveFragment();

// append zero, one, or any amount of nodes
pf.appendChild(document.createElement('TD'));
pf.appendChild(document.createElement('TD'));

// make it live
const lastTr = document.querySelector('#data tr:last-child');
lastTr.appendChild(pf);

// references are still there
pf.childNodes.length; // 2

// the node is invisible though
lastTr.lastChild === lastTr; // false
lastTr.lastChild === lastTr.childNodes[1]; // true

There should be no way to interfere with CSS and/or selectors, the fragment is either referenced somewhere else or it won't exist for the DOM.

How

The way hyper/lit-html are doing this is by abusing comment nodes as boundaries of these virtual fragments. The itchy part of these libraries is mostly represented by these virtual fragments, 'cause it's obvious if the primitive proposed here would exists, these libraries would've used it instead (happy to be corrected, but at least I would never create my own virtual fragment if I could use something else).

The way this could be implemented, is by weakly referencing nodes to such fragment only if this is held in memory.

// example implementation of the live fragment
const references = new WeakMap;
class LiveFragment extends DocumentFragment {
  #childNodes = [];
  #appendChild = node => {
    this.#removeChild(node);
    this.#childNodes.push(node);
  };
  #removeChild = node => {
    const i = this.#childNodes.indexOf(node);
    if (-1 < i)
      this.#childNodes.splice(i, 1);
  };
  appendChild(node) {
    this.#appendChild(node);
    return super.appendChild(reference.call(this, node));
  }
  append(...nodes) {
    nodes.forEach(this.#appendChild);
    return super.append(...nodes.map(reference, this));
  }
  removeChild(node) {
    this.#removeChild(node);
    references.delete(node);
    return super.removeChild(node);
  }
  // the value of this LiveFragment when moved around
  valueOf() {
    this.append(...this.#childNodes);
    return this;
  }
}
function reference(node) {
  references.set(node, this);
  return node;
}

// amend on the appendChild standard
const {appendChild} = Node.prototype;
Node.prototype.appendChild = function (node) {
  appendChild.call(this, asFragment(node));
};

const {append} = Element.prototype;
Element.prototype.append = function (...nodes) {
  append.apply(this, nodes.map(asFragment));
};

function asFragment(node) {
  return node instanceof LiveFragment ?
    // it could be just node.valueOf() for everything
    // explicit here for explanation sake
    node.valueOf() :
    node;
}

Possible F.A.Q. Answers

• if a node is manually appended somewhere it's fine. But as soon as the LiveFragment owner will append such fragment again, that node would be moved back (nodes ownership by creator)
• if the LiveFragment has no references, then nothing changes from a fragment
• a live fragment always exposes its childNodes as immutable, as it is for regular fragments
• everything is the same, except that live fragments creators, owner of their live fragment content, can use this primitive instead of polluting the DOM with comments
• it is always possible for DOM engines to know if a node belong to a fragment, as long as this is still referenced somewhere. If unnecessary, due forced re-append on valueOf(), the references part can be ignored

---

Thanks in advance for eventually considering this, happy to answer any possible question.

[WebReflection]

FWIW this is apparently a very demanded feature from the Web Developers community

Previously on a similar proposal: https://discourse.wicg.io/t/proposal-live-fragments/2507

[WebReflection]

As simplified approach, and for demo sake, I'll leave a playground that works out of the box in Chrome Canary (but not yet in Code Pen, however I've informed them about it)
https://codepen.io/WebReflection/pen/moRQRV?editors=0010

Now I'll wait for any outcome 👋

[WebReflection]

@annevk just FYI I think there's a typo in the title: onse => once

also, if I might ask, what does the label "needs implementer interest" mean?

What can I do to move this forward? Should it be me the implementer?

Thanks.

[annevk]

Do https://whatwg.org/working-mode#changes and https://whatwg.org/faq#adding-new-features help?

I suspect we'll need something like this to build templating on top of.

cc @rniwa @justinfagnani @wycats

[developit]

This is interesting. Other names I've seen used to casually refer to something along these lines are "Persistent Fragment" (given that it doesn't empty when appended) and just "Range Fragment".

Example: https://discourse.wicg.io/t/proposal-fragments/2312

[justinfagnani]

Edit: I didn't see that in this version of the idea the fragment doesn't live in the tree. I think that makes it functionally equivalent to NodePart from TemplateInstantiation, or a wrapper around StaticRange, and addresses most of the issue below, which were based on other variations of the idea I'm familiar with.

@annevk I think the Template Instantiation Part interface is essentially this, without being a Node living in the childNodes list. display: contents also covers some similar use cases.

One concern with a new node type is that much existing tree traversal code will not know how to handle it, so a live fragment and its children will likely be skipped. Depending on where such live fragments are intended to be used this may or may not be a problem. There is a lot of code out there that assumes that only Elements can contain other Elements.

Another concern is that right now I believe that ever Node subclass reachable by tree-walking a document is serializable to HTML (including <template> and it's unique innerHTML implementation). The fact that DocumentFragment cannot be placed into a childNodes list preserves this. I'm not sure how much this matters, but this would be the first node type in the tree (but not tree-of-trees) that can't survive a serialize/parse round-trip.

We'd also have to consider how other APIs work. Do live fragments show up on event.path? Can children of a live fragment be slotted into the fragment's parent's ShadowRoot? etc...

I'm not sure if the issues are insurmountable, but I've been working on Template Instantiation with the theory that the least disruption will be caused with a new non-Node interface that lives outside the tree, like Range. Then all existing tree processing code will work as-is.

[rniwa]

Yeah, we've definitely considered this approach before making the template instantiation proposal but fundamentally, all we need is tracking where the inserted contents need to be. I don't think there is any reason to create a new node type and keep it in the DOM if we can avoid it.

[WebReflection]

Nothing is kept in the DOM. It's a fragment that acts like a fragment.
I'm working on a better playground that shows the idea fully polyfilled so please wait for it to be online and evaluable before closing this, thanks.

[WebReflection]

So, I've uploaded the previously mentioned polyfill, which should have 100% code coverage.
https://github.com/WebReflection/document-persistent-fragment

There is a live test page too.

the what

The idea is to have a 1:1 DocumentFragment alter-ego that doesn't lose its nodes.

The fragment is exposed only to the owner, so that there is no way to retrieve it from third parts, unless passed around, and there's nothing live on the DOM, if not, eventually its child nodes.

The proposal exposes to the owner common nodes methods based on its content.

As example, dpf.previousSibling would return the dpf.firstChild.previousSibling if there is a firstChild. If the dpf.isConnected is false, the result is null.

The DPF (in short) has only one extra method, compared to DocumentFragment, which is dpf.remove(). Such method doesn't need much explanation: it removes all nodes from the DOM.

All operations performed through the DPF are reflected live on the document, and while this might be just a stretch goal, it is super easy and nice to simply update an owned reference and see everything changing live.

Nodes ownership

It is possible to grab random nodes and destroy these, or change these, affecting indirectly the content owned by the DPF instance, but it's always been possible to be obtrusive on the DOM and destroy third parts libraries so I think this shouldn't be concern.

However, I could implement the WeakMap that relate each node to a specific DPF instance in a way that it's not possible to move a node owned by a DPF into another DPF, or even perform any action on the DOM through elements that are not the DPF owner.

This, however, would introduce an ownership concept that is too different from what we've used so far, but I believe this proposal is for all libraries that need such persistent fragment, and that owns their own nodes, libraries that are currently somehow already breaking things if 3rd parts obtrusive libraries destroy, or manipulate, DOM nodes in the wild.

As summary

The fact, beside some Safari glitch I'm sure I can solve, this whole proposal can be already polyfilled, and the fact browsers have a way to optimize it and make it blazing fast, should be considered as a plus, 'cause instantly adoptable by the community, so that we can have quick feedbacks of how much this is welcomed or needed out there.

Please don't hesitate to file issues there or ask me more here before discarding this proposal.

Thank You.

[WebReflection]

@rniwa FYI I've filed the bug that makes current polyfill not usable * in WebKit/Safari

edit I've fixed the current polyfill with a workaround after a feature detection, so this can work on Safari/WebKit too 👋

[rniwa]

Nothing is kept in the DOM. It's a fragment that acts like a fragment.

Then what you created is indistinguishable from NodeTemplatePart we proposed. It's just a matter of syntax / naming differences.

[thysultan]

@rniwa NodeTemplatePart seems to be tightly linked to shadow dom and template instantiation; how would you use NodeTemplatePart to replicate the proposed?

[rniwa]

@rniwa NodeTemplatePart seems to be tightly linked to shadow dom and template instantiation

It's nothing to do with Shadow DOM.

how would you use NodeTemplatePart to replicate the proposed?

In the latest iterations of the proposal @justinfagnani at Google and we're working on, NoteTemplatePart is a thing that could be used without any template although we probably need to rename it to something else.

[WebReflection]

@rniwa I've no idea what is this NodeTemplatePart and I cannot find anything online, however, if you would read at least the test you'll see this proposal is literally nothing new, it's fully based on current standard, backward compatible, and polyfillable.

It's a document-fragment at all effects, it's indeed inheriting the same constructor, but it works transparently and only if the owner/creator keeps a reference around.

If this is exactly what this NodeTemplatePart does, can you please point me at it's specification or API?

Thanks.

[WebReflection]

@justinfagnani

Edit: I didn't see that in this version of the idea the fragment doesn't live in the tree.

would you mind amending/canceling that comment since nothing in there is relevant to this proposal, so that people don't get distracted by concerns that are not part of this proposal?

links to the solutions previously discussed would be more than welcome too.

Thanks.

[annevk]

The context you're missing is https://github.com/w3c/webcomponents/blob/gh-pages/proposals/Template-Instantiation.md and some F2F discussion (probably somewhere in minutes linked from issues in that repository) that encouraged making the parts there true primitives instead of tightly coupled with templating.

(Also, please try to consolidate your replies a bit. Each new comment triggers a new notification for some and there's over a hundred people watching this repository. When in doubt, edit an existing reply.)

There's another meeting coming up, and I hope @justinfagnani and @rniwa can make the current iteration a bit more concrete by then, as referencing it in this issue as if it's a thing everyone should be aware of is a lil weird.

[ryansolid]

The context you're missing is https://github.com/w3c/webcomponents/blob/gh-pages/proposals/Template-Instantiation.md and some F2F discussion (probably somewhere in minutes linked from issues in that repository) that encouraged making the parts there true primitives instead of tightly coupled with templating.

Thank you, I'm glad I read the second part of that comment about true primitives. I had read that proposal before and admittedly a little terrified that the DOM spec would include such an opinionated implementation. That spec reads like designing a render framework. I mean I'm sure we could do worse, but I'm encouraged to know that simpler proposals are under consideration.

What I like about this proposal is the transparent use with Node API's appendChild/removeChild since it can be treated interchangeably with actual DOM nodes when say returned from some sort of Template instantiation.

I think ownership is the challenge with a lot of value comes from having a clear single ancestor(whether single node or fragment). It lets JS context be tied to specific parent nodes without making more elements in the DOM. But by their very nature I don't see how you'd avoid nesting. Like a loop over these fragments with a top level conditional in it that also returns a fragment. In the end the DOM structure would be pretty flat but you'd have persistent fragments in persistent fragments. Since they aren't part of the actual rendered tree it becomes harder to understand what falls under each since nested dynamic content in this case could change what is in the top fragment. I would absolutely love to see a solution in this space having spent a lot of time figuring out tricks to accomplish similar things. Everyone writing a render library where they don't virtualize the tree hits this issue sooner or later.

[WebReflection]

you'd have persistent fragments in persistent fragments

which is fine, and since appending a fragment to itself breaks, there's nothing different on the DOM.

Current implementation / proposal allows shared nodes between fragments, which is the same as creating a fragment on the fly and append any node found in the wild: nothing stops your from doing that, everything works, no error thrown.

The current idea is that keeping it simple is the only way to quickly move forward, while any ownership concept would require bigger, non backward compatible, changes.

Libraries and frameworks authors won't worry about that anyway, 'cause they are the one creating nodes too, and they are those virtualizing thee in trees.

Having a mechanism to move N nodes at once, accordingly with any DPF appended live, it's also a feature that would simplify state-machine driven UIs.

Last, but not least, hyperHTML has this primitive since long time and it works already, but it doesn't play super nice with the DOM if thrown there as is, and it requires special treatment when used right away.

This proposal would cover that case can much more.

[Jamesernator]

I'm not sure how much this matters, but this would be the first node type in the tree (but not tree-of-trees) that can't survive a serialize/parse round-trip.

Actually CData sections and processing instruction nodes are simply completely broken in text/html because they don't deserialize (instead they just become text nodes) but do serialize. Not that anyone really cares about them.

[jhpratt]

Is there any continued interest in this proposal? Would a persistent fragment have access to some DOM methods like replaceWith()? Those could be heavily optimized in situations like element reordering.

[ryansolid]

I like this proposal a lot more after sitting with it a bit longer. At first I was thinking this was about library code managing the moving and managing of ranges of elements, but this is more. This helps with giving the users of said libraries the equivalent of React JSX Fragments. Like consider:

const div = html`<div></div>`
const frag = html`<div></div><div></div>`

// further down
const view = html`<div>${ condition ? div : frag }</div>`

I had a user ask why the div always worked, but the second time they attached the fragment why did it not render anything. The answer of course was that div.appendChild(frag) removed the childNodes from the fragment so on the second attach there were no childNodes to append. Now one could argue I could make frag an array (slice the childNodes) but that doesn't handle dynamic top level changes in the fragment.

In a library based on KVO Observables top level dynamic updates that execute independently of top down reconciliation having a standardized DOM API that works the same whether attached or not is hugely helpful in this scenarios.

Beyond that all these libraries have a similar concept. Being just a DOM node works very consistently with the whole

// tagged templates
const el = html`______` //or
// jsx
const el = <______ />

way of approaching rendering which has been gaining steam (looking at the API's on the top end of performance in the JS Frameworks Benchmark). More and more non-virtual DOM libraries are picking this approach and showing it is performant.

[WebReflection]

AFAIK there's not such thing and it would be useful only for SSR purposes where there is no DOM so I am not sure this is WHATWG responsibility but I am sure internally every engine has some sort of toString() internal representation for innerHTML and outerHTML getters purpose ... mine was just mimicking that possible hook into serialization, I don't think I need, or want, to directly control that as I don't have any useful use case for that neither but I wouldn't be against such mechanism, although it has little to do with NodeGroup proposal.

[DeepDoge]

• <li> ... these cannot be appended within an outer element
• <head> related children that could be grouped by feature-detection/UA sniffing (as ugly as that is, it's still very much used out there)
• <td> or <tr> cannot be appended without breaking tables
• <option> cannot be appended
• <source> or other strictly constrained by their parent (and vice-versa) that cannot be everywhere

Btw, I just wanted to say <li>, <option>, <title>, and more are not effected.
<td> <tr> also not effected in DOM. But HTML parsing throws the div(s) out of the <table>. Similar to how HTML parsing throws the nested <form>(s) out, but its totally fine and handled well when appended by JS.

I already use <div is="..." style="display: contents"> wrap reactive ChildNodes in the DOM, in my library, which I'm using in few projects already.
Only down side for me was doing classless CSS gets kinda hard, and external CSS libraries might trip sometimes because the selectors in them don't expect something in between. And that's the only reason I'm interested in this.

That's all I wanted to say.
Just watching where this proposal goes.

[WebReflection]

Btw, I just wanted to say <li>, <option>, <title>, and more are not effected. <td> <tr> also not effected in DOM.

what do you mean?

if I write this content (and we should keep in mind hydration is part of this story) I have all sort of unexpected things on the living DOM:

<!doctype html>
<html lang="en">
  <head>
    <group>
      <meta charset="UTF-8">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
    </group>
    <group><title>shenanigans</title></group>
  </head>
  <body>
    <ul>
      <li>1</li>
      <group><li>2</li></group>
      <li>3</li>
    </ul>
    <select>
      <option>1</option>
      <group><option>2</option></group>
      <option>3</option>
    </select>
    <table>
      <tr>
        <td>1</td>
        <group><td>2</td></group>
        <td>3</td>
      </tr>
    </table>
  </body>
</html>

Result as image:

To sum it up:

• <head> stuff is moved outside
• the table erases entirely the <group> node
• devtools shows errors for both the <ul> and the <select>

but most importantly, somebody complained about the tree being unexpected when having a wrapper node makes everything even more unexpected from tree walking perspective ... and then again, I would be all up for a "transparent node" that works as nodegroup container without breaking/moving/shifting elements while parsing to fix the unexpected but that seems like changing everything the DOM (or its parsing) knows to date so I doubt it's a viable solution unless we want to wait years before having something usable ... wouldn't you agree?

edit the table as I see it in devtools:

<table>
  <tbody>
    <tr>
      <td>1</td>
      <td>2</td>
      <td>3</td>
    </tr>
  </tbody>
</table>

[DeepDoge]

@WebReflection

group {
   display: contents;
}

HTML

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <div style="display: contents">
    <title>Test: Div Wrapped Elements</title>
  </div>
</head>
<body>
  <h1>Testing HTML Elements with &lt;div style="display: contents"&gt;</h1>

  <!-- Option inside select with div -->
  <h2>Option inside Select</h2>
  <select>
    <option>Option 1</option>
    <div style="display: contents">
      <option>Option 2 (inside div)</option>
    </div>
  </select>

  <!-- List items inside div -->
  <h2>List Items inside UL</h2>
  <ul>
    <li>Item 1</li>
    <div style="display: contents">
      <li>Item 2 (inside div)</li>
    </div>
  </ul>

  <!-- Table with div-wrapped tr and td -->
  <h2>Table with Div-Wrapped TR/TD</h2>
  <table border="1">
    <thead>
      <tr><th>Header</th></tr>
    </thead>
    <tbody>
      <div style="display: contents">
        <tr>
          <div style="display: contents">
            <td>Cell 1 (wrapped in divs)</td>
          </div>
        </tr>
      </div>
    </tbody>
  </table>

  <!-- Picture element with real images -->
  <h2>Picture Element with Source</h2>
  <picture>
    <div style="display: contents">
      <source srcset="https://picsum.photos/300/200.webp" type="image/webp">
    </div>
    <img alt="Fallback Image" width="300">
  </picture>

  <!-- Video element with source from Google CDN -->
  <h2>Video Element with Source</h2>
  <video controls width="300">
    <div style="display: contents">
      <source src="https://storage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4" type="video/mp4">
    </div>
    Your browser does not support the video tag.
  </video>

</body>
</html>

But:

<td> <tr> also not effected in DOM. But HTML parsing throws the div(s) out of the <table>. Similar to how HTML parsing throws the nested <form>(s) out, but its totally fine and handled well when appended by JS.

So:

DOM JS for table

const table = document.createElement("table");
table.border = "1";

const thead = table.createTHead();
const headerRow = thead.insertRow();
headerRow.insertCell().outerHTML = "<th>Header</th>";

const tbody = table.createTBody();
const divTR = document.createElement("div");
divTR.style.display = "contents";

const tr = document.createElement("tr");
const divTD = document.createElement("div");
divTD.style.display = "contents";

const td = document.createElement("td");
td.textContent = "Cell 1 (inside divs)";

divTD.append(td);
tr.append(divTD);
divTR.append(tr);
tbody.append(divTR);

document.body.append(table);

[WebReflection]

@DeepDoge that doesn't work if served as such from the server ... that's my point, it breaks the hydration story, it's not a solution to me, happy to learn it works for your case but personally I wouldn't abuse the DOM that way and all 3rd party library expectations around a clean tree are gone. The idea here is to never need hacks and find a standard way forward that works for SSR too.

edit ... "what doesn't work" ?

• title moved into body
• the video does not understand the source
• all other things previously mentioned

[DeepDoge]

@WebReflection I was just saying, it wasn't a problem with all the things you listed.

Btw, I just wanted to say <li>, <option>, <title>, and more are not effected.
<td> <tr> also not effected in DOM. But HTML parsing throws the div(s) out of the <table>. Similar to how HTML parsing throws the nested <form>(s) out, but its totally fine and handled well when appended by JS.

Also, I just seen it more of small change by introducing a new primitive like new attribute that makes the element transparent, which also might be useful for other use cases other than "fragments" as well. Instead of engineering something new.

I never said it should be that in the whole page.

From the start I was just saying that I don't like that childOfFragment.parentNode is giving the parent of the fragment that's all.

[WebReflection]

I don't like that childOfFragment.parentNode is giving the parent of the fragment

that's what happens with fragments though ... its content parentNode is the fragment until the fragment gets appended, then it's the place where it's been appended ... to me that's better than needing to check two levels up to know if the child is live or not and, from a fragment that is literally non-existent on the DOM, everything works wonderfully together.

Then again, I am waiting for @rniwa or whoever else to work on, or present, a PoC that satisfies the requirements and I would not complain if that's an attribute, a <nodegroup> new kind of thing, or a fragment as proposed and as it will be, inevitably, polyfilled.

[DeepDoge]

from a fragment that is literally non-existent on the DOM, everything works wonderfully together

That's probably the difference, I'm more interested it's being in the DOM rather than being something abstract. More verbose. When I see it on the DOM I wanted to say, OK this is a wrapper Node, and I can remove or add stuff in it.

Like when you are listening the DOM with MutationObserver and DocumentFragment flashes there for a second. I was like:
"If its acting like that, replacing itself with it's children, why not just keep it there."

to me that's better than needing to check two levels up to know if the child is live or not

Doesn't Node.prototype.isConnected do that?

Anyway, either way, as long as we have something useful, I don't really care what it's or how it works.

[WebReflection]

Doesn't Node.prototype.isConnected do that?

you can have NodeGroup in NodeGroup and the top level one is not necessarily connected ... the parent in that scenario (nested non live node groups) becomes even more cumbersome ... right now the parent is either the current one where the group was inserted, or null ...

OK this is a wrapper Node, and I can remove or add stuff in it.

the current prototype/idea is that anonymous groups don't share their reference so there is an ownership concept same way if you create a fragment and you don't leak it nobody can interfere with its logic (assuming is a class extends that does more and want to control or change or track or do whatever with its children when it's been created offline).

If it becomes something everyone can mess around with this simple contract won't exist anymore and personally I like that simple contract but, like you said:

Anyway, either way, as long as we have something useful, I don't really care what it's or how it works.

[FrameMuse]

WebReflection/group-nodes#12 TL;DR

Recently, I brought this idea to address concerns about multi-parenting and unclear traversal strategies.
WebReflection/group-nodes#12

But as @WebReflection mentioned the idea is not about having a transparent container, but having a reference point in a document to manage nodes without risk of being altered by other scripts unless it is explicitly shared.

Thus @WebReflection suggests that concealing the children from being discovered from outside¹ by Tree Walking (i.e. TreeWalker, parentNode, firstChild, childNodes, ...) makes sense due to ParentNode nature² since otherwise DOM would need a behaviour no one ever saw before, which brings unknown level of complexity.

¹ Though NodeGroup can be tree walked by the owner - from inside
² FYI, @WebReflection suggests to extend from DocumentFragment, which implements ParentNode

@WebReflection also suggests to conceal a reference point (i.e. NodeGroup instance) itself from being tree walked as well, which brings alive the statement of encapsulated nodes management - this is the main idea of current proposal.

---

@WebReflection Please tell me if I got something wrong 🙏

[robbiespeed]

I was originally pretty sceptical of the idea of a wrapper element that appears as a true parent in the DOM tree <fragment>contents</fragment>, there are difficulties mentioned around this approach about how some elements have very restrictive constraints on child elements (ul, table, head, etc). This fragment element could be speced to allow placement anywhere, which is hard to polyfill. These difficulties are very similar to ones that would be present with a html includes mechanism. I think there is a path that allows includes to be built off the foundation of a fragment element, and the end result could be very powerful.

Ex:

<fragment src="/content.html">
  <span class="spinner"></span>
  Loading...
</fragment>

Fragments children would be replaced once loaded, and if src isn't provided then it is a simple way to group nodes without needing to special case CSS selectors with all the benefits discussed in this proposal.

One added benefit of this approach is that event listeners can be attached to the fragment which would allow event delegation for it's children.

FragmentElement should inherit from Element but not HTMLElement, HTML specific attributes like style and hidden don't apply to a fragment since they should be invisible to CSS and styling, and it would make it less awkward to use inside embedded svg and mathml.

A wrapping element like this is also impossible to break, where as marker based fragments can be broken when a marker is moved away from it's partner (ex: extracting a range of content that starts in the middle of a fragment and ends outside).

It's also possible that similar to providing a solution for includes in the future, it could help solve out of order streaming without need of JS.

[WebReflection]

@robbiespeed about this:

Fragments children would be replaced once loaded

does this happens only at the rendering level or would that <fragment> disappear from the tree? in the latter case, hydration is broken, in the former one, all good if that works in practice and all use cases are covered 👍

at the polyfill level, a good orchestration of MutationObserver to track fragments parents (target) around should do ... but I'd like to be sure everyone is OK with this way of shipping fragments (@rniwa and/or others) so that it won't be a waste of (personal, unfortunately almost non-existent) time.

[robbiespeed]

@WebReflection that's a good question, I expect that the fragment would never disappear from the tree (unless manually removed).

A full example for those not following this thread closely

ul > li {
  color: purple;
  :first-child {
    color: green;
  }
}

<ul>
  <fragment src="/items.html">
    <li>Loading...</li>
  </fragment>
  <li>Last</li>
</ul>

When items.hmtl is loading, you'd see a list with a green "Loading" item, and a purple "Last" item.

Once items.html is loaded the tree would end up something like this:

<ul>
  <fragment src="/items.html">
    <li>First</li>
    <li>Second</li>
  </fragment>
  <li>Last</li>
</ul>

Green "First" item, purple "Second" and "Last" items.

From the perspective of CSS fragment doesn't exist, but it's children do (appearing as children of the fragments parent). From the perspective of the DOM fragment does exist like other elements.

[FrameMuse]

I was thinking about fallbacks for fragment boundaries, I think this way it should solve this problem.
I don't think actually using comments as boundaries is the way for serialization.

<template shadowRootMode="closed" as="group" name="myGroupName">
	<p>Text</p>
	<span>123</span>
</template>

This way we follow the rule proposed by @WebReflection (#736 (comment)), which is hiding the internal structure, while making a way for serialization since template is already there way some time, so it should be invisible for older versions of DOM. For newer versions that implement groups, it would actually create a group node.

Alternative

<template shadowRootMode="closed" group="myGroupName">
	<p>Text</p>
	<span>123</span>
</template>

But my choice the one above as I also consider an element that would be inert (as a separate proposal), while either assignable with an element or attachable with ShadowRoot. And the first variant is more readable.

[WebReflection]

@robbiespeed if that is:

• non-existent for CSS
• reachable via tree walkers
• non-breaking parents with child expectations (ul, table/tr/td, select, head, etc)
• works nested (<fragment> in <fragment>)

I would be sold without second thoughts ... the reason I never used that approach is that I've thought it was not possible/desired to create such element and I think the src is even a stretch 'cause everything else seems to be working the way I wanted/needed.

This should also make Justin happy as that would reflect his idea too, if I am not mistaken.

@FrameMuse let's please not go down that SD road, what @robbiespeed is proposing is exactly what we all wanted, including yourself IIRC ... it's true that <fragment> will break every browser that doesn't understand it but that was also true when template was introduced so ... let's move forward for a greater future, imho.

[FrameMuse]

@WebReflection Then we need very flexible and minimal primitive than what we have now or it's going to be ignored very likely to my opinion.

[robbiespeed]

I think the src is even a stretch 'cause everything else seems to be working the way I wanted/needed

To be clear I don't think src should be a requirement to move forward, <fragment> is compelling enough on its own.

Html includes would be great, as would persistent fragments. For includes to work though I think it requires something like <fragment> anyway, so it makes sense to spec that first with src in mind for the future.

src would be very easy to polyfill once <fragment> is available and allow some exploration into the lifecycle of when things load, what happens if they don't, events that may fire, if templates could be used for same document out of order streaming, etc.

[rniwa]

I don't think there is much appetite for introducing a new element which changes the behavior of HTML parser. We had to fight against dozens upon dozens of security bugs and crashes when we introduced template element. I don't think we want to repeat saga, not to mention that changing HTML parser resulted in a bunch of XSS issues in websites because there are non-browser implementations of HTML parser (e.g. used for server side rendering, etc...).

[robbiespeed]

@rniwa can you be more specific about what kinds of changes to parser behaviour is problematic? Template is a wildly different scenario, since it's children are placed into a DocumentFragment rather than as children of the element in the DOM tree (I see how that could've possibly lead to XSS with nonsupporting implementations). What's being proposed is something that from a parsing/DOM tree perspective behaves nearly identically to a regular element, with the difference being it may be placed in any parent element so long as it's children (or grand-children in a nested fragment case) are valid content of the fragments parent. The major change is with how the element behaves with CSS.

From a backwards compatibility scenario it would end up a HTMLUnknownElement without the special transparent CSS behaviour or placement rules (would be moved outside a table for example). That seems pretty low risk.

[rniwa]

@rniwa can you be more specific about what kinds of changes to parser behaviour is problematic?

Any kind of HTML parser change is problematic. We don't want to make any changes to the HTML parser at all at this point unless there is an extremely compelling reason to do so. Given there are alternatives to support this feature without affecting HTML parser changes, I don't see how we'd pick the alternatives that do.

[robbiespeed]

@rniwa Having such limits on parser changes sounds like a major problem for advancing HTML, are there any proposals to relax/generalise the parsing rules such that future features are not such a problem to add?

As far as I can tell the XML parser used for XHTML would have no issues, but I did take a look at the HTML parser spec and it is significantly complex, so I see why changes are avoided. If there's no path forward for improving the HTML parser situation maybe it makes sense for the web to push for an XHTML future once again.

I do still think a fragment element would be worth it considering that HTML includes would need similar spec changes, and there's something to be gained from aligning the features together.

The comment marker based approach for grouping is also not without it's downsides, it's possible for groups to break due to marker pairs being separated in the tree. That's going to impact users, and will also need to be addressed in the DOM spec.

[WebReflection]

If there's no path forward for improving the HTML parser situation maybe it makes sense for the web to push for an XHTML future once again.

I'd welcome that but right now HTML5 and XHTML are not fully interoperable (last time I've checked) so that would be a new world of potential incompatibilities between browsers, SSR and whatnot.

it's possible for groups to break due to marker pairs being separated in the tree

not in my prototype ... those markers fail fast in that scenario so if you break the contract the fragment is broken, as easy as that. At the parsing level, if those markers are messed up no group exists ... still KISS and working with all primitives we have.

That's going to impact users

not really ... it will impact bad code and libraries but those already negatively affect users ... I mean, some library still use innerHTML instead of preserving nodes identities, this is not a Web platform concern as long as specs are clear.

will also need to be addressed in the DOM spec

true that, but the contract is very simple: https://github.com/WebReflection/group-nodes/blob/main/src/lib/utils.js#L74-L88

I would still welcome a <fragment> element but that's why I've asked for @rniwa consensus (or others) because I knew before proposing the prototype that would've not been welcomed.

[FrameMuse]

I welcome any solution actually at least to try it out under experiments, but for now I see this proposal as non-serializable existing only as in-memory structure, this would still fit to my use-case, later on we can extend it to be serializable if possible.

So far, we have several cool features beyond just Persistent Fragment and more-less basic primitive (I still think we should find even more basic one), I think we should try to discuss it a bit more, demonstrate very basic use-cases, how this naturally fits to @rniwa DOMParts proposal and then frame it and probably create a PR. What do you think guys, is it the time?

[DeepDoge]

@robbiespeed
What you did with [src] can already be done using custom elements, which also allow for more flexible logic around how the content gets filled. The issue is, that it wouldn't be transparent. You might try applying :host { display: contents } via an adopted CSSStyleSheet, but as @WebReflection pointed out, that comes with its own trade-offs.

That’s why having <fragment> as a standalone, purpose-built element makes more sense—similar to how elements like <canvas>, <iframe>, <video>, or <img> exist for specific use cases. You could still build on top of it using custom elements to add your own logic—if it were an HTMLElement. But since you mentioned it wouldn't be an HTMLElement, but rather another Element type, it couldn’t be used as a custom element.

However, if the custom attributes proposal lands, that might let us bind behavior to elements regardless of whether they're HTMLElement or not—assuming they work on any Element type.

Now, why does lifecycle even matter in this context? One common reason people want a persistent fragment is to update or move nodes around as a group. If updates are involved, you'll probably need to react to some kind of signal/event—and that means you need clean up that listener when the fragment is removed from the DOM. That’s why lifecycle callbacks like connectedCallback and disconnectedCallback are crucial. If the fragment never needed to change, a one-time DocumentFragment or even an array of ChildNode references would have been enough.

Custom elements (and maybe custom attributes in the future) already provide a standardized way to handle lifecycles, so introducing a completely new system just for new fragment/group Node type might be redundant. Or maybe it's just fine.

To be clear, I’m not attached to a specific solution—I just want us to get something. My concern is that whatever we end up with might not be flexible or lifecycle-aware enough, and that would make it barely better than managing a raw array of nodes—still fragile when DOM changes since it would relay on frameworks' hacky abstract ways of handling lifecycles.

As for concerns about breaking existing tree walkers: I don’t find that argument very compelling. If a developer chooses to use a new element or node type, it's on them to handle it properly.

[FrameMuse]

As for concerns about breaking existing tree walkers: I don’t find that argument very compelling. If a developer chooses to use a new element or node type, it's on them to handle it properly.

Indeed, but what about scripts/libraries that don't know you're using a new element?

---

Now, why does lifecycle even matter in this context? One common reason people want a persistent fragment is to update or move nodes around as a group. If updates are involved, you'll probably need to react to some kind of signal/event—and that means you need clean up that listener when the fragment is removed from the DOM. That’s why lifecycle callbacks like connectedCallback and disconnectedCallback are crucial. If the fragment never needed to change, a one-time DocumentFragment or even an array of ChildNode references would have been enough.

I agree, I like and want connection events to be observable, but the concern here is use-cases like
"you as a developer do your code and you already specified where and how the nodes are attached/removed, so why you need events if you already "know" that?" or "we already have libraries..."

I don't have too much to say against it, please help fight this ^_^

[WebReflection]

I, personally, don't need/care about connected/disconnected events and I feel like it's entirely out of scope, even if proposed as <fragment>, 'cause it shouldn't be different from any other element that doesn't natively have those hooks backed in.

All these discussions will only delay more and more this proposal which is kinda sad ... we all said "whatever it is would be better than nothing" but then we all have opinions that apparently can only cause friction for this proposal to move forward.

I think @rniwa nailed it with the fact there is already an easy to implement solution so that we shouldn't keep asking stuff that is not strictly part of the proposal or a requirement when it comes to have anyway a node that can be diffed/moved/removed purposes?

The result is likely that nothing will ever land due bikeshedding/features-creep scenarios not meant or needed to start with, imho, so let's please try to stop asking for new things and focus on what's actually best for the current proposal to move forward?

It's a primitive, it should do one thing, we can iterate on more things after such primitive landed, right?

Thanks in advance for your understanding 🙏

[DeepDoge]

we can iterate on more things after such primitive landed, right?

True, totally get it. I will just be waiting for something to land.

[FrameMuse]

@WebReflection @rniwa Ok, I agree, what should be our next steps?