Content Selection

[matthew-carroll]

Content selection is a major category of use-cases for rich text editors. This document describes some selection behaviors, how super_editor thinks about selection, and how super_editor implements selection.

Understanding selection requirements

Use-Cases

Here are some common selection use-cases:

• Click to place a caret
• Double-click to select a word
• Triple-click to select a paragraph
• Press left/right arrow to move the caret one character to the left/right
• Press Shift+[left/right] arrow to expand the selection one character to the left/right
• Press up/down arrow to move the caret one line up/down
• Press Shift+[up/down] arrow to expand the selection one line up/down
• Press-and-drag to select all content within a selection rectangle
• Select some text, press a character key, replace the selected text with the new character

Required Inputs

Selection use-cases involve:

• Keyboard input
• Mouse/Touch input
• Layout flow and coordinates
• Document changes, e.g., the Input Method Engine applies an autocorrection within selected text

Copy/Paste

Copy/paste is a closely related category of use-cases. Often, selected content is copied to the clipboard, or clipboard content is pasted at the caret, or in place of an expanded selection. However, copy/paste are not intrinsic to selection. Selection should make copy/paste possible, but should not prescribe any particular approach to implementing copy/paste.

How super_editor thinks about selection

super_editor generalizes Flutter's concepts for text selection, therefore, one must understand Flutter's approach to text selection before understanding super_editor's approach to content selection.

Text Selection in Flutter

A text selection in Flutter is comprised of a "base position", "extent position", and an "affinity", which can be thought of as direction, e.g., "downstream" or "upstream".

Here are some definitions for text selection terms:

• Text Position: An offset within a blob of text, including an affinity.
• Base Position: The text position where a selection begins.
• Extent Position: The text position where a selection ends
• Start Position: The text offset of the selection boundary that appears first when reading the text. This is either the base position or the extent position
• End Position: The text offset of the selection boundary that appears last when reading the text. This is either the base position or the extent position
• Collapsed Selection: A selection where the base and extent positions are the same. A text input caret is an example of a collapsed selection.
• Expanded Selection: A selection where the base and extent positions are different, thus selection some portion of content.
• Affinity: The direction of a selection, e.g., "upstream" or "downstream". A downstream selection moves in the natural direction of the text. In English, a right-to-left language, a single-line downstream selection would start on the left and end on the right. An upstream selection is the opposite of a downstream selection. In English, a single-line upstream selection would start on the right and end on the left. Desktop use-cases are expected to always have an affinity, but due to the nature of drag handles on mobile, there may be situations where an expanded selection has no affinity, i.e., no particular selection direction.

Generalizing Text Selection to General Selection

To generalize text selection concepts to arbitrary content within an arbitrary layout, it's important to recognize the aspects of selection that will go from constants to variables.

Text content to arbitrary content: Text selection only needs to worry about text content. However, arbitrary content may include text within structures, like lists, or non-text content, like images.

Text layout to arbitrary layout: Text selection operates on a blob of text. Text selection knows how to map cartesian coordinates to characters, and map characters to cartesian coordinates. But this layout is very specific. This particular selection mapping would need adjustments for list layouts, and it would be entirely inappropriate for images.

Single-level layout to compositional layout: Text blobs exist at a single-level of layout. Other layouts are not embedded within a single blob of text. However, documents and layouts are deeply compositional. Layouts within layouts within layouts. Text selection doesn't need to worry about up/down communication in a layout hierarchy, but general selection does.

Document Selection Concepts

Given the aforementioned differences between text selection and general selection, it begs the question, which aspects of text selection can be extended to support general selection? What does it mean to select content in general?

General selection has the following characteristics:

• An unambiguous beginning
• An unambiguous end
• A direction from beginning to end

These are the only details that any given selection has in common with any other.

Generalizing text selection to document selection, we end up with the following concepts:

• Document: A list of "nodes", each of which have a unique ID
• Document Position: A visually and logically addressable offset within a document
• Node Position: A visually and logically addressable offset within a node
• Base Document Position: A document position where a selection begins
• Extent Document Position: A document position where a selection ends
• Start Position: The document position at the end of a selection that appears first when reading the document
• End Position: The document position at the end of a selection that appears last when reading the document
• Affinity: The direction of a selection, either "upstream" or "downstream"

Implementing Document Selection

Logical Document Selection

The backbone of document selection is the concept of a document position. Unlike a text position, a document position can't define itself with an integer offset, because an integer offset doesn't mean anything for non-text content. Instead, a document position is comprised of a "node ID" and a "node position".

class DocumentPosition {
  String get nodeId;

  NodePosition get nodePosition;
}

abstract class NodePosition {
  // marker interface
}

A nodeId uniquely identifies a content layout at the top level of a document, e.g., header, paragraph, image, horizontal rule, ordered list.

A nodePosition identifies a specific offset within a node. The NodePosition type is a marker interface because different types of content require different representations for offsets within that content. Text can use an integer offset, an image might use a boolean, and a table might use a row/col data structure. This choice is left to the individual content nodes, but whatever the representation, the NodePosition is stored within a DocumentPosition.

Given two document positions, we have a document selection.

class DocumentSelection {
  DocumentPosition get base;

  DocumentPosition get extent;
}

A DocumentSelection represents a selection of arbitrary content, but it's helpful to provide some tools within the concept of a DocumentNode aimed at assembling and altering a document selection.

abstract class DocumentNode {
  String get id;

  NodePosition get beginningPosition;

  NodePosition get endPosition;

  NodePosition selectUpstreamPosition(
    NodePosition position1,
    NodePosition position2,
  );

  NodePosition selectDownstreamPosition(
    NodePosition position1,
    NodePosition position2,
  );
}

When working with document selections, it may be necessary to obtain the very first position in a node or the very last position in a node. Similarly, given two positions within a node, a document may need to determine which of those positions is upstream or downstream from the other. DocumentNode provides these faculties.

DocumentSelection and DocumentNode solve the problem of a logical representation of arbitrary content selection within a document.

Selecting Content

Given the logical representation of content selection, super_editor needs a mechanism to begin a selection, e.g., clicking in the middle of some text, or clicking on an image.

Document selection interaction is handled at the document level. Gestures are captured anywhere within the document. Some gesture selection changes are made by the overall document layout, while other decisions are made by individual visual components within the document.

For example, this is the implementation of a single tap/click down:

void _onTapDown(TapDownDetails details) {
    _clearSelection();
    _selectionType = SelectionType.position;

    final docOffset = _getDocOffset(details.localPosition);
    final docPosition = _layout.getDocumentPositionAtOffset(docOffset);

    if (docPosition != null) {
      // Place the document selection at the location where the
      // user tapped.
      widget.editContext.composer.selection = DocumentSelection.collapsed(
        position: position,
      );
    }

    _focusNode.requestFocus();
}

In this example there are a couple monolithic decisions, along with a single compositional decision. First, the top-level editor UI clears any existing selection when the user taps down. Then, later, the top-level editor UI configures a collapsed selection at whatever the logical position where the user tapped. The one compositional decision is where the docPosition is obtained. Wherever the user tapped, there is some kind of UI component. That UI component is asked to determine an appropriate DocumentPosition for that (x,y) offset.

In addition to gesture handling, this same widget also handles keyboard input. Therefore, all selection modifications begin from this widget.

Ideally, selection behaviors would operate in a purely compositional manner, similar to layout and painting. However, given that selection behaviors often span independent visual components, it's unclear if there is any generally applicable, compositional API that would allow for all possible selection behaviors. If such an approach is possible, super_editor should adopt that approach, instead.

With a logical representation for document selection, and a gesture/keyboard mechanism to start and alter a selection, the only remaining requirement is painting.

Mapping Cartesian Coordinates to Document Positions

When the user starts a selection at a given (x,y) coordinate, or moves the boundary of a selection to a given (x,y) coordinate, the editor must determine the document position at that (x,y). Similarly, there are times when a given document position must be mapped to a visual (x,y) coordinate.

This mapping is handled in a compositional manner.

A given (x,y) coordinate is mapped to a specific document UI component widget by way of the render tree. From there, that document UI component maps the (x,y) coordinate in local space to a DocumentPosition.

A given DocumentPosition can also be mapped to an (x,y) coordinate. First, the document layout is asked to find the UI component that corresponds to the given node ID. Once the UI component is located, the UI component maps the DocumentPosition to an (x,y) coordinate.

Lastly, because the UI component is a widget, it may contain any number of other widgets that participate in the mapping process, e.g., a table that defers to a cell that defers to text.

Painting a Document Selection

Unlike starting and changing a selection, selection painting is handled on a per-component basis.

super_editor has an overall document layout widget, which then builds other widgets for each piece of content within the document, e.g., headers, paragraphs, list items, images. When the overall document layout builds these content UI components, the document layout calculates which portion of that piece of content is selected, and then instructs the UI component to "select" that portion of itself.

In the case of a paragraph, a TextComponent widget is constructed with a given TextSelection. The TextComponent paints selection rectangles around the content within that TextSelection.

In the case of an image, an ImageComponent is told whether the image is selected or not. If the image is selected, then the ImageComponent paints a border around the image.