Merge pull request #101 from n8willis/flag-feature-masks

Indic: Flag glyphs for potential feature-applications after initial reorder. This adds an explicit "flag for features" step 2.10 to stage 2.
n8willis · Jan 26, 2021 · 2ec2298 · 2ec2298
2 parents bd0325d + fd6db1d
commit 2ec2298
Show file tree

Hide file tree

Showing 11 changed files with 735 additions and 174 deletions.
diff --git a/opentype-shaping-bengali.md b/opentype-shaping-bengali.md
@@ -728,7 +728,8 @@ positioned before or after any post-base consonants, respectively.
 For shaping-engine implementers, the names used for the ordering
 categories matter only in that they are unambiguous. 
 
-For a definition of the "base" consonant, refer to step 2.1, which follows.
+For a definition of the "base" consonant, refer to step 2.1, which
+follows.
 
 #### 2.1: Base consonant ####
 
@@ -1054,12 +1055,56 @@ so that glyphs of the same ordering category remain in the same
 relative position with respect to each other.
 
 
+#### 2.10: Flag sequences for possible feature applications ####
+
+With the inital reordering complete, those glyphs in the syllable that
+may have GSUB or GPOS features applied in stages 3, 5, and 6 should be
+flagged for each potential feature. 
+
+This flagging is preliminary; the set of potential features varies
+between different scripts and which features are supported varies
+between fonts. It is also possible that the application of
+one feature on a glyph sequence will perform a substitution that makes
+a later feature no longer applicable to the updated sequence.
+
+Consequently, the flagging must be completed before shaping proceeds
+to the stages during which features are applied.
+
+Some shaping features, such as `locl`, can potentially apply to any
+glyphs. Therefore it is not necessary to maintain a separate flag for
+these features in the bitmask (or other data structure) used to track
+the flags -- although shaping engines may do so if desired.
+
+The sequences to flag are summarized in the list below; a full
+description of each feature's function and interpretation is provided
+in GSUB and GPOS application stages that follow.
+
+  - `nukt` should match "_Consonant_,Nukta" sequences
+  - `akhn` should match "Ka,Halant,Ssa" and "Ja,Halant,Nya"
+  - `rphf` should match initial "Ra,Halant" sequences but _not_ match
+            initial "Ra,Halant,ZWJ" sequences
+  - `blwf` should match "Halant,Ra", "Halant,Ha", and "Halant,Va" in
+            post-base positions and "Ra,Halant", "Ha,Halant", and
+            "Va,Halant" in non-initial pre-base positions
+  - `half` should match "_Consonant_,Halant" in pre-base position but
+           _not_ match "Ra,Halant" sequences flagged for `rphf` and
+           _not_ match "_Consonant_,Halant,ZWNJ,_Consonant_" sequences
+  - `pstf` should match initial "Halant,Ya" in post-base position
+  - `vatu` should match "_Consonant_,Halant,Ra",
+           "_Consonant_,Halant,Ha", and "_Consonant_,Halant,Va"
+  - `cjct` should match "_Consonant_,Halant,_Consonant_" but _not_
+            match "_Consonant_,Halant,ZWJ,_Consonant_" or
+            "_Consonant_,Halant,ZWNJ,_Consonant_"
+
+
+
+
 ### 3: Applying the basic substitution features from GSUB ###
 
 The basic-substitution stage applies mandatory substitution features
 using the rules in the font's GSUB table. In preparation for this
-stage, glyph sequences should be tagged for possible application 
-of GSUB features.
+stage, glyph sequences should be flagged for possible application 
+of GSUB features in stage 2, step 10.
 
 The order in which these substitutions must be performed is fixed for
 all Indic scripts:
@@ -1139,7 +1184,7 @@ feature must be applied before all other many-to-one substitutions.
 The `rphf` feature replaces initial "Ra,Halant" sequences with the
 "Reph" glyph.
 
-  - An initial "Ra,Halant,ZWJ" sequence, however, must not be tagged for
+  - An initial "Ra,Halant,ZWJ" sequence, however, must not be flagged for
     the `rphf` substitution.
 
 
@@ -1172,7 +1217,7 @@ forms:
 Because Bengali incorporates the `BLWF_MODE_PRE_AND_POST` shaping
 characteristic, any pre-base consonants and any post-base consonants
 may potentially match a `blwf` substitution; therefore, both cases must
-be tagged for comparison. Note that this is not necessarily the case in other
+be flagged for comparison. Note that this is not necessarily the case in other
 Indic scripts that use a different `BLWF_MODE_` shaping
 characteristic. 
 
@@ -1192,16 +1237,16 @@ base consonant or syllable base with "half forms" of the consonant
 glyphs. There are three exceptions to the default behavior, for which
 the shaping engine must test:
 
-  - Initial "Ra,Halant" sequences, which should have been tagged for
-    the `rphf` feature earlier, must not be tagged for potential
+  - Initial "Ra,Halant" sequences, which should have been flagged for
+    the `rphf` feature earlier, must not be flagged for potential
     `half` substitutions.
 
   - A sequence matching "_Consonant_,Halant,ZWJ,_Consonant_" must be
-    tagged for potential `half` substitutions, even though the presence of the
+    flagged for potential `half` substitutions, even though the presence of the
     zero-width joiner suppresses the `cjct` feature in a later step.
 
   - A sequence matching "_Consonant_,Halant,ZWNJ,_Consonant_" must not be
-    tagged for potential `half` substitutions.
+    flagged for potential `half` substitutions.
 
 
 ![Half-form formation](/images/bengali/bengali-half-ka.png)
@@ -1231,7 +1276,7 @@ The `cjct` feature replaces sequences of adjacent consonants with
 conjunct ligatures. These sequences must match "_Consonant_,Halant,_Consonant_".
 
 A sequence matching "_Consonant_,Halant,ZWJ,_Consonant_" or
-"_Consonant_,Halant,ZWNJ,_Consonant_" must not be tagged to form a conjunct.
+"_Consonant_,Halant,ZWNJ,_Consonant_" must not be flagged to form a conjunct.
 
 The font's GSUB rules might be implemented so that `cjct`
 substitutions apply to half-form consonants; therefore, this feature
@@ -1381,7 +1426,7 @@ to maintain compatibility with the other Indic scripts.
 #### 4.5: Initial matras ####
 
 Any left-side dependent vowels (matras) that are at the start of a
-word must be tagged for potential substitution by the `init` feature
+word must be flagged for potential substitution by the `init` feature
 of GSUB.
 
 > Note: Although the specification defines the `init` feature as being
@@ -1407,9 +1452,13 @@ of GSUB.
 ### 5: Applying all remaining substitution features from GSUB ###
 
 In this stage, the remaining substitution features from the GSUB table
-are applied. The order in which these features are applied is not
-canonical; they should be applied in the order in which they appear in
-the GSUB table in the font. 
+are applied. In preparation for this stage, glyph sequences should be
+flagged for possible application of GSUB features in stage 2,
+step 10.
+
+The order in which these features are applied is not canonical; they
+should be applied in the order in which they appear in the GSUB table
+in the font.
 
 	init
 	pres
@@ -1472,9 +1521,11 @@ typographically problematic.
 ### 6: Applying remaining positioning features from GPOS ###
 
 In this stage, mark positioning, kerning, and other GPOS features are
-applied. As with the preceding stage, the order in which these
-features are applied is not canonical; they should be applied in the
-order in which they appear in the GPOS table in the font.
+applied.
+
+As with the preceding stage, the order in which these features are
+applied is not canonical; they should be applied in the order in which
+they appear in the GPOS table in the font.
 
         dist
         abvm

diff --git a/opentype-shaping-devanagari.md b/opentype-shaping-devanagari.md
@@ -692,7 +692,8 @@ positioned before or after any post-base consonants, respectively.
 For shaping-engine implementers, the names used for the ordering
 categories matter only in that they are unambiguous. 
 
-For a definition of the "base" consonant, refer to step 2.1, which follows.
+For a definition of the "base" consonant, refer to step 2.1, which
+follows.
 
 #### 2.1: Base consonant ####
 
@@ -974,12 +975,56 @@ so that glyphs of the same ordering category remain in the same
 relative position with respect to each other.
 
 
+#### 2.10: Flag sequences for possible feature applications ####
+
+With the inital reordering complete, those glyphs in the syllable that
+may have GSUB or GPOS features applied in stages 3, 5, and 6 should be
+flagged for each potential feature. 
+
+This flagging is preliminary; the set of potential features varies
+between different scripts and which features are supported varies
+between fonts. It is also possible that the application of
+one feature on a glyph sequence will perform a substitution that makes
+a later feature no longer applicable to the updated sequence.
+
+Consequently, the flagging must be completed before shaping proceeds
+to the stages during which features are applied.
+
+Some shaping features, such as `locl`, can potentially apply to any
+glyphs. Therefore it is not necessary to maintain a separate flag for
+these features in the bitmask (or other data structure) used to track
+the flags -- although shaping engines may do so if desired.
+
+The sequences to flag are summarized in the list below; a full
+description of each feature's function and interpretation is provided
+in GSUB and GPOS application stages that follow.
+
+  - `nukt` should match "_Consonant_,Nukta" sequences
+  - `akhn` should match "Ka,Halant,Ssa" and "Ja,Halant,Nya"
+  - `rphf` should match initial "Ra,Halant" sequences but _not_ match
+            initial "Ra,Halant,ZWJ" sequences
+  - `blwf` should match "Halant,Ra", "Halant,Ha", and "Halant,Va" in
+            post-base positions and "Ra,Halant", "Ha,Halant", and
+            "Va,Halant" in non-initial pre-base positions
+  - `half` should match "_Consonant_,Halant" in pre-base position but
+           _not_ match "Ra,Halant" sequences flagged for `rphf` and
+           _not_ match "_Consonant_,Halant,ZWNJ,_Consonant_" sequences
+  - `pstf` should match initial "Halant,Ya" in post-base position
+  - `vatu` should match "_Consonant_,Halant,Ra",
+           "_Consonant_,Halant,Ha", and "_Consonant_,Halant,Va"
+  - `cjct` should match "_Consonant_,Halant,_Consonant_" but _not_
+            match "_Consonant_,Halant,ZWJ,_Consonant_" or
+            "_Consonant_,Halant,ZWNJ,_Consonant_"
+
+
+
+
 ### 3: Applying the basic substitution features from GSUB ###
 
 The basic-substitution stage applies mandatory substitution features
 using the rules in the font's GSUB table. In preparation for this
-stage, glyph sequences should be tagged for possible application 
-of GSUB features.
+stage, glyph sequences should be flagged for possible application 
+of GSUB features in stage 2, step 10.
 
 The order in which these substitutions must be performed is fixed for
 all Indic scripts:
@@ -1042,7 +1087,7 @@ feature must be applied before all other many-to-one substitutions.
 The `rphf` feature replaces initial "Ra,Halant" sequences with the
 "Reph" glyph.
 
-  - An initial "Ra,Halant,ZWJ" sequence, however, must not be tagged for
+  - An initial "Ra,Halant,ZWJ" sequence, however, must not be flagged for
     the `rphf` substitution.
 
 
@@ -1083,7 +1128,7 @@ in the font.
 Because Devanagari incorporates the `BLWF_MODE_PRE_AND_POST` shaping
 characteristic, any pre-base consonants and any post-base consonants
 may potentially match a `blwf` substitution; therefore, both cases must
-be tagged for comparison. Note that this is not necessarily the case in other
+be flagged for comparison. Note that this is not necessarily the case in other
 Indic scripts that use a different `BLWF_MODE_` shaping
 characteristic. 
 
@@ -1100,20 +1145,20 @@ base consonant or syllable base with "half forms" of the consonant
 glyphs. There are four exceptions to the default behavior, for which
 the shaping engine must test:
 
-  - Initial "Ra,Halant" sequences, which should have been tagged for
-    the `rphf` feature earlier, must not be tagged for potential
+  - Initial "Ra,Halant" sequences, which should have been flagged for
+    the `rphf` feature earlier, must not be flagged for potential
     `half` substitutions.
 
-  - Non-initial "Ra,Halant" sequences, which should have been tagged
-    for the `rkrf` or `blwf` features earlier, must not be tagged for
+  - Non-initial "Ra,Halant" sequences, which should have been flagged
+    for the `rkrf` or `blwf` features earlier, must not be flagged for
     potential `half` substitutions.
 
   - A sequence matching "_Consonant_,Halant,ZWJ,_Consonant_" must be
-    tagged for potential `half` substitutions, even though the presence of the
+    flagged for potential `half` substitutions, even though the presence of the
     zero-width joiner suppresses the `cjct` feature in a later step.
 
   - A sequence matching "_Consonant_,Halant,ZWNJ,_Consonant_" must not be
-    tagged for potential `half` substitutions.
+    flagged for potential `half` substitutions.
 
 ![Half-form formation](/images/devanagari/devanagari-half.png)
 
@@ -1147,7 +1192,7 @@ The `cjct` feature replaces sequences of adjacent consonants with
 conjunct ligatures. These sequences must match "_Consonant_,Halant,_Consonant_".
 
 A sequence matching "_Consonant_,Halant,ZWJ,_Consonant_" or
-"_Consonant_,Halant,ZWNJ,_Consonant_" must not be tagged to form a conjunct.
+"_Consonant_,Halant,ZWNJ,_Consonant_" must not be flagged to form a conjunct.
 
 The font's GSUB rules might be implemented so that `cjct`
 substitutions apply to half-form consonants; therefore, this feature
@@ -1292,7 +1337,7 @@ the base consonant or syllable base.
 #### 4.5: Initial matras ####
 
 Any left-side dependent vowels (matras) that are at the start of a
-word must be tagged for potential substitution by the `init` feature
+word must be flagged for potential substitution by the `init` feature
 of GSUB.
 
 Devanagari does not use the `init` feature, so this step will
@@ -1303,9 +1348,13 @@ order to maintain compatibility with the other Indic scripts.
 ### 5: Applying all remaining substitution features from GSUB ###
 
 In this stage, the remaining substitution features from the GSUB table
-are applied. The order in which these features are applied is not
-canonical; they should be applied in the order in which they appear in
-the GSUB table in the font. 
+are applied. In preparation for this stage, glyph sequences should be
+flagged for possible application of GSUB features in stage 2,
+step 10.
+
+The order in which these features are applied is not canonical; they
+should be applied in the order in which they appear in the GSUB table
+in the font.
 
 	init (not used in Devanagari)
 	pres
@@ -1358,9 +1407,11 @@ typographically problematic.
 ### 6: Applying remaining positioning features from GPOS ###
 
 In this stage, mark positioning, kerning, and other GPOS features are
-applied. As with the preceding stage, the order in which these
-features are applied is not canonical; they should be applied in the
-order in which they appear in the GPOS table in the font.
+applied.
+
+As with the preceding stage, the order in which these features are
+applied is not canonical; they should be applied in the order in which
+they appear in the GPOS table in the font.
 
         dist
         abvm