Incremental Font Transfer

1. Introduction

This section is not normative.

Incremental Font Transfer (IFT) is a technology to improve the latency of remote fonts (or "web fonts") on the web. Without this technology, a browser needs to download every last byte of a font before it can render any characters using that font. IFT allows the browser to download only some of the bytes in the file, thereby decreasing the perceived latency between the time when a browser realizes it needs a font and when the necessary text can be rendered with that font. Unlike traditional font subsetting approaches Incremental Font Transfer retains the encoding of layout rules between segments (Progressive Font Enrichment: Evaluation Report § fail-subset).

The success of WebFonts is unevenly distributed. This specification allows WebFonts to be used where slow networks, very large fonts, or complex subsetting requirements currently preclude their use. For example, even using WOFF 2 [WOFF2], fonts for CJK languages can be too large to be practical.

1.1. Technical Motivation: Evaluation Report

See the Progressive Font Enrichment: Evaluation Report [PFE-report] for the investigation which led to this specification.

1.2. Overview

An incremental font is a regular OpenType font that is reformatted to include incremental functionality, partly in virtue of two additional tables. Using these new tables the font can be augmented (eg. to cover more codepoints) by loading and applying patches to it.

The IFT technology has four main pieces:

• § 4 Extending a Font Subset: provides the algorithm that is used by a client to select and apply patches.

• § 5 Extensions to the Font Format: defines the new tables which contain a list of patches that are available to be applied to a font.

• § 6 Font Patch Formats: defines three different types of patches that can be used. Two are "generic" binary patches, one is specific to the font’s format for storing glyph data.

• § 7 Encoder: creates the font and associated patches that form an incremental font.

At a high level an incremental font is used like this:

1. The client downloads an initial font file, which contains some initial subset of data from the full version of the font along with embedded data describing the set of patches which can be used to extend the font.

2. Based on the content to be rendered, the client selects, downloads, and applies patches to extend the font to cover additional characters, layout features, and/or variation space. This step is repeated each time there is new content.

1.3. Creating an Incremental Font

It is expected that the most common way to produce an incremental font will be to convert an existing font to use the incremental encoding defined in this specification. At a high level converting an existing font to be incremental will look like this:

1. Choose the content of the initial subset, this will be included in the initial font file the client loads and usually consists of any data from the original font that is expected to always be needed.

2. Choose a segmentation of the font. Individual segments will be added to the base subset by the client using patches. Choosing an appropriate segmentation is one of the most important parts of producing an efficient encoding.

3. Based on these choices, generate a set of patches, where each patch adds the data for a particular segment relative to either the initial font file or a previous patch.

4. Generate the initial font file including the initial subset and a patch mapping. This mapping lists all of the available patch files, the url they reside at, and information on what data the patch will add to the font.

Note: this is a highly simplified description of creating an incremental font, a more in-depth discussion of generating an encoding and requirements on the encoding can be found in the § 7 Encoder section.

1.4. Performance Considerations and the use of Incremental Font Transfer

Using incremental transfer may not always be beneficial, depending on the characteristics of the font, the network, and the content being rendered. This section provides non-normative guidance to help decide when incremental transfer should be utilized.

It is common for an incremental font to trigger the loading of multiple patches in parallel. So to maximize performance, when serving an incremental font it is recommended that an HTTP server which is capable of multiplexing (such as [rfc9113] or [rfc9114]) is used.

Incrementally loading a font has a fundamental performance trade off versus loading the whole font. Simplistically, under incremental transfer less bytes may be transferred at the potential cost of increasing the total number of network requests being made, and/or increased request processing latency. In general incremental font transfer will be beneficial where the reduction in latency from sending less bytes outweighs additional latency introduced by the incremental transfer method.

The first factor to consider is the language of the content being rendered. The evaluation report contains the results of simulating incremental font transfer across three categories of languages (Progressive Font Enrichment: Evaluation Report § langtype). See it’s conclusions Progressive Font Enrichment: Evaluation Report § conclusions for a discussion of the anticipated performance of incremental font transfer across the language categories.

Next, how much of the font is expected to be needed? If it’s expected that most of the font will be needed to render the content, then incremental font transfer is unlikely to be beneficial. In many cases however only part of a font is expected to be needed. For example:

• If the font contains support for several languages but a user is expected to only render content in a subset of those languages.

• If the content being rendered uses a small subset of the total characters in a font. This is often the case for Chinese, Japanese, Korean, Emoji, and Icon fonts.

• Only a small amount of text is being rendered. For example a font that is only used for a headline.

An alternative to incremental transfer is to break a font into distinct subsets (typically by script) and use the unicode range feature of @font-face to load only the subsets needed. However, this can alter the rendering of some content Progressive Font Enrichment: Evaluation Report § fail-subset if there are layout rules between characters in different subsets. Incremental font transfer does not suffer from this issue as it can emcompass the original font and all of it’s layout rules.

1.4.1. Reducing the Number of Network Requests

As discussed in the previous section the most basic implementation of incremental font transfer will tend to increase the total number of requests made vs traditional font loading. Since each augmentation will typically require at least one round trip time, performance can be negatively impacted if too many requests are made. Depending on which patch types are available and how much information is provided in the patch mapping, a client might preemptively request patches for codepoints that are not currently needed, but expected to be needed in the future. Intelligent use of this feature by an implementation can help reduce the total number of requests being made. The evaluation report explored this by testing the performance of a basic character frequency based codepoint prediction scheme and found it improved overall performance.

2. Opt-In Mechanism

Web pages can choose to opt-in to incremental transfer for a font via the use of a CSS font tech keyword (CSS Fonts 4 § 11.1 Font tech) inside the ''@font-face'' block. The keyword incremental is used to indicate the referenced font contains IFT data and should only be loaded by a user agent which supports incremental font transfer.

@font-face {
    font-family: "MyCoolWebFont";
    src: url("MyCoolWebFont.otf") tech(incremental);
}

Note: Each individual @font-face block may or may not opt-in to IFT. This is due to the variety of ways fonts are used on web pages. Authors have control over which fonts they want to use this technology with, and which they do not.

Note: the IFT tech keyword can be used in conjunction with other font tech specifiers to perform font feature selection. For example a @font-face could include two URIs one with tech(incremental, color-COLRv1) and the other with tech(incremental, color-COLRv0).

2.1. Offline Usage

Special consideration must be taken when saving a page for offline usage that uses an incrementally transferred font since the saved page won’t be able to increment the font if content changes (eg. due to JavaScript execution). In these cases the page saving mechanism should fully expand the incremental font by invoking Fully Expand a Font Subset and replace references to the incremental font with the fully expanded one.

3. Definitions

3.1. Font Subset

A font subset is a modified version of a font file [iso14496-22] that contains only the data needed to render a subset of:

• the codepoints,

• layout features,

• and design-variation space.

supported by the original font. When a subsetted font is used to render text using any combination of the subset codepoints, layout features, or design-variation space it should render identically to the original font. This includes rendering with the use of any optional typographic features that a renderer may choose to use from the original font, such as hinting instructions. Design variation spaces are specified using the user-axis scales (OpenType Specification § otvaroverview#coordinate-scales-and-normalization).

A font subset definition describes the minimum data (codepoints, layout features, variation axis space) that a font subset should support.

Note: For convenience the remainder of this document links to the [open-type] specification which is a copy of [iso14496-22].

3.2. Font Patch

A font patch is a file which encodes changes to be made to an IFT-encoded font. Patches are used to extend an existing font subset and provide expanded coverage.

A patch format is a specified encoding of changes to be applied relative to a font subset. A set of changes encoded according to the format is a font patch. Each patch format has an associated patch application algorithm which takes a font subset and a font patch encoded in the patch format as input and outputs an extended font subset.

3.3. Patch Map

A patch map is an open type table which encodes a collection of mappings from font subset definitions to URIs which host patches that extend the incremental font. A patch map table encodes a list of patch map entries, where each entry has a key and value. The key is a font subset definition and the value is a URI, the § 6 Font Patch Formats used by the data at the URI, and the compatibility ID of the patch map table. More details of the format of patch maps can be found in § 5 Extensions to the Font Format.

Patch Map Entry summary:

3.4. Explanation of Data Types

Encoded data structures in the remainder of this specification are described in terms of the data types defined in OpenType Specification § otff#data-types. As with the rest of OpenType, all fields use "big-endian" byte ordering.

4. Extending a Font Subset

This section defines the algorithm that a client uses to extend an incremental font subset to cover additional codepoints, layout features and/or design space.

4.1. Patch Invalidations

The application of a patch to an incremental font subset may invalidate some or all of the other mapped patches, so that if one of those were to be applied the result would be a corrupt font. Patch validity is tracked by the compatibility ID from the § 5.2 Patch Map Table. Every patch has a compatibility ID encoded within it which needs to match the compatibility ID from the § 5.2 Patch Map Table which lists that patch. Every patch is categorized into one of three types, depending on how it does or does not invalidate other patches in the map(s):

• Full Invalidation: when this patch is applied all other patches currently listed in the font subset are invalidated. The compatibility ID in both the 'IFT ' and 'IFTX' § 5.2 Patch Map Table will be changed.

• Partial Invalidation: when this patch is applied all other patches in the same § 5.2 Patch Map Table will be invalidated. The compatibility ID of only the § 5.2 Patch Map Table which contains this patch will be changed.

• No Invalidation: no other patches will be invalidated by the application of this patch. The compatibility ID of the 'IFT ' and 'IFTX' § 5.2 Patch Map Table will not change.

The invalidation behavior of a specific patch is encoded in its format number, which can be found in § 6.1 Formats Summary.

4.2. Incremental Font Extension Algorithm

Extend an Incremental Font Subset

The inputs to this algorithm are:

• font subset: an incremental font subset.

• original font subset URI: an absolute URI which identifies the location of the original incremental font that font subset was derived from.

• target subset definition: the font subset definition that the client wants to extend font subset to cover.

The algorithm outputs:

• extended font subset: an extended version of font subset. May or may not be an incremental font.

The algorithm:

1. Set extended font subset to font subset.

2. Load the 'IFT ' and 'IFTX' (if present) mapping tables from extended font subset. Both tables are formatted as a § 5.2 Patch Map Table. Check that they are valid according to the requirements in § 5.2 Patch Map Table. If either table is not valid, invoke Handle errors. If extended font subset does not have an 'IFT ' table, then it is not an incremental font and cannot be extended, return extended font subset.

3. For each of tables 'IFT ' and 'IFTX' (if present): convert the table into a list of entries by invoking Interpret Format 1 Patch Map or Interpret Format 2 Patch Map. Concatenate the returned entry lists into a single list, entry list.

4. For each entry in entry list invoke Check entry intersection with entry and target subset definition as inputs, if it returns false remove entry from entry list.

5. If entry list is empty, then the extension operation is finished, return extended font subset.

6. Pick one entry from entry list with the following procedure:

   • If entry list contains one or more patch map entries which have a patch format that is Full Invalidation then, select exactly one of the Full Invalidation entries in entry list. The criteria for selecting the single entry is left up to the implementation to decide.

   • Otherwise if entry list contains one or more patch map entries which have a patch format that is Partial Invalidation then, select exactly one of the Partial Invalidation entries in entry list. The criteria for selecting the single entry is left up to the implementation to decide.

   • Otherwise select exactly one of the No Invalidation entries in entry list. The criteria for selecting the single entry is left up to the implementation to decide.

7. Load patch file by invoking Load patch file with the original font subset URI as the original font URI and the entry patch URI as the patch URI.

8. Apply patch file using the appropriate application algorithm (matching the patches format in entry) from § 6 Font Patch Formats to apply the patch file using the patch URI and the compatibility id from entry to extended font subset.

9. Go to step 2.

Note: the algorithm here presents patch loads as being done one at a time; however, to improve performance client implementations are encouraged to pre-fetch patch files that will be applied in later iterations by the algorithm. The invalidation categories can be used to predict which intersecting patches from step 4 will remain be valid to be applied. For example: in a case where there are only "No Invalidation" intersecting patches the client could safely load all intersecting patches in parallel, since no patch application will invalidate any of the other intersecting patches.

Check entry intersection

The inputs to this algorithm are:

• mapping entry: a patch map entry.

• subset definition: a font subset definition.

The algorithm outputs:

• intersects: true if subset definition intersects mapping entry, otherwise false.

The algorithm:

1. For each set in subset definition (codepoints, feature tags, design space) check if the set intersects the corresponding set from mapping entry. A set intersects when:

   	subset definition set is empty	subset definition set is not empty
   mapping entry set is empty	true	true
   mapping entry set is not empty	false	true if the two sets intersect

2. If all sets checked in step 1 intersect, then return true for intersects otherwise false.

Load patch file

The inputs to this algorithm are:

• Patch URI: A URI Reference identifying the patch file to load. As a URI reference this may be a relative path.

• Original Font URI: An absolute URI which identifies the original incremental font that the patch URI was derived from.

The algorithm outputs:

• patch file: the content (bytes) identified by Patch URI.

The algorithm:

1. Perform reference resolution on Patch URI using Original Font URI as the base URI to produce the target URI.

2. Retrieve the contents of target URI using the fetching capabilities of the implementing user agent. For web browsers, [fetch] should be used. Return the retrieved contents as patch file, or an error if the fetch resulted in an error.

Handle errors

Coming soon.

4.3. Fully Expanding a Font

This sections defines an algorithm that can be used to transform an incremental font into a fully expanded non-incremental font. This process loads all available data provided by the incremental font and produces a single static font file that contains no further patches to be applied.

Fully Expand a Font Subset

The inputs to this algorithm are:

• font subset: an incremental font subset.

The algorithm outputs:

• expanded font: an [open-type] font that is not incremental.

The algorithm:

1. Invoke Extend an Incremental Font Subset with font subset. The input target subset definition is a special one which is considered to intersect all entries in the Check entry intersection step. Return the resulting font subset as the expanded font.

5. Extensions to the Font Format

An incremental font follows the existing OpenType format, but includes two new tables identified by the 4-byte tags 'IFT ' and 'IFTX'. These new tables are both patch maps. All incremental fonts must contain the 'IFT ' table. The 'IFTX' table is optional. When both tables are present, the mapping of the font as a whole is the union of the mappings of the two tables. The two new tables are used only in this specification and are not being added to the Open-Type specification.

Note: allowing the mapping to be split between two distinct tables allows an incremental font to more easily make use of multiple patch types. For example all patches of one type can be specified in the 'IFT ' table, and all patches of a second type in the 'IFTX' table. Those patches can make updates only to one of the mapping tables and avoid making conflicting updates.

5.1. Incremental Font Transfer and WOFF2

[WOFF2] is a commonly used to encode fonts for transfer on the web. Incremental fonts can be encoded with WOFF2 as long as special care is taken. If an incremental font will be encoded by WOFF2 for transfer:

1. The incremental font should not make use of § 6.2 Brotli Patch patches. The WOFF2 format does not guarantee the ordering of tables in the decoded font. § 6.2 Brotli Patch patches are relative to a specific fixed set of bytes and thus cannot be used if the decoded font has unpredictable decoded bytes. § 6.3 Per Table Brotli patches do not depend on a specific table ordering and may be used.

2. If the WOFF2 encoding will include a transformed glyf and loca table (WOFF 2.0 § 5.1 Transformed glyf table format) then, the incremental font should not contain § 6.3 Per Table Brotli patches which modify either the glyf or loca table. The WOFF2 format does not guarantee the specific bytes that result from decoding a transformed glyf and loca table, so as with #1 brotli patches cannot be used. § 6.4 Glyph Keyed patches may be used in conjunction with a transformed glyf and loca table.

3. When utilizing a WOFF2 encoded IFT font, the client must first fully decode the WOFF2 font before § 4 Extending a Font Subset.

The 'IFT ' and 'IFTX' tables can be processed and brotli encoded by a WOFF2 encoder following the standard process defined in WOFF 2.0 § 5 Compressed data format.

5.2. Patch Map Table

A patch map is encoded in one of two formats:

• Format 1: a limited, but more compact encoding. It encodes a one-to-one mapping from glyph id to patch URIs. It does not support font subset definitions with design space or entries with overlapping subset definitions.

• Format 2: can encode arbitrary mappings including ones with design space or overlapping subset definitions. However, it is typically less compact than format 1.

Each format defines an algorithm for interpreting bytes encoded with that format to produce the list of entries it represents. The Extend an Incremental Font Subset algorithm invokes the interpretation algorithms and operates on the resulting entry list. The encoded bytes are the source of truth at all times for the patch map. Patch application during subset extension will alter the encoded bytes of the patch map and as a result the entry list derived from the encoded bytes will change. The extension algorithm reinterprets the encoded bytes at the start of every iteration to pick up any changes made in the previous iteration.

5.2.1. Patch Map Table: Format 1

Format 1 Patch Map encoding:

Glyph Map encoding:

A glyph map table associates each glyph index in the font with an entry index.

Feature Map encoding:

A feature map table associates combinations of feature tags and glyphs with an entry index.

FeatureRecord encoding:

EntryMapRecord encoding:

An entry map record matches any entry indices that are greater than or equal to firstEntryIndex and less than or equal to lastEntryIndex.

5.2.1.1. Interpreting Format 1

This algorithm is used to convert a format 1 patch map into a list of patch map entries.

Interpret Format 1 Patch Map

The inputs to this algorithm are:

• patch map: a Format 1 Patch Map encoded patch map.

• font subset: the font subset which contains patch map.

The algorithm outputs:

• entry list: a list of patch map entries.

The algorithm:

1. Check that the patch map has format equal to 1 and is valid according to the requirements in § 5.2.1 Patch Map Table: Format 1. If it is not return an error.

2. For each unique entry index in entryIndex:

   • a. If the bit for entry index in appliedEntriesBitMap is set to 1, skip this entry index.

   • b. Collect the set of glyph indices that map to the entry index.

   • c. Convert the set of glyph indices to a set of unicode code points using the code point to glyph mapping in the cmap table of font subset. Ignore any glyph indices that are not mapped by cmap.

   • d. Convert entry index into a URI by applying uriTemplate following § 5.2.3 URI Templates.

   • e. Add an entry to entry list whose subset definition contains only the unicode code point set and maps to the generated URI, the patch encoding specified by patchEncoding, and compatibilityId.

3. For each FeatureRecord and associated EntryMapRecord in featureRecords and entryMapRecords:

   • For each entry index between firstEntryIndex (inclusive) and lastEntryIndex (inclusive), find the set of unicode codepoints associated with that entry index using the same process as in step 2b through 2c.

     • Compute mapped entry index = FeatureRecord::firstEntryIndex + entry index - EntryMapRecord::firstEntryIndex.

     • Convert mapped entry index into a URI by applying uriTemplate following § 5.2.3 URI Templates.

     • If the bit for mapped entry index in appliedEntriesBitMap is set to 1, skip this entry index.

     • Add an entry to entry list whose subset definition contains the unicode code point set and a feature tag set containing only featureTag. The entry maps to the generated URI, the patch encoding specified by patchEncoding, and compatibilityId.

4. Return entry list.

5.2.1.2. Remove Entries from Format 1

This algorithm is used to remove entries from a format 1 patch map. This removal modifies the bytes of the patch map but does not change the number of bytes.

Remove Entries from Format 1 Patch Map

The inputs to this algorithm are:

• patch map: a Format 1 Patch Map encoded patch map. May be modified by this procedure.

• patch URI: URI for a patch which identifies the entries to be removed.

The algorithm:

1. Check that the patch map has format equal to 1 and is valid according to the requirements in § 5.2.1 Patch Map Table: Format 1. If it is not return an error.

2. For each unique entry index in entryIndex of patch map:

   • If the bit for entry index in appliedEntriesBitMap is set to 1, skip this entry index.

   • Convert entry index into a URI by applying uriTemplate following § 5.2.3 URI Templates.

   • If the generated URI is equal to patch URI then set the bit for entry index in appliedEntriesBitMap to 1.

5.2.2. Patch Map Table: Format 2

Format 2 Patch Map encoding:

Mapping Entries encoding:

Mapping Entry encoding:

If an encoder is producing patches that will be stored on a file system and then served it’s recommended that only numeric entry IDs be used (via entryIdDelta) as these will generally produce the smallest encoding of the format 2 patch map. String IDs are useful in cases where patches are not being stored in advance and the ID strings can be then used to encode information about the patch being requested.

If an encoder is using string IDs via entryIdStringData and entryIdStringLength then, the encoder should only use unreserved uri characters in the ID strings. This will prevent the need for percent encoding of the ID values when expanding the URI templates and maximize compatibility of the URLs and associated file names. Particularly, if patch files are to be stored on a file system which does not support unicode encoded file names or is case insensitive then special care needs to be taken when choosing the set of characters to use in the file names.

Design Space Segment encoding:

5.2.2.1. Interpreting Format 2

This algorithm is used to convert a format 2 patch map into a list of patch map entries.

Interpret Format 2 Patch Map

The inputs to this algorithm are:

• patch map: a Format 2 Patch Map encoded patch map.

The algorithm outputs:

• entry list: a list of patch map entries.

The algorithm:

1. Check that the patch map has format equal to 2 and is valid according to the requirements in § 5.2.2 Patch Map Table: Format 2. If it is not return an error.

2. Initialize last entry id to 0, current byte to 0, and current id string byte to 0.

3. Invoke Interpret Format 2 Patch Map Entry, entryCount times. For each invocation:

   • pass in the bytes from patch map starting from entries[current byte] to the end of patch map, the bytes from patch map starting from entryIdStringData[current id string byte] to the end of patch map if entryIdStringData is non zero, last entry id, defaultPatchEncoding, and uriTemplate.

   • Set last entry id to the returned entry id.

   • Add the returned consumed byte count to current byte.

   • Add the returned consumed id string byte count to current id string byte.

   • If the returned value of ignored is false, then set the compatibility ID of the returned entry to compatibilityId and add the entry to entry list.

4. Return entry list.

Interpret Format 2 Patch Map Entry

The inputs to this algorithm are:

• entry bytes: a byte array that contains an encoded Mapping Entry.

• id string bytes (optional): a byte array the contains entry ID strings.

• last entry id: the entry id of the entry preceding this one.

• default patch encoding: the default patch encoding if one isn’t specified.

• uri template: the URI template used to locate patches.

The algorithm outputs:

• entry id: the numeric or string id of this entry.

• entry: a single entry.

• consumed bytes: the number of bytes used to encode the entry.

• consumed id string bytes: the number of bytes used to encode the entry id string.

• ignored: if true, then this entry should be ignored.

The algorithm:

1. For the all steps whenever data is loaded from entry bytes increment consumed bytes with the number of bytes read.

2. If id string bytes is not present then, set entry id = last entry id + 1.

3. Set the patch encoding of entry to default patch encoding.

4. Read formatFlags from entry bytes.

5. If formatFlags bit 0 is set, then the feature tag and design space lists are present:

   • Read the feature tag list specified by featureCount and featureTags from entry bytes and add the loaded tags to entry.

   • Read the design space segment list specified by designSpaceCount and designSpaceSegments from entry bytes and add the design space segments to entry. Each segment defines an interval from start to end inclusive for the axis identified by tag.

6. If formatFlags bit 1 is set, then the copy indices list is present:

   • Read the copy indices list specified by copyCount and copyIndices from entry bytes.

   • The copy indices refer to previously loaded entries. 0 is the first Mapping Entry in entries, 1 the second and so on. For each index in copyIndices locate the previously loaded entry with a matching index and add all code points, feature tags, and design space segments from it to entry.

7. If formatFlags bit 2 is set, then an id delta or id string length is present:

   • If id string bytes is not present then, read the id delta specified by entryIdDelta from entry bytes and add the delta to entry id. If entry id is negative this is an error and the mapping is malformed.

   • Otherwise if id string bytes is present then, read entryIdStringLength bytes from id string bytes and interpret the bytes as a [UTF-8] string. Set entry id to the result.

8. If formatFlags bit 3 is set, then a patch encoding is present. Read the encoding specified by patchEncoding from entry bytes and set the patch encoding of entry to the read value.

9. If one or both of formatFlags bit 4 and bit 5 are set, then a codepoint list is present:

   • If formatFlags bit 4 is 0 and bit 5 is 1, then read the 2 byte (uint16) bias value from entry bytes.

   • If formatFlags bit 4 is 1 and bit 5 is 1, then read the 3 byte (uint24) bias value from entry bytes.

   • Otherwise the bias is 0.

   • Read the sparse bit set codepoints from entry bytes following § 5.2.2.3 Sparse Bit Set. For each codepoint add the bias value to it and then add the result to the codepoint set in entry.

10. If formatFlags bit 6 is set, then set ignored to true. Otherwise ignored is false.

11. Convert entry id into a URI by applying uri template following § 5.2.3 URI Templates. Set the patch uri of entry to the generated URI.

12. Return entry id, entry, consumed bytes, entryIdStringLength as consumed id string bytes, and ignored.

5.2.2.2. Remove Entries from Format 2

This algorithm is used to remove entries from a format 2 patch map. This removal modifies the bytes of the patch map but does not change the number of bytes.

Remove Entries from Format 2 Patch Map

The inputs to this algorithm are:

• patch map: a Format 2 Patch Map encoded patch map. May be modified by this procedure.

• patch URI: URI for a patch which identifies the entries to be removed.

This algorithm is a modified version of Interpret Format 2 Patch Map, invoke Interpret Format 2 Patch Map with patch map as an input but with the following changes:

• After step 11 of Interpret Format 2 Patch Map Entry: compare the URI generated in step 11 to patch URI if they are equal then, set bit 6 of formatFlags to 1.

• The return value of Interpret Format 2 Patch Map is not used.

5.2.2.3. Sparse Bit Set

A sparse bit set is a data structure which compactly stores a set of distinct unsigned integers. The set is represented as a tree where each node has a fixed number of children that recursively sub-divides an interval into equal partitions. A tree of height H with branching factor B can store set membership for integers in the interval [0 to B^H-1] inclusive. The tree is encoded into an array of bytes for transport.

Sparse Bit Set encoding:

The exact length of treeData is initially unknown, it’s length is determined by executing the decoding algorithm. When using branch factors of 2 or 4 the last node may only partially consume the bits in a byte. In that case all remaining bits are unused and ignored.

Branch Factor Encoding:

Decoding sparse bit set treeData

The inputs to this algorithm are:

• H: the tree height, H, from the header byte.

• B: the branch factor, B, from the header byte.

• treeData: array of bytes to be decoded.

The algorithm outputs:

• S: a set of unsigned integers.

The algorithm, using a FIFO (first in first out) queue Q:

1. If H is equal to 0, then this is an empty set and no bytes of treeData are consumed. Return an empty set.

2. Insert the tuple (0, 1) into Q.

3. Initialize S to an empty set.

4. treeData is interpreted as a string of bits where the least significant bit of treeData[0] is the first bit in the string, the most significant bit of treeData[0] is the 8th bit, and so on.

5. If Q is empty return S.

6. Extract the next tuple t from Q. The first value of in t is start and the second value is depth.

7. Remove the next B bits from the treeData bit string. The first removed bit is v₁, the second is v₂, and so on until the last removed bit which is v_B. If prior to removal there were less than B bits left in treeData, then treeData is malformed, return an error.

8. If all bits v₁ through v_B are 0, then insert all integers in the interval [start, start + B^{H - depth + 1}) into S. Go to step 5.

9. For each v_i which is equal to 1 in v₁ through v_B: If depth is equal to H add integer start + i - 1 to S. Otherwise, insert the tuple (start + (i - 1) * B^{H - depth}, depth + 1) into Q.

10. Go to step 5.

Note: when encoding sparse bit sets the encoder can use any of the possible branching factors, but it is recommended to use 4 as that has been shown to give the smallest encodings for most unicode codepoint sets typically encountered.

bit string:
|-- header --|- lvl 0 |---- level 1 ----|------- level 2 -----------|
| B=8 H=3    |   n0   |   n1       n2   |   n3       n4       n5    |
[ 01  11000 0 10000100 10001000 10000000 00100000 01000000 00010000 ]

Which then becomes the byte string:
[
  0b00001110,
  0b00100001,
  0b00010001,
  0b00000001,
  0b00000100,
  0b00000010,
  0b00001000
]

bit string:
|-- header -- |
| B=2 H=0     |
[ 00  00000 0 ]

Which then becomes the byte string:
[
  0b00000000,
]

bit string:
|-- header --| l0 |- lvl 1 -| l2  |
| B=4 H=3    | n0 | n1 | n2 | n3  |
[ 10  11000 0 1100 0000 1000 1100 ]

byte string:
[
  0b00001101,
  0b00000011,
  0b00110001
]

5.2.3. URI Templates

URI templates [rfc6570] are used to convert numeric or string IDs into URIs where patch files are located. A string ID is a sequence of Unicode code point values obtained from decoding a [UTF-8] string (see URI Template § section-1.6). Several variables are defined which are used to produce the expansion of the template:

The variables d1 through d4 select a specific character of the id variable. In this context a character is a [unicode] code point. If the code point is not an ASCII code point then during expansion it will need to be converted to [UTF-8] and percent encoded as required by URI Template § section-1.6.

6. Font Patch Formats

In incremental font transfer font subsets are extended by applying patches. This specification defines three patch formats, each appropriate to its own set of augmentation scenarios. A single encoding can make use of more than one patch format.

6.1. Formats Summary

The following patch formats are defined by this specification:

More detailed descriptions of each algorithm can be found in the following sections.

6.2. Brotli Patch

In a brotli patch the target file is encoded with brotli compression using the source file as a shared LZ77 dictionary. A brotli encoded patch consists of a short header followed by brotli encoded data.

Brotli patch encoding:

6.2.1. Applying Brotli Patches

This patch application algorithm is used to apply a brotli patch to extend a font subset to cover additional codepoints, features, and/or design-variation space.

Apply brotli patch

The inputs to this algorithm are:

• base font subset: a font subset which is to be extended.

• patch: a brotli patch to be applied to base font subset.

• compatibility id: The compatibility ID from the 'IFT ' or 'IFTX' table of base font subset which listed this patch.

The algorithm outputs:

• extended font subset: a font subset that has been extended by the patch.

The algorithm:

1. Check that the format field in patch is equal to 'ifbr', if it is not equal, then patch is not correctly formatted. Patch application has failed, return an error.

2. Check that the compatibilityId field in patch is equal to compatibility id. If there is no match, or base font subset does not have either an 'IFT ' or 'IFTX' table, then patch application has failed, return an error.

3. Decode the brotli encoded data in brotliStream following Brotli Compressed Data Format § section-10 and using the base font subset as a shared LZ77 dictionary. Return the decoded result as the extended font subset

6.3. Per Table Brotli

A per table brotli patch contains a collection of patches which are applied to the individual font tables in the input font file. Each table patch is encoded with brotli compression using the corresponding table from the input font file as a shared LZ77 dictionary. A per table brotli encoded patch consists of a short header followed by one or more brotli encoded patches. In addition to patching tables, patches may also replace (existing table data is not used) or remove tables in a font subset.

Per table brotli patch encoding:

The difference between two consecutive offsets in the patches array gives the size of that TablePatch.

TablePatch encoding:

6.3.1. Applying Per Table Brotli Patches

This patch application algorithm is used to apply a per table brotli patch to extend a font subset to cover additional codepoints, features, and/or design-variation space.

Apply per table brotli patch

The inputs to this algorithm are:

• base font subset: a font subset which is to be extended.

• patch: a per table brotli patch to be applied to base font subset.

• compatibility id: The ID number from the 'IFT ' or 'IFTX' table of base font subset which listed this patch.

The algorithm outputs:

• extended font subset: a font subset that has been extended by the patch.

The algorithm:

1. Initialize extended font subset to be an empty font with no tables.

2. Check that the format field in patch is equal to 'ifbt', if it is not equal then patch is not correctly formatted. Patch application has failed, return an error.

3. Check that the compatibilityId field in patch is equal to compatibility id. If there is no match, or base font subset does not have either an 'IFT ' or 'IFTX' table, then patch application has failed, return an error.

4. In the following steps, adding a table to extended font subset consists of adding the table’s data to the font and inserting a new entry into the table directory according to the requirements of the open type specification. That entry includes a checksum for the table data. When an existing table is copied unmodified, the client can re-use the checksum from the entry in the source font. Otherwise a new checksum will need to be computed.

5. For each entry in patches, with index i:

   • Find the TablePatch associated with index i. The object starts at the offset patches[i] (inclusive) and ends at the offset patches[i+1] (exclusive). Both offsets are relative to the start of the patch.

   • If an entry in patches was previously applied that has the same tag as this entry, then ignore this entry and continue the iteration to the next one. Entries are processed in same order as they are listed in the patches array.

   • If bit 0 (least significant bit) of flags is set, then decode brotliStream following Brotli Compressed Data Format § section-10. No shared dictionary is used. Add a table to extended font subset identified by tag with it’s contents set to the decoded brotliStream.

   • If bit 1 of flags is set, then do not copy or add a table to extended font subset identified by tag.

   • Otherwise, decode brotliStream following Brotli Compressed Data Format § section-10 and using the table identified by tag in base font subset as a shared LZ77 dictionary. Add a table to extended font subset identified by tag with it’s contents set to the decoded brotliStream.

6. For each table in base font subset which has a tag that was not found in any of the entries processed in step 5, add a copy of that table to extended font subset.

6.4. Glyph Keyed

A glyph keyed patch contains a collection of data chunks that are each associated with a glyph index and a font table. The encoded data replaces any existing data for that glyph index in the referenced table. Glyph keyed patches can encode data for glyf/loca, gvar, CFF, and CFF2 tables.

Glyph keyed patch encoding:

GlyphPatches encoding:

The difference between two consecutive offsets in the glyphDataOffsets array gives the size of that glyph data.

6.4.1. Applying Glyph Keyed Patches

This patch application algorithm is used to apply a glyph keyed patch to extend a font subset to cover additional codepoints, features, and/or design-variation space.

Apply glyph keyed patch

The inputs to this algorithm are:

• base font subset: a font subset which is to be extended.

• patch: a glyph keyed patch to be applied to base font subset.

• patch uri: the URI where the patch data is located.

• compatibility id: The compatibility ID from the 'IFT ' or 'IFTX' table of base font subset which listed this patch.

The algorithm outputs:

• extended font subset: a font subset that has been extended by the patch.

The algorithm:

1. Check that the format field in patch is equal to 'ifgk', if it is not equal, then patch is not correctly formatted and patch application has failed, return an error.

2. Check that the compatibilityId field in patch is equal to compatibility id. If there is no match, or base font subset does not have either an 'IFT ' or 'IFTX' table, then patch application has failed, return an error.

3. Decode the brotli encoded data in brotliStream following Brotli Compressed Data Format § section-10. The decoded data is a GlyphPatches table.

4. For each font table listed in tables, with index i:

   • Using the corresponding table in base font subset, synthesize a new table where the data for each glyph is replaced with the data corresponding to that glyph index from glyphData if present, otherwise copied from the corresponding table in base font subset for that glyph index.

   • The patch glyph data for a glyph index is located by finding glyphIds[j] equal to the glyph index. The offset to the associated glyph data is glyphDataOffsets[i * glyphCount + j]. The length of the associated glyph data is glyphDataOffsets[i * glyphCount + j + 1] minus glyphDataOffsets[i * glyphCount + j].

   • The specific process for synthesizing the new table, depends on the specified format of the table. Any non-glyph associated data should be copied from the table in base font subset. Tables of the type glyf/loca, gvar, CFF, or CFF2 are supported. Entries for tables of any other types must be ignored.

   • If base font subset does not have a matching table, return an error.

   • Insert the synthesized table into extended font subset.

5. Locate the § 5.2 Patch Map Table which has the same compatibilityId as compatibility id. If it is a format 1 patch map then, invoke Remove Entries from Format 1 Patch Map with the patch map table and patch uri as an input. Otherwise if it is a format 2 patch map then, invoke Remove Entries from Format 2 Patch Map with the patch map table and patch uri as an input. Copy the modified patch map table into extended font subset.

6. For each table in base font subset which has a tag that was not found in any of the entries processed in step 4 or 5, add a copy of that table to extended font subset.

7. If the contents of any font table was modified during the previous steps then, for each modified table: update the checksums in the fonts table directory to match the table’s new contents.

7. Encoder

An encoder is a tool which produces an incremental font and set of associated patches. The incremental font and associated patches produced by a compliant encoder:

1. Must meet all of the requirements in § 5 Extensions to the Font Format and § 6 Font Patch Formats.

2. Must be consistent, that is: for any possible font subset definition the result of invoking Extend an Incremental Font Subset with that subset definition and the incremental font must always be the same regardless of the particular order of patch selection chosen in step 6 of Extend an Incremental Font Subset.

3. Should preserve the functionality of the fully expanded font, that is: given the fully expanded font derived from the incremental font and any content, then the font subset produced by invoking Extend an Incremental Font Subset with the incremental font and the minimal subset definition covering that content should render identically to the fully expanded font for that content.

4. When an encoder is used to transform an existing font into an incremental font the associated fully expanded font should be equivalent to the existing font. An equivalent fully expanded font should have all of the same tables as the existing font (excluding the incremental IFT/IFTX tables) and each of those tables should be functionally equivalent to the corresponding table in the existing font. Note: the fully expanded may not always be an exact binary match with the existing font.

When an encoder is used to transform an existing font file into and incremental font and a client is implemented according to the other sections of this document, the intent of the IFT specification is that appearance and behavior of the font in the client will be the same as if the entire file were transferred to the client. A primary goal of the IFT specification is that the IFT format and protocol can serve as a neutral medium for font transfer, comparable to WOFF2. If an encoder produces an encoding from a source font which meets all of the above requirements (1. through 4.), then the encoding will preserve all of the functionality of the original font.

This may be important for cases where a foundry or other rights-owner of a font wants be confident that the encoding and transfer of that font using IFT will not change its behavior and therefore the intent of the font’s creators. Licenses or contracts might then include requirements about IFT conformance, and situations in which re-encoding a font in WOFF2 format is de facto permissible due to its content-neutrality might also permit IFT encoding of that font.

However, nothing about these requirements on encoding conformance is meant to rule out or deprecate the possibility and practical use of encodings that do not preserve all of the functionality of a source font. Any encoding meeting the minimum requirements (1. and 2. above) is valid and may have an appropriate use. Under some circumstances it might be desirable for an encoded font to omit support for some functionality/data from all of its patch files even if those were included in the original font file. In other cases a font might be directly encoded into the IFT format from font authoring source files.

7.1. Encoder Considerations

This section is not normative.

The details of the encoding process may differ by encoder and are beyond the scope of this document. However, this section provides guidance that encoder implementations may want to consider.

Utilize an existing font subsetter implementation

As discussed in § 7 Encoder a high quality encoder should preserve the functionality of the original font. One way to achieve this is to leverage an existing font subsetter implementation to produce font subsets that retain the functionality of the original font. The IFT patches can then be derived from these subsets.

A font subsetter produces a font subset from an input font based on a desired font subset definition. The practice of reliably subsetting a font is well understood and has multiple open-source implementations (a full formal description is beyond the scope of this document). It typically involves a reachability analysis, where the data in tables is examined relative to the font subset definition to see which portions can be reached by any possible content covered by the subset definition. Any reachable data is retained in the generated font subset, while any unreachable data may be removed.

In the following example psuedo code a font subsetter is used to generate an IFT encoded font that utilizes only brotli patches:

# Encodes a font (full_font) into an incremental font that starts at base_subset_def
# and can incrementally add any of subset_definitions. Returns the IFT encoded font
# and set of associated patches.
encode_as_ift(full_font, base_subset_def, subset_definitions):
  base_font = subset(full_font, base_subset_def)
  base_font, patches  = encode_node(full_font, base_font, base_subset_def, subset_definitions)
  return base_font, patches

# Update base_font to add all of the ift patch mappings to reach any of
# subset_definitions and produces the associated patches.
encode_node(full_font, base_font, cur_def, subset_definitions):
  patches = []
  next_fonts = []
  
  for each subset_def in subset_definitions not fully covered by cur_def:
    next_def = subset_def union cur_def
    next_font = subset(full_font, next_def)
    let patch_url be a new unique url

    add a mapping from, (subset_def - cur_def) to patch_url, into base_font
    next_font, patches += encode_node(full_font, next_font, next_def, subset_definitions)

    next_fonts += (next_font, next_def, patch_url)

  for each (next_font, next_def, patch_url) in next_fonts:
    patch = brotli_patch_diff(base_font, next_font)
    patches += (patch, patch_url)
  
  return base_font, patches

In this example implementation if the union of the input base subset definition and the list of subset definitions fully covers the input full font, and the subsetter implementation used correctly retains all functionality then, the above implementation should meet the requirements in § 7 Encoder to be a neutral encoding. This basic encoder implementation is for demonstration purposes and not meant to be representative of all possible encoder implementations. Notably it does not make use of nor demonstrate utilizing glyph keyed patches. Most encoders will likely be more complex and need to consider additional factors some of which are discussed in the remaining sections.

Choosing segmentations

One of the most important and complex decisions an encoder needs to make is how to segment the data in the encoded font. To maximize efficiency the encoder should try and group data (eg. codepoints) that are commonly used together into the same segments. This will reduce the amount of unneeded data load by clients when extending the font. The encoder must also decide the size of segments. Smaller segments will produce more patches, and thus incur more overhead by requiring more network requests, but will typically result in less unneeded data in each segment compared to larger segments. When segmenting codepoints, data on codepoint usage frequency can be helpful to guide segmentation.

During segmentation an encoder should also consider how codepoints interact with each other in the font. Keeping interacting codepoints in the same segment can avoid having to duplicate data between patches, which may be necessary to preserve the functionality of the original font when interacting codepoints reside in different segments.

Patches can form graphs

Brotli based patches can modify the mapping table in the current font subset, which allows a patch to update the mapping table to point to a different set of patches which are valid against the result of the patch application. This allows a graph to be formed where a font subset is the node and each patch is an edge.

Choosing patch formats

An encoder must choose the appropriate patch type to use. Brotli patches allow all types of data in the font to be patched, while glyph keyed patches are limited to updating outline and variation delta data. However, using brotli patches can cause the number of patches to increase exponentially as the number of segments increases. Glyph keyed patches on the other hand scale linearly with the number of segments.

Brotli patches are most useful when a font has sizable non-outline data and requires only a small number of segments. Glyph keyed patches are most useful when a fonts data is primarily in the outline tables and the font would benefit from using a large segment count.

Using per-table brotli patches and two mapping tables it becomes possible to mix both brotli and glyph keyed patching in the same incremental font. This can be accomplished by:

1. Keep all brotli patch entries in one mapping table and all glyph keyed entries in the other mapping table.

2. Use per-table brotli patches to update all tables except for the tables touched by the glyph keyed patches (outline, variation deltas, and the glyph keyed patch mapping table). These patches should use a small number of large segments to keep the patch count reasonable.

3. Lastly, use glyph keyed patches to update the remaining tables, here much smaller fine-grained segments can be utilized without required too many patches.

Managing the number of patches

Privacy Considerations

Coming soon.

Security Considerations

Coming soon.

Changes

Since the Working Draft of 30 May 2023 (see commit history):

• Complete rewrite of the specification. Separate 'Patch Subset' and 'Range Request' methods have been removed in favour of a single unified approach.