Reference Label Generation Rules for the Thai Language

Overview

This document specifies a reference set of Label Generation Rules (LGR) for the Thai language for the second level. The starting point for the development of this LGR can be found in the related Root Zone LGR [RZ-LGR-3-Thai]. For details and additional background on the script, see "Proposal for a Thai Script Root Zone LGR [Proposal-Thai]". The format of this file follows [RFC 7940].

This is a DRAFT document released for public comments and not final. Please see the announcement on the ICANN website for public comments on the Second Level Reference LGRs for details on how to submit comments.

Repertoire

The repertoire contains 68 code points for letters used in the Thai language. In addition, three sequences have been defined. The sequence U+0E4D U+0E32 (ํา) was defined to replace the disallowed U+0E33 (SARA AM) and to facilitate implementation of the WLE rule follows-consonant-tone as a context rule. The other two sequences were defined to restrict U+0E45 (LAKKHANGYAO) from appearing in any context other than these sequences. Accordingly, while U+0E45 (ๅ) is not listed by itself this brings the total of code points avaliable for letters to 69.

The repertoire is a subset of [Unicode 6.3]. For more detail, see Section 5, "Repertoire" in [Proposal-Thai]. (The proposal cited has been adopted for the Thai script portion of the Root Zone LGR.)

For the second level, the repertoire has been augmented with the ASCII digits, U+0030 (0) to U+0039 (9); Thai digits, U+0E50 (๐) to U+0E59 (๙); U+002D (-) HYPHEN-MINUS; the abbreviation mark U+0E2F (ฯ) THAI CHARACTER PAIYANNOI; and the repetition mark U+0E46 (ๆ) THAI CHARACTER MAIYAMOK. Also, a special sequence using the abbreviation mark has been added for a total of 95 repertoire elements.

Each code point or range is tagged with the script or scripts that the code point is used with, and one or more references documenting sufficient justification for inclusion in the repertoire, see "References" below.

Variants

This LGR defines no variants for letters, see Section 6, "Variants" in [Proposal-Thai].

Digit Variants: All Thai digits are treated as semantic variants of the corresponding common (ASCII) digits. By transitivity, they are also semantic variants of any native digits in scripts that also include the common digits. Such transitive relations are deemed to exist implicitly but are not listed explicitly in each reference LGR. (Omitting the listing of these other cross script digit variants does not affect index variant calculation, as the ASCII digit variant being smallest would always be the index variant.) There is a strong resemblance between Thai and Khmer digits, and certain Lao digits. In addition, Thai digit ZERO is a cross-script homoglyph or near homoglyph of digit ZERO in many other scripts; all of these are already implicit semantic variants by transitivity and therefore not listed here. To keep digit variant sets manageable in zones where multiple scripts are present, no attempt has been made at identifying cross-script variants among digits of different numeric value or between a digit in one script and a letter in another, such as between digit zero and Latin letter 'o'. Other mechanisms may be required to prevent homograph labels.

Character Classes

The Thai Script is an abugida in which consonant–vowel sequences are written as a unit: each unit is based on a consonant letter, and vowel, tone mark or diacritic notation are secondary. It is written with the combining marks stacked above or below the base consonant, like diacritics in European languages. However, although the concepts are quite similar, the implementations are significantly different.

Consonants: There are 44 characters that are classified as consonants; code points from this subset have been given the tag "cons". See Section 5.1, "Consonants" in [Proposal-Thai].

Vowels: The 18 vowel symbols pronounced after a consonant are non-sequential: they can be located before (lv) , after (fv), above (av) or below (bv) the consonant, or in a combination of these positions, code points from this subset have been given the tags "fv1","fv2", "fv3", "av", "bv", or "lv". There are three code point sequences defined that include vowels. (Code point sequences do not carry tag values; instead, for code point sequences the subset values are identified in comments). See Section 5.2, "Vowels" in [Proposal-Thai].

Tones: There are 5 phonemic tones: mid, low, falling, high, and rising. These 5 tones are represented by 4 tone marks plus the absence of a mark. Code points from this subset have been given the tag "tone". See section 5.3, "Tone Marks" in [Proposal-Thai].

Diacritical Marks: There are 3 above diacritic symbols that have been included here and given the tag "ad". They differ in their frequency and purpose of usage. See also the discussion in Section 5.,4 "Diacritics" in [Proposal-Thai].

• U+0E47 (MAITAIKHU) and U+0E4C (THANTHAKHAT) are commonly used for words in everyday communication
• U+0E4D (NIKHAHIT) is included because of its use to decompose U+0E33 ( ําา ) THAI CHARACTER SARA AM, which is in common use. However, NIKHAHIT may also be used by itself.

A fourth above diacritic, U+0E4E (YAMAKKAN), has been excluded from the Root Zone LGR repertoire because it is rarely used in Modern Thai or even in older Pali manuscripts; it is more common to replace it with U+0E3A (PHINTHU), a below diacritic, which has been given the tag "bd". Moreover, excluding U+0E4E (YAMAKKAN) also eliminates the chance of confusion between U+0E4E (YAMAKKAN) and U+0E4C (THANTHAKHAT). Both look similar, are always placed at the same position in the word cell, and they are normally displayed in a small size.

Abbreviation Mark: The abbreviation mark (or ellipsis) U+0E2F (PAIYANNOI) should not be at the beginning position and must end the label. An exception to this rule has been provided for by a special sequence.

Repetition Mark: The repetition mark U+0E46 (MAIYAMOK) repeats anything before it. Therefore, it should not be at the beginning position. The repetition mark must not be followed by any combining mark, or follow a leading vowel to prevent unstable rendering. A repetition mark may be repeated. A context rule has been defined to limit the repetition mark to acceptable context, however, some of the restrictions are taken care of by other rules, for example, the rule that leading vowel must be followed by a consonant.

Thai Digits: U+0E50 (๐) to U+0E59 (๙) are a set of Thai-specific digits. They are used in alternation with the European (common) digits.

Common Digits: U+0030 (0) to U+0039 (9) are the set of digits from the ASCII range.

Whole Label Evaluation (WLE) and Context Rules

Common Rules

• Hyphen Restrictions — restrictions on the allowable placement of hyphens (no leading/ending hyphen and no hyphen in positions 3 and 4). These restrictions are described in section 4.2.3.1 of RFC 5891 [150]. They are implemented here as context rule on U+002D (-) HYPHEN-MINUS.
• Leading Combining Marks — restrictions on the allowable placement of combining marks (no leading combining mark). This rule is described in section 4.2.3.2 of RFC 5891 [150].
• One non-ASCII code point — An IDN label must contain one non-ASCII code point, as described in Section 4.2.4 of RFC 5891 [150].
• Digit-mixing — no mixing between different digit sets (European and Thai digits) is allowed; implemented here as a WLE rule with associated action.

Default Actions

Actions include the default actions for LGRs as well as that needed to invalidate labels with misplaced combining marks. They are marked with ⍟. For a description see [RFC 7940].

Thai-specific Rules

The rules provided in this LGR as described in Section 7 of [Proposal-Thai] reasonably restrict labels so that they conform to Thai syllable structure. These constraints are exclusively presented as context rules.

The rules are:

• A leading-vowel must precede a consonant — See Section 7.2 in [Proposal-Thai]
• A below-vowel must follow a consonant — See Section 7.3 in [Proposal-Thai]
• An above-vowel must follow a consonant — See Section 7.3 in [Proposal-Thai]
• A below diacritic must follow a consonant — See Section 7.3 in [Proposal-Thai]
• An above-diacritic MAITAIKHU must follow a consonant — See Section 7.3 in [Proposal-Thai]
• A vowel MAIHAN AKAT must be in between a consonant and either tone or consonant — See Section 7.4 in [Proposal-Thai]
• A vowel SARA A can follow a consonant, a tone or a vowel SARA AA — See Section 7.5 in [Proposal-Thai]
• A vowel SARA AA, or an above diacritic NIKHAHIT followed by a vowel SARA AA can follow a consonant or a tone — See Sections 7.6 and 7.9 in [Proposal-Thai]
• A tone mark, THANTHAKHAT, NIKHAHIT can only follow a consonant, above-vowel or below-vowel — See section 7.7 and 7.8 in [Proposal-Thai]
• An abbreviation mark, PAIYANNOI cannot be at the beginning of the label and must end it — rule SL1 for second level
• A repetition mark, MAIYAMOK can precede a repetition mark, an independent vowel, or end of label; it cannot start a label — rule SL2 for second level

Methodology and Contributors

This reference LGR for Thai for the 2nd Level has been developed by Michel Suignard and Asmus Freytag, based on the Root Zone LGR for Thai and information contained or referenced therein, see [RZ-LGR-3-Thai]. Suitable extensions for the second level have been applied according to the [Guidelines] and with community input. The original proposal for a Root Zone LGR for the Thai script, that this reference LGR is based on, was developed by the Thai Generation Panel. For more information on methodology and contributors to the underlying Root Zone LGR, see Sections 4 and 8 in [Proposal-Thai], as well as [RZ-LGR-Overview].

References

The following general references are cited in this document:

[Guidelines]

ICANN, "Guidelines for Developing Reference LGRs for the Second Level, (Los Angeles, California: ICANN, 27 May 2020),
https://www.icann.org/en/system/files/files/lgr-guidelines-second-level-06jan16-en.pdf

[Proposal-Thai]

Proposal for a Thai Script Root Zone LGR, 25 May 2017,
https://www.icann.org/en/system/files/files/proposal-thai-lgr-25may17-en.pdf

[RFC 7940]

Davies, K. and A. Freytag, "Representing Label Generation Rulesets Using XML", RFC 7940, August 2016,
http://www.rfc-editor.org/info/rfc7940

[RZ-LGR-Overview]

Integration Panel, "Root Zone Label Generation Rules - LGR-3: Overview and Summary", 01 March 2020 (PDF),
https://www.icann.org/sites/default/files/lgr/lgr-3-overview-10jul19-en.pdf

[RZ-LGR-3-Thai]

ICANN, Root Zone Label Generation Rules for the Thai Script (und-Thai) , 10 July 2019 (XML)
https://www.icann.org/sites/default/files/lgr/lgr-3-thai-script-10jul19-en.xml

[Unicode 6.3]

The Unicode Consortium. The Unicode Standard, Version 6.3.0, (Mountain View, CA: The Unicode Consortium, 2013. ISBN 978-1-936213-08-5)
http://www.unicode.org/versions/Unicode6.3.0/

For references consulted particularly in designing the repertoire for the Thai script for the second level. please see details in the Table of References below. Reference [0] refers to the Unicode Standard version in which corresponding code points were initially encoded. References [100] and [101] correspond to sources given in [Proposal-Thai] for justifying the inclusion of for the corresponding code points. Entries in the table may have multiple source reference values.

4.1 Character Classes

The following table lists all named and implicit classes with their definition and a list of their members intersected with the current repertoire (for larger classes, this list is elided).

Name	Definition	Count	Members or Ranges	Ref	Comment
above-vowel	Tag=av	5	{0E31 0E34-0E37}		Any Thai above vowel
below-vowel	Tag=bv	2	{0E38-0E39}		Any Thai below vowel
consonant	Tag=cons	44	{0E01-0E23 0E25 0E27-0E2E}		Any Thai consonant
leading-vowel	Tag=lv	5	{0E40-0E44}		Any Thai leading vowel
repetition	Tag=rep	1	{0E46}		The Thai repetition mark, MAIYAMOK
sara-aa	Tag=sara-aa	1	{0E32}		Thai SARA AA
tone	Tag=tone	4	{0E48-0E4B}		Any Thai tone mark
c-av-bv	combined = [[:consonant:] ∪ [:above-vowel:] ∪ [:below-vowel:]]	51	{0E01-0E23 0E25 0E27-0E2E 0E31 0E34-0E39}		Any Thai consonant, vowel-above or vowel-below
ct	combined = [[:consonant:] ∪ [:tone:]]	48	{0E01-0E23 0E25 0E27-0E2E 0E48-0E4B}		Any Thai consonant or tone mark
ctaa	combined = [[:consonant:] ∪ [:tone:] ∪ [:sara-aa:]]	49	{0E01-0E23 0E25 0E27-0E2E 0E32 0E48-0E4B}		Any Thai consonant, tone or sara-aa
common-digits	Tag=Common-digit	10	{0030-0039}		Digits from the ASCII range; ⍟
thai-digits	Tag=Thai-digit	10	{0E50-0E59}		Thai digits
hyphen	Tag=Hyphen	1	{002D}		The Hyphen-minus character ⍟
implicit	Tag=abbr	1	{0E2F}		The character tagged as abbr
implicit	Tag=ad	3	{0E47 0E4C-0E4D}		Any character tagged as ad
implicit	Tag=bd	1	{0E3A}		The character tagged as bd
implicit	Tag=fv1	2	{0E30 0E32}		Any character tagged as fv1
implicit	Tag=fv3	2	{0E24 0E26}		Any character tagged as fv3
implicit	Tag=sc:Thai	80	{0E01-0E32 0E34-0E3A 0E40-0E44 0E46-0E4D 0E50-0E59}		Any character tagged as Thai
implicit	Tag=sc:Zyyy	11	{002D 0030-0039}		Any character tagged as Common

4.2 Whole label evaluation and context rules

The following table lists all named rules defined in the LGR and indicates whether they are used as trigger in an action or as context (when or not-when) for a code point or variant.

Name	Regular Expression	Used as Trigger	Anchor	Used as Context	Ref	Comment
leading-combining-mark	(start)[[\p{gc=Mn}] ∪ [∅=\p{gc=Mc}]]	✔			[150]	RFC 5891 restrictions on placement of combining marks ⍟
hyphen-minus-disallowed	(((start))← ⚓)|(⚓ →((end)))|(((start)..[:hyphen:])← ⚓)		✔	C	[150]	RFC 5891 restrictions on placement of U+002D (-) ⍟
precedes-consonant	⚓ →([:consonant:])		✔	C		WLE 7.2: check if current cp is preceding a consonant
follows-consonant	([:consonant:])← ⚓		✔	C		WLE 7.3: check if current cp is following a consonant
between-consonant-and-ct	([:consonant:])← ⚓ →([:ct:])		✔	C		WLE 7.4: check if current cp is in between a consonant and either tone or consonant
follow-consonant-tone-sara-aa	([:ctaa:])← ⚓		✔	C		WLE 7.5: U+0E30 (THAI CHARACTER SARA A, ะ) can follow a consonant, a tone or U+0E32 (THAI CHARACTER SARA AA, า)
follows-consonant-tone	([:ct:])← ⚓		✔	C		WLE 7.6, 7.9: check if current cp is following a consonant or a tone
follows-consonant-av-bv	([:c-av-bv:])← ⚓		✔	C		WLE 7.7, 7.8: A tone-mark, THANTHAKHAT, NIKAHIT can only follow a consonant, above-vowel or below-vowel
follows-any-precedes-end	(.)← ⚓ →((end))		✔	C	[102]	WLE SL1: An abbreviation mark, PAIYANNOI cannot be at the beginning of the label and must end it
follows-any-precedes-rep-cons-lv-end	(.)← ⚓ →([:consonant:]|[:repetition:]|[:leading-vowel:]|(end))		✔	C	[102]	WLE SL2: A repetition mark, MAIYAMOK cannot be at the beginning of the label and must be followed by itself, a consonat or a leading vowel
digit-mixing	([:common-digits:].*[:thai-digits:])|([:thai-digits:].*[:common-digits:])	✔				restrictions on mixing digits
ascii-only-label	(start)[\u002D\u0030-\u0039]+(end)	✔			[150]	RFC 5891 restriction requiring at least one non-ASCII code point ⍟

4.3 Actions

The following table lists the actions that are used to assign dispositions to labels and variant labels based on the specified conditions. The order of actions defines their precedence: the first action triggered by a label is the one defining its disposition.

#	Condition	Rule / Variant Set		Disposition	Ref	Comment
1	if label matches	leading-combining-mark	→	invalid	[150]	labels with leading combining marks are invalid ⍟
2	if label matches	ascii-only-label	→	invalid	[150]	ascii-only labels invalid (not IDNs) ⍟
3	if label matches	digit-mixing	→	invalid		a label violating the restriction on digit mixing is invalid
4	if at least one variant is in	{out-of-repertoire-var}	→	invalid		any variant label with a code point out of repertoire is invalid ⍟
5	if at least one variant is in	{blocked}	→	blocked		any variant label containing blocked variants is blocked ⍟
6	if each variant is in	{allocatable}	→	allocatable		variant labels with all variants allocatable are allocatable ⍟
7	if any label (catch-all)		→	valid		catch all (default action) ⍟