Well-formed, valid and the ISO code lists
A language tag can be right in two different ways: it can have the right form, and its subtags can mean something. RFC 5646 §2.2.9 defines a conformance class for each, and Bcp47LanguageTag implements both. It also offers a third check, against the ISO code lists, for callers that can’t load a registry. The LanguageTagConformance enum names all three.
| Conformance | What is checked | How to ask for it |
|---|---|---|
WellFormed | The grammar of RFC 5646 §2.1, and nothing else | Create(string), TryCreate, Parse, IsWellFormed |
Valid | RFC 5646 §2.2.9, against a loaded copy of the registry | The overloads that take a SubtagRegistry |
IsoCodeLists | The grammar, plus the code lists in the Data Standardizer packages | Create(string, LanguageTagConformance.IsoCodeLists) and the other overloads that take a conformance |
Well-formed
RFC 5646 §2.2.9 says:
A tag is considered “well-formed” if it conforms to the ABNF (Section 2.1). Language tags may be well-formed in terms of syntax but not valid in terms of content.
A well-formed tag has subtags of the right length and characters in the right order. Nothing has been looked up, so zz is well-formed even though no language has that code. The 26 grandfathered tags, such as i-klingon, are well-formed because the grammar lists them.
This is what you get by default, because it’s the only check that needs no data.
var languageTag = Bcp47LanguageTag.Create("zz");
var conformance = languageTag.Conformance; // LanguageTagConformance.WellFormed
var isWellFormed = Bcp47LanguageTag.IsWellFormed("en--US"); // false: an empty subtag
Valid
RFC 5646 §2.2.9 says a tag is valid if it satisfies these conditions:
- The tag is well-formed.
- Either the tag is in the list of grandfathered tags or all of its primary language, extended language, script, region, and variant subtags appear in the IANA Language Subtag Registry as of the particular registry date.
- There are no duplicate variant subtags.
- There are no duplicate singleton (extension) subtags.
Validity needs a registry, so you pass one in. IANA publishes the registry with a line feed alone at the end of each line, not the CRLF that RFC 5646 §3.1.1 requires, so load its file with SubtagRegistryLoadOptions.AcceptBareLineFeed.
var subtagRegistry = SubtagRegistry.CreateFromFile("language-subtag-registry.txt", SubtagRegistryLoadOptions.AcceptBareLineFeed);
var languageTag = Bcp47LanguageTag.Create("en-US", subtagRegistry);
var conformance = languageTag.Conformance; // LanguageTagConformance.Valid
The Prefix and Deprecated fields of the registry’s records aren’t checked, because §2.2.9 doesn’t list them. A deprecated subtag is still valid, and a registered extended language subtag is valid after any two- or three-letter primary language, whatever its Prefix says.
Duplicates
A repeated variant subtag or extension singleton, as in de-1901-1901 or en-a-bb-a-cc, matches the grammar, so it’s well-formed. §2.2.9 lists duplicates under validity alone, so it isn’t valid. The comparison ignores case.
The registry date
The second condition says “as of the particular registry date”, so validity depends on the copy of the registry you load. A subtag added after that copy’s FileDate isn’t valid against it. To keep up with new subtags, load a newer copy.
The ISO code lists
LanguageTagConformance.IsoCodeLists is for a caller that wants more than the grammar and can’t load a registry. It checks that a tag is well-formed and that:
- The primary language is an ISO 639-1 code, an ISO 639-2/T or ISO 639-3 code of a language with no ISO 639-1 code, an ISO 639-5 code, or in
qaatoqtz. A primary language of four to eight letters is checked by shape only. - The script is an ISO 15924 code, or in
QaaatoQabx. - The region is an ISO 3166-1 code, a code that RFC 5646 §2.2.4 reserves, or a UN M.49 area other than 830.
- No variant subtag or extension singleton is repeated.
This approximates validity, and it isn’t RFC 5646 validity. It has these gaps:
- It doesn’t check variant subtags, so
de-abcdepasses. - It doesn’t check primary languages of four to eight letters, so
abcdepasses. - It doesn’t check extended language subtags or their prefixes, so
en-abcpasses. - Its lists are only as current as the
DataStandardizer.LanguageandDataStandardizer.Geographypackages you have installed. - It can disagree with the registry in either direction. For Serbo-Croatian, it rejects
sh, which the registry has but ISO 639-1 doesn’t, and acceptshbs, which ISO 639-3 has but the registry doesn’t.
A three-letter code of a language that has an ISO 639-1 code, such as eng, is rejected, because RFC 5646 §2.2.1 says only the two-letter code is defined.
var languageTag = Bcp47LanguageTag.Create("en-NZ", LanguageTagConformance.IsoCodeLists);
var isListed = Bcp47LanguageTag.IsValid("eng", LanguageTagConformance.IsoCodeLists); // false
The three side by side
| Tag | Well-formed | ISO code lists | Valid against the IANA registry |
|---|---|---|---|
en-US | Yes | Yes | Yes |
i-klingon | Yes | Yes | Yes |
x-a | Yes | Yes | Yes |
en-Zzzz | Yes | Yes | Yes |
zz | Yes | No | No |
en-Abcd | Yes | No | No |
en-830 | Yes | No | No |
eng | Yes | No | No |
de-1901-1901 | Yes | No | No |
en-a-bb-a-cc | Yes | No | No |
de-abcde | Yes | Yes | No |
hbs | Yes | Yes | No |
sh | Yes | No | Yes |
en--US | No | No | No |
en-a-x-a-y | No | No | No |
Choosing
- You read tags from elsewhere and pass them on. Create them well-formed. A tag that uses a subtag newer than your registry or your packages shouldn’t stop you reading the file it came in. Check validity later, where it matters, with
IsValid. - You accept tags from people and want to catch mistakes. Load a registry and use validity. It’s the only check that covers every subtag.
- You want to catch mistakes but can’t ship or load a registry. Use the ISO code lists, and accept the gaps above.
The Conformance of a tag records what was checked when it was created, and never changes, because the struct is read-only. IsValid on an existing tag reports a result without changing it. To carry Valid, create the tag with the registry.
var languageTag = Bcp47LanguageTag.Create("en-US");
var isValid = languageTag.IsValid(subtagRegistry); // true
var conformance = languageTag.Conformance; // still LanguageTagConformance.WellFormed