Add u: options namespace

[eemeli]

Closes #452
See also #845

Following the registry separations proposed in #634, this PR adds a new folder spec/registry/ and moves the current default function definitions there as default.md, as well as adding a new u: Unicode Registry as unicode.md.

Initially, this includes only the u:id, u:locale, and u:dir definitions from the design doc.

A new function context definition is added to Function Resolution, to allow for it to be affected by the u: options. This is intended to encapsulate the parts of the formatting context that are made available when calling functions.

Regarding the options, the text is almost directly as in the design doc. For u:locale, I specified that literal values matching the langtag rule from RFC 5646 are always supported, but that other tags and locale definitions MAY also be supported. This is roughly in line with what we have for e.g. :datetime operands, where we define something relatively strict that will work ~everywhere, while allowing other values beyond that definition to also be processed without error.

Note that u:locale and u:dir are ignored when used on markup.

A new test suite unicode.json is added; this also ends up testing our Default Bidi Strategy.

[aphillips]

Good start. Mostly word-smithing suggestions...

[aphillips]

This seems like special pleading on our part. The u: namespace is "just a namespace"?

Perhaps:

Suggested change

	If the resolved mapping of _options_ includes any `u:` options
	Implementations are encouraged to support _options_ defined in
	the Unicode Reserved Namespace (`u:`).

[eemeli]

It's not "just a namespace", because it needs special powers to affect the function context, though.

[mihnita]

I think that the "function context" context distinction is an implementation detail.

In most formatters implementations the locale and the options on how to format are passed to the constructor as parameters, at the same time.

I would rather look at this as "universal function parameters" that might be recognized and honored by several / all functions.

[aphillips]

I think the problem here is that the current set of u: options behaves that way, but we might introduce one that isn't function context affecting in the future? If our intention to is to require that all such options be context-affecting, we should set that as a requirement in the design doc and elsewhere. We might need to contemplate an additional namespace in the future as well, although I can't think of any universal options just at the moment that we wouldn't just put in the default namespace.

Note: I am not disagreeing with doing this. Just making sure we're consistent and clear about it.

[macchiati]

We should be referencing the Unicode BCP 47 locale identifier, not RFC 5646. (In particular, we don't want to force people to support extlang.

[aphillips]

@macchiati Can you be specific? I think the 5646 reference is to the langtag grammar. We can (and should) refer to Unicode Locale for locale stuff.

[macchiati]

Here is a reference. https://cldr-smoke.unicode.org/spec/main/ldml/tr35.html#BCP_47_Language_Tag_Conversion

I've felt for a while that we need to hoist the Unicode BCP 47 locale identifier definition, I think we could do that in this release.

[aphillips]

That's not what I mean. Where in the text does it need to change?

[aphillips]

@macchiati noted:

I think allowing "implementation-defined locale identifiers" is dangerous,

The text doesn't allow that. It allows the list to be formatted in an implementation defined way, e.g. a List<String> or Intl.Locale[]. This might be phrased better to make it clear, though.

I think the are-also formulation is overkill. We can just say "valid CLDR locale identifiers", although that places some burden on implementations that can guarantee langtag but don't have a local implementation of the CLDR rules. Well-formed tags might be acceptable in certain cases.

[eemeli]

Resolved conflicts with the main branch and renamed the files to not refer to a "registry", but to a "namespace".

[aphillips]

Looking good. Some suggestions.

[aphillips]

Very close to ready. Comments are somewhat nitpicky.

[catamorphism]

Should this PR also include a change to registry.md explaining how the :string function uses the value of u:dir? As it is, the tests are the only thing documenting how u:dir gets used, as far as I can see.

[aphillips]

@catamorphism asked:

Should this PR also include a change to registry.md explaining how the :string function uses the value of u:dir? As it is, the tests are the only thing documenting how u:dir gets used, as far as I can see.

We need to update and rename registry.md soon.

u:dir is used by any expression as part of the default bidi isolation strategy.

[aphillips]

In the 2024-10-21 call we merged this so that it is visible in the spec. Open issues against the results if iteration is needed.