Mapping between browser Intl and ICU4X

[mosuem]

I am looking into merging the browser Intl object options and the ICU4X options into a single API surface for a new Dart package, starting with the collator as one of the simpler functionalities.

For the collator, the browser supports the extension keys co, kn, and kf, while ICU4X supports ks, ka, kc, kf, kb, and kn. The overlap, and thus the API surface of the Dart package, would then only be kn and kf. Is this correct? I would have assumed ICU4X and the browser would have a similar set of options. Also, the browser supports usage and sensitivity, which don't seem to have a corresponding option in ICU4X.

[Manishearth]

cc @hsivonen

I think we probably do want to support these things, but I don't understand the component well enough to comment on how much work that would be.

[hsivonen]

https://unicode-org.github.io/icu4x/rustdoc/icu_collator/struct.CollatorOptions.html explains why certain BCP47 options aren't supported. It doesn't explain the inclusion criteria for the options that are supported.

In ICU4X, all the options that are supported from locale data are also supported as a matter of principle as part of the CollatorOptions API surface. I'm not particularly comfortable with providing the CollatorOptions surface for backward second level: Browsers support it, but only from locale data. That is, the only way to get it in browsers is to ask for fr-CA. If the authorities for Canadian French decide to make fr-CA to adopt the behavior of en-CA and fr-FR, from the browser perspective, it's OK to go ahead and delete the code for backward second level, but now it would be, in principle, an API break for ICU4X.

"Usage" in browsers simply means special-casing -u-co-search not to work via -u-co but via "usage". Given the API surface in browsers (only full-string matching, no prefix or substring matching), I think it's very, very questionable that usage: "search" is part of the API. ICU4X doesn't have a search API, either.

"Sensitivity" in browsers maps to a combination of strength and case level. ICU4X simply provides more fine-grained control by having separate options for these as well as providing broader options by giving access to the quaternary and identical levels. This is consistent with ICU4C.

When quaternary strength is available explicitly, it makes sense to provide explicit access to ka as opposed to leaving it settable only via data for Thai. (Moving things to the quaternary level only serves to ignore those things when you have access only at most to the tertiary strength.)

[hsivonen]

Filed #4528

[hsivonen]

all the options that are supported from locale data are also supported as a matter of principle as part of the CollatorOptions API

While writing the PR to document this stuff, I noticed that the above isn't true of reordering. We don't have API surface for it, because building the requisite data structure is not trivial. (For reordering implied by locale, the builder is in ICU4C and ICU4X only knows how to consume the reordering data.)