ICU4X in a JS polyfill?

[justingrant]

Would ICU4X ever be appropriate for a JS polyfill? I'm mostly wondering about bundle size. Any idea how many gzip-compressed KB to expect? And what options would maintainers (or developers who want to fork the code) have to control size? I'm mostly thinking about limiting the volume of calendar and/or TZ data included. But also tree-shaking of ICU4X code that's not used on the JS side.

I just did a minified build of @js-temporal/polyfill and it came up as 43K compressed. Adding another 10-20K for better perf and maintainability might be worth it, but past that it's hard to see it being worth it for a client-side-friendly polyfill, esp. in a world (hopefully <18 months away) where evergreen browsers have native Temporal and may not need a polyfill at all.

I suspect that I already know the answer ("no!") but would a 20K-30K compressed download of ICU4X (including minimal locale data and TZ data) ever be realistic?

Assuming the answer is no, I could imagine someone (aka not me!) building a server-only polyfill to support downlevel Node implementations where paying 300KB might be fine in return for orders-of-magnitude speed gains. But that seems hypothetical at this point-- not sure that use case would be worth ICU4X investing in without a real customer and use case.

For context for folks who don't know me, I'm one of the maintainers of @js-temporal/polyfill, which is an intended-for-production-use polyfill that's under development. Other polyfills will certainly be built, and all will likely have similar needs. One of the reasons I'm filing this issue is that other polyfill maintainers are probably wondering the same thing, so centralizing the discussion here may be useful.

[justingrant]

cc @ptomato @sffc @12wrigja

[sffc]

Thanks for your post!

It's a goal of ICU4X to support the time zone and calendar logic required to implement Temporal.

It is also a goal of ICU4X to be a viable choice for running in a web app; this means building a WebAssembly binary optimized for size.

It's hard for me to say right now whether or not the WebAssembly code and data for time zone and calendar logic will fit in a "20K-30K compressed download". Both time zone and calendar logic are in progress and not yet fully implemented; see #533 (time zones) and #534 (calendars). What I can say is that we (myself in particular) are working very hard to get the code and data size down as much as possible. I understand that this is the # 1 blocker for adoption, and we are treating it as such in the design of the code base. Some examples of work we have already done in this area are a wasm size benchmarking tool with a dashboard, which is code size only without data or FFI, and a full-stack example with FFI, currently built on x86-64.

Question for you: what subset of functionality from the following list does the polyfill actually need from ICU4X?

1. Time zone calculations with the tzdb
2. Localized time zone formatting
3. Calendrical calculations
4. Localized date formatting
5. Localized duration formatting
6. Glue code, such as a Temporal.ZonedDateTime-like object directly in ICU4X

Items 1-4 are in scope and scheduled; item 5 is in scope and will get scheduled once Intl.DurationFormat stabilizes; and item 6 is something we may need to consider given that we hope to use ICU4X to implement Temporal in Firefox.

No matter which subset of features you need, ICU4X will be sliceable ("tree-shakeable") to just that subset of functionality.

[justingrant]

what subset of functionality from the following list does the polyfill actually need from ICU4X?

Not sure we can know the answer without knowing the download price tag for each. 😄 We've already built (1), (3), and (6) in the core polyfill code. I'm less familiar with the localization polyfilling (2, 4, 5) so you'd probably know better than me the current status of that work. But for any of them, deciding to replace JS with ICU4X code would be a cost/benefit analysis.

FWIW, the hackiest part of the current polyfill is the non-ISO calendar implementation (3 on your list), which uses (abuses) Intl.DateTimeFormat as well as using per-calendar special-case code and metadata to create the following duct-tape "APIs":

• Converting Calendar => ISO dates is the biggest pain point. The polyfill currently implements this by estimating a conversion, checking the result using DateTimeFormat#formatToParts, and guessing again using a binary-ish search until we get the right date. It's a painful hack.
• Vice-versa conversions (ISO => Calendar) also rely on DateTimeFormat, but all that's required is parsing the string output of formatToParts and working around a few browser bugs. So this isn't a big pain point now, nor is it as big of a perf issue because we only need to make one Intl call for each conversion, unlike conversions in the other direction which can sometimes require 10+ calls.
• Determining months-in-year, days-in-year, and days-in-month for non-ISO calendars. This is implemented on top of the conversion APIs above. "Get the lengths of all the months in a Chinese year" is an example of a particularly expensive operation because it has to make so many guesses to determine month lengths.
• Validating and constraining dates, for example in Hebrew this would be either constraining or rejecting 10 Adar I in a non-leap year. This depends on the months-in-year/days-in-months code above.
• Converting between era-less and era-ful dates and vice versa. To support this, the polyfill includes hard-coded metadata for each ICU calendar about era start/end dates and behavior (e.g. does this era count years backwards or forwards).
• Determining leap years in each non-ISO calendar, which similarly relies on per-calendar bespoke code for Gregorian- or Julian-based calendars like ethiopic, but calendars with more complicated leap rules (e.g. islamic, hebrew, chinese) currently use DateTimeFormat conversions plus bespoke code to determine if it's a leap year.

In the current implementation there's also a lot of code for caching and other tweaks to reduce calls to Intl.DateTimeFormat. A library that only implemented the APIs above would probably let us remove all per-calendar special-case code, plus all our caching and perf-optimizing weirdness.

That said, the hacks above are slow but they work. Replacing them would be great, but if we had to double the size of the polyfill for it then I doubt we'd take that deal. One option could be to do what other libraries do where there's a "skinny" polyfill (e.g. ISO calendar only) and a "fat" polyfill with full non-ISO calendar support. If we did that, then paying 100KB could be reasonable.

One idea: instead of thinking of this as part of the Temporal polyfill, maybe what would be more helpful to the ecosystem would be an "Intl.CalendarInfo" API and accompanying standalone polyfill which could provide the features above downlevel, not just to Temporal but to anyone who needs it? For example, something like this:

type ISODate = { year: number, month: number; day: number };

type CalendarDate = {
  calendar: string;
  year: number;
  eraYear: number | undefined;
  era: string | undefined;
  monthCode: string;
  month: number;
  day: number;
};

type CalendarDateInput = {
  calendar: string;
  day: number
} & ({ monthCode: string; } | { month: number })
  & ({ year: number} | { era: string, eraYear: number });

/** If the input is not a valid date, either constrain to nearest valid date, or throw */
type CalendarInfoOptions = { overflow: 'constrain' | 'reject' };

interface CalendarInfo {
  toISODate (date: CalendarDateInput, CalendarInfoOptions): ISODate;
  fromISODate (date: ISODate, CalendarInfoOptions): CalendarDate;
  monthsInYear(date: CalendarDateInput, CalendarInfoOptions): number;
  daysInYear(date: CalendarDateInput, CalendarInfoOptions): number;
  daysInMonth(date: CalendarDateInput, CalendarInfoOptions): number;
  inLeapYear(date: CalendarDateInput, CalendarInfoOptions): boolean;
  /** If the date is not valid in this calendar, convert it to a nearby valid one ('constrain') or throw ('reject') */
  normalizeDate (date: CalendarDateInput, CalendarInfoOptions): CalendarDate;
}