Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update from main #914

Merged
merged 49 commits into from
Oct 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
2d28dc8
Create notes-2024-08-19.md
aphillips Aug 19, 2024
47c1062
Accept attributes design & remove spec note (#845)
eemeli Aug 20, 2024
92d8940
Update selection-declaration design doc based on mtg / issue discussi…
echeran Aug 26, 2024
b4fbf04
Add tests for pattern selection (#863)
eemeli Aug 26, 2024
e64c987
Add Duplicate Variant to table in test/README.md (#861)
eemeli Aug 26, 2024
343ef56
Add new selection-declaration alternative: Require annotation of sele…
eemeli Aug 26, 2024
42c1f97
Update the stability policy (#834)
aphillips Aug 26, 2024
14fd7b5
Refine error handling text (#816)
eemeli Aug 26, 2024
abff42c
Create notes-2024-08-26.md
aphillips Aug 26, 2024
a35e524
Select "Match on variables instead of expressions" for selection-decl…
eemeli Aug 28, 2024
42620a1
Fix "Allow immutable input declarative selectors" example (#874)
bearfriend Aug 29, 2024
d4fb35b
Update README.md (#875)
srl295 Sep 2, 2024
6657b80
[DESIGN] Update bidi design document to show proposed design (#871)
aphillips Sep 2, 2024
3eb6c86
Miscellaneous test fixes (#862)
eemeli Sep 3, 2024
aa8d293
Update contributing and joining info (#876)
aphillips Sep 5, 2024
058a6d6
Clarify error & fallback handling (#879)
eemeli Sep 9, 2024
749a502
Clarify rule selection (#878)
aphillips Sep 9, 2024
45ca7e3
[DESIGN] Maintaining the Standard, Optional and Unicode Namespace Fun…
aphillips Sep 9, 2024
a57e09e
Create notes-2024-09-09.md
aphillips Sep 9, 2024
01f2f40
Fix a typo in an example (#880)
aphillips Sep 9, 2024
40db4e0
Remove forward-compatibility promise and all reserved & private synta…
eemeli Sep 10, 2024
7dad8fc
Match on variables instead of expressions (#877)
eemeli Sep 10, 2024
80bec52
Create notes-2024-09-10.md
aphillips Sep 10, 2024
c3d06cf
Add bidi support and address UAX31/UTS55 requirements (#884)
aphillips Sep 16, 2024
a49b9dc
Specify `bad-option` for bad digit size option values (#882)
aphillips Sep 16, 2024
95ec6d5
Create notes-2024-09-16.md
aphillips Sep 16, 2024
6f5ad39
Address name and literal equality (#885)
aphillips Sep 17, 2024
57740d8
Update list of normative changes during the LDML45 period (#890)
aphillips Sep 18, 2024
6c3704f
Fix typos in data-model-errors tests (#892)
aphillips Sep 19, 2024
daf7e0b
Update note on exact numeric match for v46 (#891)
aphillips Sep 19, 2024
869005f
Fix attribute value to be literal (#894)
aphillips Sep 19, 2024
22707c7
Create notes-2024-09-30.md
aphillips Sep 30, 2024
662c97f
Add Resolved Values and Function Handler sections to formatting (#728)
eemeli Oct 7, 2024
b729d38
Define function composition for :number and :integer values (#823)
eemeli Oct 7, 2024
e9fc136
Create notes-2024-10-07.md
aphillips Oct 7, 2024
a93fcc1
Apply NFC normalization during :string key comparison (#905)
eemeli Oct 14, 2024
9e595f6
Add tests for changes due to bidi/whitespace (#902)
catamorphism Oct 14, 2024
2334e16
Define function composition for date/time values (#814)
eemeli Oct 14, 2024
f6b724b
DESIGN: Add alternative designs to the design doc on function composi…
catamorphism Oct 14, 2024
335b5b5
Update function-composition-part-1.md status
aphillips Oct 14, 2024
240fe22
Create notes-2024-10-14.md
aphillips Oct 14, 2024
4ea56e4
Add test for :integer and :number composition (#907)
catamorphism Oct 15, 2024
69b94e8
Fix `:integer` option `useGrouping` values (#912)
aphillips Oct 21, 2024
218c61c
Drop syntax note on additional bidi changes (#910)
eemeli Oct 21, 2024
efde4ff
Add tests for changes due to #885 (name/literal equality) (#904)
catamorphism Oct 21, 2024
7f9a4de
Add u: options namespace (#846)
eemeli Oct 21, 2024
1291773
Define function composition for :string values (#798)
eemeli Oct 21, 2024
0eb6109
Drop data model request for feedback on "name" (#909)
eemeli Oct 21, 2024
1baac19
Allow surrogates in content, issue #895 (#906)
mihnita Oct 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 1 addition & 8 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,6 @@
# Contributing to this project

## Joining the Working Group

We are looking for participation from software developers, localization engineers and others with experience
in Internationalization (I18N) and Localization (L10N). If you wish to contribute to this work, please review
the information on the Contributor License Agreement below. In addition, you should:

1. Apply to join our [mailing list](https://groups.google.com/a/chromium.org/forum/#!forum/message-format-wg)
2. Watch this repository (use the "Watch" button in the upper right corner)
To join this Working Group, please read the information in the [README.md](./README.md) as well as the Contributor License Agreement information just below:

<!-- boilerplate follows - do not edit -->

Expand Down
47 changes: 38 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,8 @@ Functions can optionally take _options_:
Messages can use a _selector_ to choose between different _variants_,
which correspond to the grammatical (or other) requirements of the language:

.match {$count :integer}
.input {$count :integer}
.match $count
0 {{You have no notifications.}}
one {{You have {$count} notification.}}
* {{You have {$count} notifications.}}
Expand Down Expand Up @@ -105,6 +106,23 @@ The `main` branch of this repository contains changes implemented since the tech
Implementers should be aware of the following normative changes during the tech preview period.
See the [commit history](https://github.com/unicode-org/message-format-wg/commits)
after 2024-04-13 for a list of all commits (including non-normative changes).
- [#885](https://github.com/unicode-org/message-format-wg/issues/885) Address equality of `name` and `literal` values, including requiring keys to use NFC
- [#884](https://github.com/unicode-org/message-format-wg/issues/884) Add support for bidirectional isolates and strong marks in syntax and address UAX31/UTS55 requirements
- [#883](https://github.com/unicode-org/message-format-wg/issues/883) Remove forward-compatibility promise and all reserved/private syntax.
- [#882](https://github.com/unicode-org/message-format-wg/issues/882) Specify `bad-option` error for bad digit size options in `:number` and `:integer` functions
- [#878](https://github.com/unicode-org/message-format-wg/issues/878) Clarify "rule" selection in `:number` and `:integer` functions
- [#877](https://github.com/unicode-org/message-format-wg/issues/877) Match on variables instead of expressions.
- [#854](https://github.com/unicode-org/message-format-wg/issues/854) Allow whitespace at complex message start
- [#853](https://github.com/unicode-org/message-format-wg/issues/853) Add a "duplicate-variant" error
- [#845](https://github.com/unicode-org/message-format-wg/issues/845) Define "attributes" feature
- [#834](https://github.com/unicode-org/message-format-wg/issues/834) Modify the stability policy (not currently in effect due to Tech Preview)
- [#816](https://github.com/unicode-org/message-format-wg/issues/816) Refine error handling
- [#815](https://github.com/unicode-org/message-format-wg/issues/815) Removed machine-readable function registry as a deliverable
- [#813](https://github.com/unicode-org/message-format-wg/issues/813) Change default of `:date` and `:datetime` date formatting from `short` to `medium`
- [#812](https://github.com/unicode-org/message-format-wg/issues/812) Allow trailing whitespace for complex messages
- [#793](https://github.com/unicode-org/message-format-wg/issues/793) Recommend the use of escapes only when necessary
- [#775](https://github.com/unicode-org/message-format-wg/issues/775) Add formal definitions for variable, external variable, and local variable
- [#774](https://github.com/unicode-org/message-format-wg/issues/774) Refactor errors, adding Message Function Errors
- [#771](https://github.com/unicode-org/message-format-wg/issues/771) Remove inappropriate normative statement from errors.md
- [#767](https://github.com/unicode-org/message-format-wg/issues/767) Add a test schema and
[#778](https://github.com/unicode-org/message-format-wg/issues/778) validate tests against it
Expand All @@ -113,7 +131,9 @@ after 2024-04-13 for a list of all commits (including non-normative changes).
- [#769](https://github.com/unicode-org/message-format-wg/issues/769) Add `:test:function`,
`:test:select` and `:test:format` functions for implementation testing
- [#743](https://github.com/unicode-org/message-format-wg/issues/743) Collapse all escape sequence rules into one (affects the ABNF)
- _more to be added as they are merged_

In addition to the above, the test suite is significantly modified and updated.


## Implementations

Expand All @@ -137,18 +157,27 @@ We invite feedback about the current syntax draft, as well as the real-life use-
- General questions and thoughts → [post a discussion thread](https://github.com/unicode-org/message-format-wg/discussions).
- Actionable feedback (bugs, feature requests) → [file a new issue](https://github.com/unicode-org/message-format-wg/issues).

## Participation
## Participation / Joining the Working Group

We are looking for participation from software developers, localization engineers and others with experience
in Internationalization (I18N) and Localization (L10N).
If you wish to contribute to this work, please review the information about the Contributor License Agreement below.

To join in:
To follow this work:
1. Apply to join our [mailing list](https://groups.google.com/a/chromium.org/forum/#!forum/message-format-wg)
2. Watch this repository (use the "Watch" button in the upper right corner)

1. Review [CONTRIBUTING.md](./CONTRIBUTING.md)
2. Apply to join our [mailing list](https://groups.google.com/a/chromium.org/forum/#!forum/message-format-wg)
3. Watch this repository (use the "Watch" button in the upper right corner)
To contribute to this work, in addition to the above:
1. Each individual MUST have a copy of the CLA on file. See below.
2. Individuals who are employees of Unicode Member organizations SHOULD contact their member representative.
Individuals who are not employees of Unicode Member organizations MUST contact the chair to request Invited Expert status.
Employees of Unicode Member organizations MAY also apply for Invited Expert status,
subject to approval from their member representative.

### Copyright & Licenses

Copyright © 2019-2024 Unicode, Inc. Unicode and the Unicode Logo are registered trademarks of Unicode, Inc. in the United States and other countries.

The project is released under [LICENSE](./LICENSE).

A CLA is required to contribute to this project - please refer to the [CONTRIBUTING.md](./CONTRIBUTING.md) file (or start a Pull Request) for more information.

The contents of this repository are governed by the Unicode [Terms of Use](https://www.unicode.org/copyright.html) and are released under [LICENSE](./LICENSE).
71 changes: 49 additions & 22 deletions exploration/bidi-usability.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,39 @@ Not allowing these to mix could produce annoying parse errors.

_Describe the proposed solution. Consider syntax, formatting, errors, registry, tooling, interchange._

I propose adopting a hybrid approach in which we permit "super-loose isolation".
This allows user to include isolates and strongly directional characters into the whitespace
portions of the syntax in order to make messages appear correctly.

The second part of the hybrid approach would be to recommend ("SHOULD") the "strict isolation"
design for serializers.
(Note that "strict" and "super-loose" use non-identical productions with the name `bidi`.
These serve different purposes and are consistent with strict being narrower with super-loose.)
This syntax is a subset of the super-loose syntax and can be applied selectively to messages that
have RTL sequences or which have problematic display.


## Alternatives Considered

_What other solutions are available?_
_How do they compare against the requirements?_
_What other properties they have?_

### Nothing
We could do nothing.

A likely outcome of doing nothing is that RTL users would insert bidi controls into
_messages_ in an attempt to make the _pattern_ and/or _placeholders_ display correctly.
These controls would become part of the output of the _message_,
showing up inappropriately at runtime.
Because these characters are invisible, users might be very frustrated trying to manage
the results or debug what is wrong with their messages.

By contrast, if users insert too many or the wrong controls using the recommended design,
the _message_ would still be functional and would emit no undesired characters.

### LTR Messages with isolating sequences

The syntax of a _message_ assumes a left-to-right base direction
both for the complete text of the _message_ as well as for each line (paragraph)
contained therein.
Expand Down Expand Up @@ -383,7 +416,7 @@ ns-separator = [bidi] ":"
bidi = [ %x200E-200F / %x061C ]
```

### Open Issues with Proposed Design
**Open Issues**

The ABNF changes found above put isolates and strongly directional marks into specific locations,
such as directly next to `{`/`}`/`{{`/`}}` markers
Expand All @@ -393,32 +426,24 @@ A more permissive design would add the isolates and strongly directional marks t
whitespace in the syntax and depend on users/editors to appropriately pair or position the marks
to get optimal display.

## Alternatives Considered

_What other solutions are available?_
_How do they compare against the requirements?_
_What other properties they have?_

### Nothing
We could do nothing.

A likely outcome of doing nothing is that RTL users would insert bidi controls into
_messages_ in an attempt to make the _pattern_ and/or _placeholders_ display correctly.
These controls would become part of the output of the _message_,
showing up inappropriately at runtime.
Because these characters are invisible, users might be very frustrated trying to manage
the results or debug what is wrong with their messages.

By contrast, if users insert too many or the wrong controls using the recommended design,
the _message_ would still be functional and would emit no undesired characters.

### Super-loose isolation

Add isolates and strongly directional marks to required and optional whitespace in the syntax.
This would permit users to get the effects described by the above design,
as long as they use isolates/marks in a "responsible" way.

(Omitting other changes found in #673)
The exception to this is the namespace separator, used in `identifier`.
This requires the ability to insert isolates or strongly directional marks
between the namespace and name portions, where whitespace is not permitted.
This is the only location in the syntax where such characters might be needed
but whitespace is not at least optional.
This could be defined as:
```abnf
ns-separator = [bidi] ":" [bidi]
```

Here are the other ABNF changes:

```abnf
; strongly directional marks and bidi isolates
Expand Down Expand Up @@ -447,7 +472,7 @@ s = ( SP / HTAB / CR / LF / %x3000 )
### Strict isolation all the time

Apply bidi isolates in a strict way.
The main differences to the proposed solution is:
In this design:
1. The open/close isolate characters are syntactically required to be paired.
This introduces parse errors for unpaired invisible characters,
which could lead to bad user experiences.
Expand All @@ -467,7 +492,7 @@ markup = "{" [s] "#" identifier [bidi] *(s option) *(s attribute) [s] ["
/ "{" [s] "/" identifier [bidi] *(s option) *(s attribute) [s] "}" ; close
/ "{" LRI [s] "/" identifier [bidi] *(s option) *(s attribute) [s] close-isolate "}" ; close
identifier = [(namespace ns-separator)] name
ns-separator = [bidi] ":"
ns-separator = [bidi] ":" [bidi]
bidi = [ %x200E-200F / %x061C ]
```

Expand Down Expand Up @@ -610,6 +635,8 @@ adherence to the stricter grammar.
syntax errors
- Provides a foundation for tools to claim strict conformance and message normalization
as well as guidance to implementers to make them want to adopt it
- Messages are valid while being edited (such as when the open or close isolate has been
inserted but the corresponding opposite isolate hasn't been entered yet)

**Cons**
- Requires additional effort to maintain the grammar
Expand Down
4 changes: 3 additions & 1 deletion exploration/expression-attributes.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Expression Attributes

Status: **Proposed**
Status: **Accepted**

<details>
<summary>Metadata</summary>
Expand All @@ -15,6 +15,8 @@ Status: **Proposed**
<dd><a href="https://github.com/unicode-org/message-format-wg/pull/772">#772</a></dd>
<dd><a href="https://github.com/unicode-org/message-format-wg/pull/780">#780</a></dd>
<dd><a href="https://github.com/unicode-org/message-format-wg/pull/792">#792</a></dd>
<dd><a href="https://github.com/unicode-org/message-format-wg/pull/845">#845</a></dd>
<dd><a href="https://github.com/unicode-org/message-format-wg/pull/846">#846</a></dd>
</dl>
</details>

Expand Down
Loading