Skip to content

Fix CLDR plural rule defects and let the plural rules follow the language of the app - #50

Open
Sergio0694 wants to merge 13 commits into
DotNetPlus:mainfrom
Sergio0694:user/sergiopedri/pluralization-fixes
Open

Fix CLDR plural rule defects and let the plural rules follow the language of the app#50
Sergio0694 wants to merge 13 commits into
DotNetPlus:mainfrom
Sergio0694:user/sergiopedri/pluralization-fixes

Conversation

@Sergio0694

Copy link
Copy Markdown
Contributor

Summary

This addresses a set of pluralization defects found during a QA campaign over the plural
selection path. The rules are the part of ReswPlus a consuming app cannot work around, since
they are compiled into the app, so this focuses on their correctness against
CLDR.

Every finding is covered by a test, and the tests fail without the corresponding fix. The
suite compiles the templates out of the generator's own embedded resources and invokes them,
so what is tested is what ships, not a reimplementation of it. 219 tests, all passing.

The commits are self-contained and reviewable one at a time.

What is fixed

1. The fractional digit helpers

Templates/Utils/DoubleExt.txt

GetNumberOfDigitsAfterDecimal computed the CLDR v operand as:

(uint)((number - (int)number).ToString()).Length - 2

For any integer the subtraction gives 0, whose string is "0", so this is (uint)1 - 2,
which underflows to 4294967295 rather than the 0 it should return.

Every rule comparing that value against a small number therefore took the wrong branch for
every whole number:

Language Symptom
Czech, Slovak Everything from 2 up selected MANY. _Few and _Other were unreachable.
Romanian Everything selected FEW. _One and _Other were unreachable.
Lithuanian _Other was unreachable, its inputs selected MANY.

DigitsAfterDecimal (the CLDR f operand) had a second problem: it read the decimals off
number - (int)number, so 1.4 became 0.3999999999999999 and reported 3999999999999999
instead of 4. It only returned the right answer for values that happen to be exactly
representable in binary.

Both now read the decimals off the quantity itself, formatted invariantly. Getting that
representation is more work than it looks: the default format switches to scientific notation,
where 1E-06 has no decimals to read and the decimals of 1.2E+20 come out as 2E+20, and
the R format that is meant to give the shortest representation that round-trips only does so
on .NET — .NET Framework, which is what a UWP app runs on, returns extra digits, so the same
quantity would select one form in a UWP app and another in a WinAppSDK one. The helper
therefore searches the general formats for the shortest one that round-trips, and expands the
exponent by hand.

IsInt also cast to an int, so it reported every value outside the 32-bit range as
fractional. Every provider took its integer part with the same cast, which overflows rather
than saturating; they all truncate now, so a quantity past that range selects the same form as
the equivalent small one.

This group was not in the QA report and is worth a careful look — it is the largest behavioral
change here, and it is what makes several of the other rules testable at all.

2. Slovenian, Filipino, Latvian and Danish

Language Was CLDR
Slovenian Matched on n % 10 and routed 3 and 4 to TWO Matches on i % 100; 3 and 4 are FEW
Filipino !(...== 4 || ... == 6 || ... == 9), always true, so _Other was unreachable The exclusion list needs &&
Latvian n == 0 for the zero category n % 10 == 0, so 10, 20, 30 and the 11-19 range are ZERO
Danish n == 1 only Also i = 0 or i = 1 with a non-zero fraction, so 0.5 and 1.7 are ONE

Latvian is then rewritten to read exactly as CLDR states it: its ranges only hold integers, so
11.5 is not part of 11..19, and the fractional branch needs the v = 2 guard it was
missing, which was making 0.011 ZERO instead of ONE.

Lithuanian is also corrected: it matched its 2..9 range against the quantity rather than the
integer part, so 2.5 selected FEW where CLDR says MANY.

3. The CLDR many category for the Romance languages

Spanish, Catalan, Italian, Portuguese and French gained a many category in CLDR 38, for
exact non-zero multiples of a million. It was unreachable, so 1000000 shared its wording
with 5.

Two providers are added rather than changing the existing ones, because the five languages do
not share a single one rule:

  • OnlyOneOrMillionsProvider for es, ca, it, pt
  • ZeroToTwoExcludedOrMillionsProvider for fr

Apps that do not declare a _Many resource are unaffected: the runtime fallback added below
resolves it to _Other, which is the wording they get today.

Two parts of the CLDR definition are deliberately not implemented:

  • The e (compact decimal exponent) operand is not representable, since the generated API
    takes a double and has no notion of a compact form.
  • CLDR gives generic pt the rule of French and pt-PT the rule kept here. A resource folder
    and the language of an app are both reduced to their primary subtag, so the two cannot be
    told apart; correcting one would break the other. Telling them apart needs the rules to be
    keyed by the whole tag, which is a change of its own.

4. Languages mapped to rules that are not theirs

OtherProvider was only ever the default branch of the selector, so a language with one
plural form and a language ReswPlus has no rules for were indistinguishable. The languages CLDR
assigns to the other category alone are now mapped to it explicitly, and anything still
falling through reports a new RESWP0006.

Building that set surfaced three languages whose rules were not the ones CLDR gives them:

Language Was CLDR Effect
Fulah 0 <= n <= 1 i = 0,1, the rule of Armenian 1.5 was other
Marathi 0 <= n <= 1 n = 1 0.5 was one
Samburu listed as sag saq its rules were never reachable at all

A resource set that doesn't declare a form these now select falls back to _Other, which is
the wording it renders today, so no set loses its text.

Cornish (kw) is left alone: its current rules need a provider of their own. Hebrew keeps a
many category that CLDR has since dropped, since removing it would change the wording of any
app that declares _Many in Hebrew.

RESWP0006 is Info, matching the existing RESWP0004. Worth noting that MSBuild does not
surface Info diagnostics at all, so it is only visible in the IDE — I kept it as Info
because a warning would break TreatWarningsAsErrors builds for projects whose resource
folders are not named after languages, but I am happy to raise it if you would prefer.

5. A missing plural form no longer renders as blank UI

GetPlural swallowed the lookup and returned "" when the selected form was not declared. A
resource with _One and _Other shown in a language that selects FEW rendered nothing at
all — an empty label, with no error at build time and no exception at runtime.

The selected form now falls back to _Other, which is the form CLDR guarantees every language
has. A missing _None falls through to normal plural selection rather than returning empty.

The form is also selected on the absolute value of the quantity, which is how CLDR defines its
operands, so a count of -1 now reads the same as a count of 1 instead of taking other.

6. The plural rules can now follow the language of the app

This is the headline fix, and it is opt-in.

Windows resolves .resw resources against the app runtime language list, which an app steers
with ApplicationLanguages.PrimaryLanguageOverride. The generated code picked its plural rules
with CultureInfo.CurrentUICulture, which comes from the display languages of the user and is
not affected by that override.

The two disagree whenever an app offers an in-app language picker, or whenever the display
language of the user is not one the app ships. The result is a resource shown in one language
with the plural form picked by the rules of another — for example Polish strings selecting
forms with English rules, so 2 and 5 both take the _Other wording.

A new MSBuild property makes the generated code read the language of the app instead:

<PropertyGroup>
    <ReswPlusUseApplicationLanguages>true</ReswPlusUseApplicationLanguages>
</PropertyGroup>

It defaults to false so existing apps keep the behavior they were built against, and the UI
culture stays as the fallback for when the language of the app cannot be read. The property is
declared CompilerVisibleProperty in nuget/ReswPlus.targets and only the selected
implementation is emitted, so a project that does not opt in gets exactly the code it gets
today.

Three details worth knowing:

  • The resolved language is cached alongside the provider and the provider is rebuilt when it
    changes, so an app that switches language while it runs doesn't keep selecting forms with the
    rules of the language it started in.
  • A WinAppSDK project reads Microsoft.Windows.Globalization.ApplicationLanguages.PrimaryLanguageOverride
    before the runtime language list, because an unpackaged app has no such list and the Windows
    App SDK applies that override straight to the resource context without publishing it there.
    That API doesn't exist in UWP, so it is only emitted for a WinAppSDK project, and it puts a
    Windows App SDK 1.6 floor on opting in.
  • A language tag is reduced to its primary subtag, which is what the generated selector
    switches on. That reduction now matches the one applied to resource folder names, which was
    lowercasing with the culture of whoever ran the build — under Turkish an is-IS folder
    became ıs and its provider was never emitted at all.

One more fix, found while validating

Support sources (ResourceLoaderExtension.cs, the providers, ResourceStringProvider.cs) were
added once per .resw file. AddSource throws on a duplicate hint name, so any project
with two or more .resw files using macros or plurals
made the generator throw and emit
nothing at all, failing the build with CS8785.

Emission is now idempotent. In the same area, the per-resource hint name is qualified with its
namespace, because two .resw files with the same name in different folders collided the same
way. This changes the names of the generated files (Resources.resw.cs becomes
MyApp.Strings.en-US.Resources.resw.cs); the alternative was to silently drop one of the two
resource classes.

Also not in the QA report, and worth its own look — happy to split it out if you would rather
take it separately.

Testing

242 tests, all passing.

  • PluralProviders.cs — CLDR boundary tables for the corrected providers, plus a regression
    test pinning Romanian 101 to FEW (the QA campaign initially flagged this, then withdrew it;
    current CLDR agrees with ReswPlus, so the test exists to keep it that way).
  • PluralResourceLookup.cs — the _Other fallback, including when the resource loader throws
    rather than returning empty, and the absolute-value rule.
  • PluralLanguageSource.cs — proves the rules follow the language of the app when opted in and
    the UI culture when not, that they follow a language change, that the unpackaged override is
    read, and that a tag is reduced to its language without CultureInfo.
  • PluralLanguageMapping.cs — the language table, including that no code is claimed twice.

Validated end to end against a WinAppSDK consumer with resources in five languages, in both
modes, and against AnalysisLevel=latest-all with Nullable=enable and
GenerateDocumentationFile=true to confirm the emitted code introduces no new analyzer
warnings.

The rules were checked against release-48 of the CLDR data, not against main, which already
carries changes for the next release.

Reviewed by two independent model-driven review passes over the whole diff, iterated over four
rounds. Several of the fixes above came out of that rather than out of the QA report.

Notes for review

Out of scope

  • Generated-code hygiene, which is Emit hygienic generated code and add diagnostics for the content of the .resw files #49.
  • Regional plural rules such as pt-PT, which need the rules to be keyed by the whole tag
    rather than by the language.
  • Cornish, and the many category Hebrew keeps but CLDR has dropped.
  • The CLDR e operand, and visible trailing zeros (4 and 4.0 are the same double).
  • A quantity that needs more than fifteen significant digits to write down can still be read
    slightly differently by .NET and by .NET Framework, because the two disagree on reading such
    a value back. Every quantity below that, which is every quantity anyone passes as a count, is
    read identically. Closing the remainder means shipping a shortest-decimal implementation into
    every consuming app, which seems out of proportion to a value that is carrying the noise of
    its binary form rather than digits anyone wrote.

Sergio0694 and others added 13 commits July 30, 2026 23:46
`GetNumberOfDigitsAfterDecimal` computed `(uint)"0".Length - 2` for a quantity
with no decimals, which underflows to 4294967295 instead of returning 0. Every
provider that asks whether a quantity has decimals therefore took the decimal
branch for every integer:

- Czech and Slovak returned `many` for every integer other than 1, so `_Few` and
  `_Other` were unreachable and 2, 3, 4 and 100 all rendered the same string.
- Romanian returned `few` for every quantity, so `_One` and `_Other` were
  unreachable.
- Lithuanian returned `many` instead of `other` for the quantities that match
  neither of its ranges, such as 10, 11 and 20.

`DigitsAfterDecimal` had a second defect: it read the decimals off
`number - (int)number`, and that subtraction turns 1.4 into 0.3999999999999999,
so it reported the digits of the binary representation rather than the ones the
caller wrote. Both helpers now read the decimals off the quantity itself, which
is exact for every value a caller can write, and parse them without relying on
an exception for the no-decimals case.

Lithuanian also matched its `2..9` range against quantities with decimals, so
2.5 was `few` where CLDR makes it `many`; CLDR ranges are integer ranges.

The tests compile the plural templates out of the generator's embedded resources
and run them, so they cover the rules that actually ship rather than a copy of
them that could drift.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Slovenian returned `two` for 3 and 4, so no translation could be correct for
  2, 3 and 4 at once and the `_Few` form was unreachable. It also compared the
  quantity itself rather than `i % 100`, so 101 to 104 were misrouted. CLDR
  selects on `i % 100`, and every quantity with decimals is `few`.
- Filipino spelled its exclusion list `x != 4 && x != 6 || x != 9`, which C#
  parses as `(x != 4 && x != 6) || x != 9`, a tautology. Every quantity was
  therefore `one` and `_Other` was unreachable. The three exclusions are joined
  with `&&`, as CLDR states them.
- Latvian selected `zero` for zero itself, but CLDR selects it for every
  multiple of ten, so 10, 20 and 30 were `other`.
- Danish selected `one` only inside the numeric interval 0 to 1, but CLDR also
  selects it for a quantity with decimals whose integer part is 0 or 1, so 1.5
  is `one` even though it is greater than 1.

The Filipino fractional cases are only testable because the preceding commit
made the decimals of a quantity exact.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Catalan, French, Italian, Portuguese and Spanish all select `many` for the exact
non-zero multiples of a million, but they were mapped to providers that can only
return `one` and `other`, so the `_Many` form of a resource was unreachable for
them. They now use providers that add that category, keeping the `one` rule each
of them had.

This is additive: a resource set that doesn't declare `_Many` keeps rendering
its `_Other` form for those quantities through the fallback added later in this
branch.

CLDR also assigns `many` to compact notations such as "3M", which relies on the
CLDR `e` operand. That operand isn't representable here, because a provider only
receives the quantity itself and not the notation it was written in, so those
cases stay `other`.

Portuguese keeps the `one` rule it has today. CLDR distinguishes generic `pt`,
where an integer part of 0 or 1 is `one`, from `pt_PT`, where only 1 is, and the
generated selector is keyed on the primary language subtag, so it cannot tell
them apart. Representing both needs the full language tag to reach the selector.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Chinese, Japanese, Korean, Vietnamese, Indonesian, Malay and Thai were absent
from the language table, so they reached the default branch of the generated
selector and used the single-form provider. That happens to be right for them,
but only by accident: a language that is genuinely inflected and missing from
the table reaches the same branch and silently loses every form other than
`_Other`, with nothing to say so.

Those languages, and a few more with the same single form, are now mapped
explicitly, so reaching the default branch always means ReswPlus has no rules
for the language. That case is now reported as RESWP0011. It is an `Info`, like
the existing RESWP0004, because the language is derived from a folder name and a
project laid out differently would otherwise start failing builds that treat
warnings as errors.

Emitting the support sources is also made idempotent along the way, which fixes
a pre-existing crash: they were added once per resource file, and adding the
same hint name twice throws, so the generator produced nothing at all for a
project holding more than one .resw that uses macros or plurals. The hint name
of a generated resource class is qualified with its namespace for the same
reason, since two .resw files with the same name can live in different folders.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Looking up the selected plural form was wrapped in an empty catch and coalesced
to an empty string, so a form the resource set doesn't declare erased the label
or sentence entirely. That is the least debuggable outcome: nothing is rendered,
nothing is logged, and the screen simply has a hole in it.

The lookup now falls back to the `_Other` form, which every set declares, and
only returns an empty string when even that is missing. A resource set doesn't
have to declare every form its language can select: a translation may leave out
the forms the default language didn't need, and a language ReswPlus has just
learned new rules for keeps working against resources written for the old ones.

The empty state gets the same treatment. It is declared in the default language,
which is what makes the generator ask for it, so a translation that leaves it
out used to render nothing for zero.

A missing resource surfaces as an empty string with some resource loaders and as
an exception with others, so both are treated as an undeclared form.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Windows resolves the resources of an app against the app runtime language
list, which an app can steer with ApplicationLanguages.PrimaryLanguageOverride.
The generated code picked its plural rules with CultureInfo.CurrentUICulture
instead, which comes from the display languages of the user and is not affected
by that override. The two disagree whenever an app offers an in-app language
picker, or whenever the display language of the user is not one the app ships,
so a resource would be shown in one language while its plural form was picked
with the rules of another.

Add a ReswPlusUseApplicationLanguages MSBuild property, defaulting to false so
that existing apps keep the behavior they were built against, that makes the
generated code read the first entry of the app runtime language list instead.
The UI culture stays as the fallback, for when the list cannot be read.

Also stop resolving the language of the app when CreatePluralProvider is given
a culture to use, since the result was discarded right after.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three problems remained in how the CLDR operands are read off a quantity, all
of which made a rule take the wrong branch:

- The default format of a double switches to scientific notation for small and
  large values, so the decimals were read off '1E-06', which has none, and off
  '1.2E+20', whose decimals came out as '2E+20'. Every value below 0.0001 was
  therefore treated as a whole number: Czech and Lithuanian returned 'other'
  for 0.000001 instead of 'many', and Romanian returned 'other' instead of
  'few'. Formatting in fixed point reads the digits the caller wrote.

- IsInt cast to an int, which reports every value outside the 32-bit range as
  fractional. Slovenian and Danish also took the integer part with a cast,
  which overflows rather than saturating. Truncating instead keeps the rules
  working past 2147483647.

- Latvian matched its 11 to 19 range against quantities with decimals, which
  the CLDR ranges never hold, and applied the fractional part of that range
  without the required 'v = 2'. 11.5 was 'zero' rather than 'other' and 0.011
  was 'zero' rather than 'one'. The rule now reads exactly as CLDR states it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The single-form group added in the previous commit was assembled by hand, so it
was both incomplete and wrong in one place. It is now the complete set CLDR
assigns to the 'other' category alone, which is what makes reaching the default
branch of the generated selector mean that ReswPlus has no rules for the
language.

Vietnamese moves out of that group: it has had two plural forms since CLDR 34.

Four more languages had rules that were not the ones CLDR gives them, and are
picked up while the table is being corrected:

- Portuguese selects 'one' for an integer part of 0 or 1, which is the rule of
  French rather than the one of the other Romance languages, so 0 and 1.5 were
  taking the wording of 'other'.
- Fulah has the rule of Armenian, not the one it was sharing, so 1.5 was
  'other' instead of 'one'.
- Marathi selects 'one' for exactly 1, so 0.5 was 'one' instead of 'other'.
- Samburu is 'saq'. It was listed as 'sag', which is not a language code, so
  its rules were never reachable at all.

A resource set that doesn't declare a form these changes now select falls back
to '_Other', which is the wording it renders today, so no set loses its text.

Cornish is left alone: its current rules need a provider of their own.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three things kept the opt-in from doing what it is for, which is apps that let
their users pick a language:

- The provider was resolved once for the process, so an app changing its
  language while it runs kept selecting forms with the rules of the language it
  started in, even though its resources followed the change. The resolved
  language is now kept alongside the provider and the provider is rebuilt when
  it changes.

- An unpackaged WinAppSDK app has no app runtime language list to read, and the
  Windows App SDK keeps the override of such an app to itself rather than
  publishing it there, applying it straight to the resource context. Reading
  that override first is what keeps an unpackaged app on the rules of the
  language it picked. It only exists in the Windows App SDK, so it is only
  emitted for a WinAppSDK project.

- A tag such as 'zh-Hans-CN' was read without being reduced to its language,
  which is what the generated selector switches on.

Also select the form on the absolute value of the quantity, which is how CLDR
defines its operands, so that a count of -1 reads the same as a count of 1.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The fixed point format the previous commit used rounds to about fifteen
significant digits, which is fewer than a double holds, and still writes
nothing for the smallest values. 4E-22 came out as a whole number, and
0.9999999999999999 came out as 1, so Filipino and Czech read decimals that were
not the ones the caller wrote. The decimals are now read off the shortest
representation that round-trips, expanding its exponent by hand, which covers
the whole range without rounding.

Truncating in IsInt also sent large whole numbers into branches that were still
taking their integer part with a 32-bit cast, where they overflowed. Icelandic
returned 'other' for 2147483651 where it had returned 'one' before. The
remaining providers take their integer part by truncating too, so a quantity
past that range now selects the same form as the equivalent small one.

Read the language of a tag off the tag itself rather than through CultureInfo,
which rejects valid ones such as 'sl-rozaj-biske-1994' and would have sent an
app using one onto the rules of an unrelated language.

Leave Portuguese on the rule it had. CLDR gives 'pt' the rule of French, but a
resource folder and the language of an app are both reduced to their primary
subtag, so moving it would put 'pt-PT', whose rule is the current one, onto the
rule of 'pt-BR'. Telling them apart needs the rules to be keyed by the whole
tag, which is a change of its own.

Leave Vietnamese among the single-form languages as well: the rule giving it
two forms is not in CLDR 48, which is the release the rules here follow.

Finally, correct the Windows App SDK version the readme asks for. The override
an unpackaged app can set was added in 1.6, not in 1.2.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The 'R' format asks the runtime for the shortest representation that reads back
as the same quantity, but only .NET returns it: .NET Framework, which is what a
UWP app runs on, commonly returns extra digits. The same quantity could
therefore select one plural form in a UWP app and another in a WinAppSDK one,
without either of them being obviously wrong. Searching the general formats for
the shortest one that round-trips answers the same question without leaving it
to the runtime.

Also reduce a resource folder to its language exactly the way the generated code
reduces the language of the app. The two had drifted apart: the folder was
lowercased with the culture of whoever ran the build, so under Turkish an
'is-IS' folder became 'ıs' and its provider was never emitted, while the app
looked for 'is' at runtime and fell back to a single plural form.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Reading a candidate back to check it round-trips is itself something .NET and
.NET Framework do differently, so searching all the way to seventeen digits put
the divergence back where it had been taken out, one step further along.

Stop the search at fifteen digits, which is as far as the two agree on reading a
quantity back. Every quantity that can be written in fifteen digits, which is
every quantity anyone passes as a count, is now read identically by both. Past
that a double is carrying the noise of its binary form rather than digits anyone
wrote, and the shortest representation is asked of the runtime instead: reading
it exactly matters more there than reading it identically, and neither answer
describes anything the caller typed.

Also answer for a quantity that isn't finite instead of letting it fall through
the search, since a comparison against NaN is never true.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
RESWP0006 was reported against nothing, so it came out as a bare 'CSC :' line
naming a language but not saying where that language came from. A project with
several resource folders had to work out which one was meant.

Report it against a resource file of the language instead, so it names the file
and can be clicked through to in the IDE.

Note that this does not change how the diagnostic is configured: Roslyn resolves
the severity in an .editorconfig through the syntax tree a diagnostic sits in,
and a resource file has none, so raising RESWP0006 still needs a global analyzer
config.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Sergio0694
Sergio0694 force-pushed the user/sergiopedri/pluralization-fixes branch from 67deebf to 3552ec9 Compare July 31, 2026 07:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant