Fix CLDR plural rule defects and let the plural rules follow the language of the app - #50
Open
Sergio0694 wants to merge 13 commits into
Open
Fix CLDR plural rule defects and let the plural rules follow the language of the app#50Sergio0694 wants to merge 13 commits into
Sergio0694 wants to merge 13 commits into
Conversation
`GetNumberOfDigitsAfterDecimal` computed `(uint)"0".Length - 2` for a quantity with no decimals, which underflows to 4294967295 instead of returning 0. Every provider that asks whether a quantity has decimals therefore took the decimal branch for every integer: - Czech and Slovak returned `many` for every integer other than 1, so `_Few` and `_Other` were unreachable and 2, 3, 4 and 100 all rendered the same string. - Romanian returned `few` for every quantity, so `_One` and `_Other` were unreachable. - Lithuanian returned `many` instead of `other` for the quantities that match neither of its ranges, such as 10, 11 and 20. `DigitsAfterDecimal` had a second defect: it read the decimals off `number - (int)number`, and that subtraction turns 1.4 into 0.3999999999999999, so it reported the digits of the binary representation rather than the ones the caller wrote. Both helpers now read the decimals off the quantity itself, which is exact for every value a caller can write, and parse them without relying on an exception for the no-decimals case. Lithuanian also matched its `2..9` range against quantities with decimals, so 2.5 was `few` where CLDR makes it `many`; CLDR ranges are integer ranges. The tests compile the plural templates out of the generator's embedded resources and run them, so they cover the rules that actually ship rather than a copy of them that could drift. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Slovenian returned `two` for 3 and 4, so no translation could be correct for 2, 3 and 4 at once and the `_Few` form was unreachable. It also compared the quantity itself rather than `i % 100`, so 101 to 104 were misrouted. CLDR selects on `i % 100`, and every quantity with decimals is `few`. - Filipino spelled its exclusion list `x != 4 && x != 6 || x != 9`, which C# parses as `(x != 4 && x != 6) || x != 9`, a tautology. Every quantity was therefore `one` and `_Other` was unreachable. The three exclusions are joined with `&&`, as CLDR states them. - Latvian selected `zero` for zero itself, but CLDR selects it for every multiple of ten, so 10, 20 and 30 were `other`. - Danish selected `one` only inside the numeric interval 0 to 1, but CLDR also selects it for a quantity with decimals whose integer part is 0 or 1, so 1.5 is `one` even though it is greater than 1. The Filipino fractional cases are only testable because the preceding commit made the decimals of a quantity exact. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Catalan, French, Italian, Portuguese and Spanish all select `many` for the exact non-zero multiples of a million, but they were mapped to providers that can only return `one` and `other`, so the `_Many` form of a resource was unreachable for them. They now use providers that add that category, keeping the `one` rule each of them had. This is additive: a resource set that doesn't declare `_Many` keeps rendering its `_Other` form for those quantities through the fallback added later in this branch. CLDR also assigns `many` to compact notations such as "3M", which relies on the CLDR `e` operand. That operand isn't representable here, because a provider only receives the quantity itself and not the notation it was written in, so those cases stay `other`. Portuguese keeps the `one` rule it has today. CLDR distinguishes generic `pt`, where an integer part of 0 or 1 is `one`, from `pt_PT`, where only 1 is, and the generated selector is keyed on the primary language subtag, so it cannot tell them apart. Representing both needs the full language tag to reach the selector. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Chinese, Japanese, Korean, Vietnamese, Indonesian, Malay and Thai were absent from the language table, so they reached the default branch of the generated selector and used the single-form provider. That happens to be right for them, but only by accident: a language that is genuinely inflected and missing from the table reaches the same branch and silently loses every form other than `_Other`, with nothing to say so. Those languages, and a few more with the same single form, are now mapped explicitly, so reaching the default branch always means ReswPlus has no rules for the language. That case is now reported as RESWP0011. It is an `Info`, like the existing RESWP0004, because the language is derived from a folder name and a project laid out differently would otherwise start failing builds that treat warnings as errors. Emitting the support sources is also made idempotent along the way, which fixes a pre-existing crash: they were added once per resource file, and adding the same hint name twice throws, so the generator produced nothing at all for a project holding more than one .resw that uses macros or plurals. The hint name of a generated resource class is qualified with its namespace for the same reason, since two .resw files with the same name can live in different folders. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Looking up the selected plural form was wrapped in an empty catch and coalesced to an empty string, so a form the resource set doesn't declare erased the label or sentence entirely. That is the least debuggable outcome: nothing is rendered, nothing is logged, and the screen simply has a hole in it. The lookup now falls back to the `_Other` form, which every set declares, and only returns an empty string when even that is missing. A resource set doesn't have to declare every form its language can select: a translation may leave out the forms the default language didn't need, and a language ReswPlus has just learned new rules for keeps working against resources written for the old ones. The empty state gets the same treatment. It is declared in the default language, which is what makes the generator ask for it, so a translation that leaves it out used to render nothing for zero. A missing resource surfaces as an empty string with some resource loaders and as an exception with others, so both are treated as an undeclared form. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Windows resolves the resources of an app against the app runtime language list, which an app can steer with ApplicationLanguages.PrimaryLanguageOverride. The generated code picked its plural rules with CultureInfo.CurrentUICulture instead, which comes from the display languages of the user and is not affected by that override. The two disagree whenever an app offers an in-app language picker, or whenever the display language of the user is not one the app ships, so a resource would be shown in one language while its plural form was picked with the rules of another. Add a ReswPlusUseApplicationLanguages MSBuild property, defaulting to false so that existing apps keep the behavior they were built against, that makes the generated code read the first entry of the app runtime language list instead. The UI culture stays as the fallback, for when the list cannot be read. Also stop resolving the language of the app when CreatePluralProvider is given a culture to use, since the result was discarded right after. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three problems remained in how the CLDR operands are read off a quantity, all of which made a rule take the wrong branch: - The default format of a double switches to scientific notation for small and large values, so the decimals were read off '1E-06', which has none, and off '1.2E+20', whose decimals came out as '2E+20'. Every value below 0.0001 was therefore treated as a whole number: Czech and Lithuanian returned 'other' for 0.000001 instead of 'many', and Romanian returned 'other' instead of 'few'. Formatting in fixed point reads the digits the caller wrote. - IsInt cast to an int, which reports every value outside the 32-bit range as fractional. Slovenian and Danish also took the integer part with a cast, which overflows rather than saturating. Truncating instead keeps the rules working past 2147483647. - Latvian matched its 11 to 19 range against quantities with decimals, which the CLDR ranges never hold, and applied the fractional part of that range without the required 'v = 2'. 11.5 was 'zero' rather than 'other' and 0.011 was 'zero' rather than 'one'. The rule now reads exactly as CLDR states it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The single-form group added in the previous commit was assembled by hand, so it was both incomplete and wrong in one place. It is now the complete set CLDR assigns to the 'other' category alone, which is what makes reaching the default branch of the generated selector mean that ReswPlus has no rules for the language. Vietnamese moves out of that group: it has had two plural forms since CLDR 34. Four more languages had rules that were not the ones CLDR gives them, and are picked up while the table is being corrected: - Portuguese selects 'one' for an integer part of 0 or 1, which is the rule of French rather than the one of the other Romance languages, so 0 and 1.5 were taking the wording of 'other'. - Fulah has the rule of Armenian, not the one it was sharing, so 1.5 was 'other' instead of 'one'. - Marathi selects 'one' for exactly 1, so 0.5 was 'one' instead of 'other'. - Samburu is 'saq'. It was listed as 'sag', which is not a language code, so its rules were never reachable at all. A resource set that doesn't declare a form these changes now select falls back to '_Other', which is the wording it renders today, so no set loses its text. Cornish is left alone: its current rules need a provider of their own. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three things kept the opt-in from doing what it is for, which is apps that let their users pick a language: - The provider was resolved once for the process, so an app changing its language while it runs kept selecting forms with the rules of the language it started in, even though its resources followed the change. The resolved language is now kept alongside the provider and the provider is rebuilt when it changes. - An unpackaged WinAppSDK app has no app runtime language list to read, and the Windows App SDK keeps the override of such an app to itself rather than publishing it there, applying it straight to the resource context. Reading that override first is what keeps an unpackaged app on the rules of the language it picked. It only exists in the Windows App SDK, so it is only emitted for a WinAppSDK project. - A tag such as 'zh-Hans-CN' was read without being reduced to its language, which is what the generated selector switches on. Also select the form on the absolute value of the quantity, which is how CLDR defines its operands, so that a count of -1 reads the same as a count of 1. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The fixed point format the previous commit used rounds to about fifteen significant digits, which is fewer than a double holds, and still writes nothing for the smallest values. 4E-22 came out as a whole number, and 0.9999999999999999 came out as 1, so Filipino and Czech read decimals that were not the ones the caller wrote. The decimals are now read off the shortest representation that round-trips, expanding its exponent by hand, which covers the whole range without rounding. Truncating in IsInt also sent large whole numbers into branches that were still taking their integer part with a 32-bit cast, where they overflowed. Icelandic returned 'other' for 2147483651 where it had returned 'one' before. The remaining providers take their integer part by truncating too, so a quantity past that range now selects the same form as the equivalent small one. Read the language of a tag off the tag itself rather than through CultureInfo, which rejects valid ones such as 'sl-rozaj-biske-1994' and would have sent an app using one onto the rules of an unrelated language. Leave Portuguese on the rule it had. CLDR gives 'pt' the rule of French, but a resource folder and the language of an app are both reduced to their primary subtag, so moving it would put 'pt-PT', whose rule is the current one, onto the rule of 'pt-BR'. Telling them apart needs the rules to be keyed by the whole tag, which is a change of its own. Leave Vietnamese among the single-form languages as well: the rule giving it two forms is not in CLDR 48, which is the release the rules here follow. Finally, correct the Windows App SDK version the readme asks for. The override an unpackaged app can set was added in 1.6, not in 1.2. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The 'R' format asks the runtime for the shortest representation that reads back as the same quantity, but only .NET returns it: .NET Framework, which is what a UWP app runs on, commonly returns extra digits. The same quantity could therefore select one plural form in a UWP app and another in a WinAppSDK one, without either of them being obviously wrong. Searching the general formats for the shortest one that round-trips answers the same question without leaving it to the runtime. Also reduce a resource folder to its language exactly the way the generated code reduces the language of the app. The two had drifted apart: the folder was lowercased with the culture of whoever ran the build, so under Turkish an 'is-IS' folder became 'ıs' and its provider was never emitted, while the app looked for 'is' at runtime and fell back to a single plural form. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Reading a candidate back to check it round-trips is itself something .NET and .NET Framework do differently, so searching all the way to seventeen digits put the divergence back where it had been taken out, one step further along. Stop the search at fifteen digits, which is as far as the two agree on reading a quantity back. Every quantity that can be written in fifteen digits, which is every quantity anyone passes as a count, is now read identically by both. Past that a double is carrying the noise of its binary form rather than digits anyone wrote, and the shortest representation is asked of the runtime instead: reading it exactly matters more there than reading it identically, and neither answer describes anything the caller typed. Also answer for a quantity that isn't finite instead of letting it fall through the search, since a comparison against NaN is never true. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
RESWP0006 was reported against nothing, so it came out as a bare 'CSC :' line naming a language but not saying where that language came from. A project with several resource folders had to work out which one was meant. Report it against a resource file of the language instead, so it names the file and can be clicked through to in the IDE. Note that this does not change how the diagnostic is configured: Roslyn resolves the severity in an .editorconfig through the syntax tree a diagnostic sits in, and a resource file has none, so raising RESWP0006 still needs a global analyzer config. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sergio0694
force-pushed
the
user/sergiopedri/pluralization-fixes
branch
from
July 31, 2026 07:16
67deebf to
3552ec9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This addresses a set of pluralization defects found during a QA campaign over the plural
selection path. The rules are the part of ReswPlus a consuming app cannot work around, since
they are compiled into the app, so this focuses on their correctness against
CLDR.
Every finding is covered by a test, and the tests fail without the corresponding fix. The
suite compiles the templates out of the generator's own embedded resources and invokes them,
so what is tested is what ships, not a reimplementation of it. 219 tests, all passing.
The commits are self-contained and reviewable one at a time.
What is fixed
1. The fractional digit helpers
Templates/Utils/DoubleExt.txtGetNumberOfDigitsAfterDecimalcomputed the CLDRvoperand as:For any integer the subtraction gives
0, whose string is"0", so this is(uint)1 - 2,which underflows to 4294967295 rather than the
0it should return.Every rule comparing that value against a small number therefore took the wrong branch for
every whole number:
MANY._Fewand_Otherwere unreachable.FEW._Oneand_Otherwere unreachable._Otherwas unreachable, its inputs selectedMANY.DigitsAfterDecimal(the CLDRfoperand) had a second problem: it read the decimals offnumber - (int)number, so1.4became0.3999999999999999and reported3999999999999999instead of
4. It only returned the right answer for values that happen to be exactlyrepresentable in binary.
Both now read the decimals off the quantity itself, formatted invariantly. Getting that
representation is more work than it looks: the default format switches to scientific notation,
where
1E-06has no decimals to read and the decimals of1.2E+20come out as2E+20, andthe
Rformat that is meant to give the shortest representation that round-trips only does soon .NET — .NET Framework, which is what a UWP app runs on, returns extra digits, so the same
quantity would select one form in a UWP app and another in a WinAppSDK one. The helper
therefore searches the general formats for the shortest one that round-trips, and expands the
exponent by hand.
IsIntalso cast to anint, so it reported every value outside the 32-bit range asfractional. Every provider took its integer part with the same cast, which overflows rather
than saturating; they all truncate now, so a quantity past that range selects the same form as
the equivalent small one.
This group was not in the QA report and is worth a careful look — it is the largest behavioral
change here, and it is what makes several of the other rules testable at all.
2. Slovenian, Filipino, Latvian and Danish
n % 10and routed 3 and 4 toTWOi % 100; 3 and 4 areFEW!(...== 4 || ... == 6 || ... == 9), always true, so_Otherwas unreachable&&n == 0for the zero categoryn % 10 == 0, so 10, 20, 30 and the 11-19 range areZEROn == 1onlyi = 0 or i = 1with a non-zero fraction, so 0.5 and 1.7 areONELatvian is then rewritten to read exactly as CLDR states it: its ranges only hold integers, so
11.5is not part of11..19, and the fractional branch needs thev = 2guard it wasmissing, which was making
0.011ZEROinstead ofONE.Lithuanian is also corrected: it matched its
2..9range against the quantity rather than theinteger part, so 2.5 selected
FEWwhere CLDR saysMANY.3. The CLDR
manycategory for the Romance languagesSpanish, Catalan, Italian, Portuguese and French gained a
manycategory in CLDR 38, forexact non-zero multiples of a million. It was unreachable, so
1000000shared its wordingwith
5.Two providers are added rather than changing the existing ones, because the five languages do
not share a single
onerule:OnlyOneOrMillionsProviderfores,ca,it,ptZeroToTwoExcludedOrMillionsProviderforfrApps that do not declare a
_Manyresource are unaffected: the runtime fallback added belowresolves it to
_Other, which is the wording they get today.Two parts of the CLDR definition are deliberately not implemented:
e(compact decimal exponent) operand is not representable, since the generated APItakes a
doubleand has no notion of a compact form.ptthe rule of French andpt-PTthe rule kept here. A resource folderand the language of an app are both reduced to their primary subtag, so the two cannot be
told apart; correcting one would break the other. Telling them apart needs the rules to be
keyed by the whole tag, which is a change of its own.
4. Languages mapped to rules that are not theirs
OtherProviderwas only ever thedefaultbranch of the selector, so a language with oneplural form and a language ReswPlus has no rules for were indistinguishable. The languages CLDR
assigns to the
othercategory alone are now mapped to it explicitly, and anything stillfalling through reports a new RESWP0006.
Building that set surfaced three languages whose rules were not the ones CLDR gives them:
0 <= n <= 1i = 0,1, the rule of Armenianother0 <= n <= 1n = 1onesagsaqA resource set that doesn't declare a form these now select falls back to
_Other, which isthe wording it renders today, so no set loses its text.
Cornish (
kw) is left alone: its current rules need a provider of their own. Hebrew keeps amanycategory that CLDR has since dropped, since removing it would change the wording of anyapp that declares
_Manyin Hebrew.RESWP0006 is
Info, matching the existing RESWP0004. Worth noting that MSBuild does notsurface
Infodiagnostics at all, so it is only visible in the IDE — I kept it asInfobecause a warning would break
TreatWarningsAsErrorsbuilds for projects whose resourcefolders are not named after languages, but I am happy to raise it if you would prefer.
5. A missing plural form no longer renders as blank UI
GetPluralswallowed the lookup and returned""when the selected form was not declared. Aresource with
_Oneand_Othershown in a language that selectsFEWrendered nothing atall — an empty label, with no error at build time and no exception at runtime.
The selected form now falls back to
_Other, which is the form CLDR guarantees every languagehas. A missing
_Nonefalls through to normal plural selection rather than returning empty.The form is also selected on the absolute value of the quantity, which is how CLDR defines its
operands, so a count of -1 now reads the same as a count of 1 instead of taking
other.6. The plural rules can now follow the language of the app
This is the headline fix, and it is opt-in.
Windows resolves
.reswresources against the app runtime language list, which an app steerswith
ApplicationLanguages.PrimaryLanguageOverride. The generated code picked its plural ruleswith
CultureInfo.CurrentUICulture, which comes from the display languages of the user and isnot affected by that override.
The two disagree whenever an app offers an in-app language picker, or whenever the display
language of the user is not one the app ships. The result is a resource shown in one language
with the plural form picked by the rules of another — for example Polish strings selecting
forms with English rules, so
2and5both take the_Otherwording.A new MSBuild property makes the generated code read the language of the app instead:
It defaults to
falseso existing apps keep the behavior they were built against, and the UIculture stays as the fallback for when the language of the app cannot be read. The property is
declared
CompilerVisiblePropertyinnuget/ReswPlus.targetsand only the selectedimplementation is emitted, so a project that does not opt in gets exactly the code it gets
today.
Three details worth knowing:
changes, so an app that switches language while it runs doesn't keep selecting forms with the
rules of the language it started in.
Microsoft.Windows.Globalization.ApplicationLanguages.PrimaryLanguageOverridebefore the runtime language list, because an unpackaged app has no such list and the Windows
App SDK applies that override straight to the resource context without publishing it there.
That API doesn't exist in UWP, so it is only emitted for a WinAppSDK project, and it puts a
Windows App SDK 1.6 floor on opting in.
switches on. That reduction now matches the one applied to resource folder names, which was
lowercasing with the culture of whoever ran the build — under Turkish an
is-ISfolderbecame
ısand its provider was never emitted at all.One more fix, found while validating
Support sources (
ResourceLoaderExtension.cs, the providers,ResourceStringProvider.cs) wereadded once per
.reswfile.AddSourcethrows on a duplicate hint name, so any projectwith two or more
.reswfiles using macros or plurals made the generator throw and emitnothing at all, failing the build with
CS8785.Emission is now idempotent. In the same area, the per-resource hint name is qualified with its
namespace, because two
.reswfiles with the same name in different folders collided the sameway. This changes the names of the generated files (
Resources.resw.csbecomesMyApp.Strings.en-US.Resources.resw.cs); the alternative was to silently drop one of the tworesource classes.
Also not in the QA report, and worth its own look — happy to split it out if you would rather
take it separately.
Testing
242 tests, all passing.
PluralProviders.cs— CLDR boundary tables for the corrected providers, plus a regressiontest pinning Romanian 101 to
FEW(the QA campaign initially flagged this, then withdrew it;current CLDR agrees with ReswPlus, so the test exists to keep it that way).
PluralResourceLookup.cs— the_Otherfallback, including when the resource loader throwsrather than returning empty, and the absolute-value rule.
PluralLanguageSource.cs— proves the rules follow the language of the app when opted in andthe UI culture when not, that they follow a language change, that the unpackaged override is
read, and that a tag is reduced to its language without
CultureInfo.PluralLanguageMapping.cs— the language table, including that no code is claimed twice.Validated end to end against a WinAppSDK consumer with resources in five languages, in both
modes, and against
AnalysisLevel=latest-allwithNullable=enableandGenerateDocumentationFile=trueto confirm the emitted code introduces no new analyzerwarnings.
The rules were checked against
release-48of the CLDR data, not againstmain, which alreadycarries changes for the next release.
Reviewed by two independent model-driven review passes over the whole diff, iterated over four
rounds. Several of the fixes above came out of that rather than out of the QA report.
Notes for review
AnalyzerReleases.Shipped.md.Templates/Plurals/ResourceLoaderExtension.txt, which Emit hygienic generated code and add diagnostics for the content of the.reswfiles #49 also touches, soexpect a conflict there depending on merge order. Happy to rebase whichever lands second.
Out of scope
.reswfiles #49.pt-PT, which need the rules to be keyed by the whole tagrather than by the language.
manycategory Hebrew keeps but CLDR has dropped.eoperand, and visible trailing zeros (4and4.0are the samedouble).slightly differently by .NET and by .NET Framework, because the two disagree on reading such
a value back. Every quantity below that, which is every quantity anyone passes as a count, is
read identically. Closing the remainder means shipping a shortest-decimal implementation into
every consuming app, which seems out of proportion to a value that is carrying the noise of
its binary form rather than digits anyone wrote.