Skip to content

Do not read the day as a two-digit year after the year position - #1358

Merged
serhii73 merged 7 commits into
scrapinghub:masterfrom
serhii73:fix-519-japanese-day-parsing
Jul 29, 2026
Merged

Do not read the day as a two-digit year after the year position#1358
serhii73 merged 7 commits into
scrapinghub:masterfrom
serhii73:fix-519-japanese-day-parsing

Conversation

@serhii73

Copy link
Copy Markdown
Collaborator

Fixes #519 (Close #519 is in the commit message).

Root cause

_parser.parse_number() tries the numeric directives in the resolved DATE_ORDER. For year-first orders (e.g. YMD, which the Japanese locale uses), the %y directive is tried first for every token, so any two-digit number that appears after the month is consumed as a year. In 4月20日 18:10 the 20 became the year 2020, the day was never parsed, and it was silently filled from the current date:

>>> dateparser.parse("4月20日 18:10")  # reported on 2019-04-20
datetime.datetime(2020, 4, 20, 18, 10)  # expected 2019-04-20 18:10

The bug is locale-independent — the same happens for dateparser.parse("4-20 18:10", settings={"DATE_ORDER": "YMD"}).

Fix

If a component that the date order places after the year has already been found, the year position in the date string has already been passed, so a two-digit number cannot be the year anymore. parse_number() now defers the %y directive in that case, reading the number as a two-digit year only if it cannot be any other component (so 4-99 still parses as April 1999, since 99 can be neither month nor day).

Year-last orders (DMY, MDY — the majority of locales) have no components after the year, so their behavior is unchanged by construction. Four-digit years (%Y) are never deferred.

Regression tests

test_two_digit_day_is_not_confused_with_year in tests/test_date_parser.py covers:

  • 4月20日 18:10 and 3月8日 (ja) → day is the day, year comes from the reference date
  • 4-20 18:10 with explicit DATE_ORDER: YMD → same, locale-independent
  • 4-99 (YMD) → still April 1999 (two-digit year as last resort)
  • 2020年4月20日 18:10 and 19年4月20日 → explicit years keep working

No existing tests were modified; the full suite passes (24103 passed, 18 skipped, 1 xfailed) and pre-commit (ruff, ruff-format) is clean.

serhii73 and others added 7 commits June 22, 2026 11:16
- Add month locative/dative forms (lednu, únoru, březnu, dubnu, květnu,
  červnu, červenci, srpnu, říjnu, prosinci) for dates like "v lednu 2023"
- Add July abbreviation "črv"
- Fix "za měsíc/týden/rok" not parsing: add accusative simplification
  patterns (previous patterns only matched instrumental -em forms)
- Add tests covering the new expressions

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…scrapinghub#519)

In year-first date orders (e.g. YMD, used by the Japanese locale),
parse_number tried the %y directive on every two-digit number, so in
"4月20日 18:10" the day 20 was consumed as the year 2020 and the day was
silently filled from the current date.

If a component that the date order places after the year has already
been found, the year position in the date string has already been
passed, so a two-digit number is now read as a year only if it cannot
be any other component (e.g. 99 in "4-99").
@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.08%. Comparing base (315e396) to head (c0e1d4b).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1358      +/-   ##
==========================================
+ Coverage   97.07%   97.08%   +0.01%     
==========================================
  Files         236      236              
  Lines        2975     2987      +12     
==========================================
+ Hits         2888     2900      +12     
  Misses         87       87              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@serhii73
serhii73 merged commit 06ca032 into scrapinghub:master Jul 29, 2026
15 checks passed
@serhii73
serhii73 deleted the fix-519-japanese-day-parsing branch July 29, 2026 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

parse error

2 participants