2024-09-05 contributor meeting

[rlskoeser]

Goals

• check in on open PRs and questions

Open PRs

• integrate numpy datetime64 for greater range of years #77 : Use numpy datetime64 instead of datetime.date #77 #84
• consolidate pyproject.toml and setup.cfg  #80 : Consolidate setup.cfg and pyproject.toml; drop tox; revamp sphinx docs #88
• Make EDTF parser available as undate formatter; handle 5+ digit years #89 (part of EDTF support - level 0, 1 and parts of level 2 #73)
• support using different formatters for input and output #87 : preliminary str parse and format methods #90
• Enforce types #29 : Configure mypy pre-commit check #91

Discussion / open questions

• Any feedback on the logo I created when I overhauled the docs?
• PR reviewer assignment
• AI code review
• Should we switch pre-commit hook to ruff?
• EDTF
  • Is a season a date or a date interval?
  • proposal for EDTF support checklist (EDTF support - level 0, 1 and parts of level 2 #73) : code has unit tests, but what about a code notebook to demonstrate the working
    support?
  • need to add support to track uncertain + approximate for parts of dates
    • how does this impact calculations based on dates? support fuzzy/approximate dates with configurable fuzziness  #8
      • propose that the approximate fuzziness should be relative to the granularity of the date (days, years, months); what is a good default, and how do we make it easy to modify?
  • how to compare approximate/uncertain dates?
    • python-edtf uses a fixed percentage; using both: double the fuzziness
    • configurable but not too many people use that

Next steps

• Next scheduled meeting: Thursday October 3

[rlskoeser]

PR status

• approved: mypy check, pyproject refactor
  • numpy: good but benchmark before and after; test suite time before/after? install size? lambda usage is probably edge case
    • not worth the headache of making it optional, just document
• EDTF PR review still to do; Cole will handle

discussion

• if a formatter doesn't support part of a date, should throw an exception (e.g. uncertain date to ISO8601)
• season: date or interval? more like a year or a month, a date with known granularity
  • Robert: example, "written in the spring of x year"
  • does EDTF define seasons as month; how does it handle winter?
    • Cole: python-edtf may have a bug with winter, it's just one month
    • Robert: winter is ambiguous!
    • RSK: could check how it's handled in other edtf implementations
  • Julia: maybe it's an interval with a specific type?
  • would be helpful to have example use cases
• notebook for EDTF sounds like a good solution
• how to compare approximate/uncertain dates?
  • python-edtf uses a fixed percentage; using both: double the fuzziness
  • configurable but not too many people use that
  • Robert can look for examples and share some uncertain/approximate dates to help us think through what's needed
• edtf ranges: open-ended = ongoing; unknown end date; not sure we distinguish between these right now (and maybe we don't need to)
• use the parts of edtf that are useful/relevant for us ; document what we do and don't support

AI code review: leave on for now or disable?

• Cole: leave on for now? it found one or two things
• Robert: yes, leave running since we want to talk about; get some more examples of how it behaves

possibility of submitting a paper on this work to CHR missing data special issue - of interest to the group? sounds like yes, worth thinking about

next work

• merge approved PRs
• add support for edtf uncertain/approx flags
• maybe calendars next?
• import S&co date formatting code as first pass at date format, for discussion about the problem at a future meeting
• Robert will add some examples (RSK create issue; check documentation about this)
• Cole share example projects using ruff pre-commit hook

[ColeDCrawford]

EDTF - season ambiguity. Looks like a bug.

>>> import edtf
>>> w = edtf.parse_edtf("1979-24")
>>> w
Season: '1979-24'
>>> w.lower_fuzzy()[:3]
(1979, 12, 1)
>>> w.upper_fuzzy()[:3]
(1979, 12, 31)
>>> f = edtf.parse_edtf("1979-23")
>>> f
Season: '1979-23'
>>> f.lower_fuzzy()
time.struct_time(tm_year=1979, tm_mon=9, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=0, tm_yday=0, tm_isdst=-1)
>>> f.upper_fuzzy()[:3]
(1979, 11, 30)
>>> spring = edtf.parse_edtf("1979-21")
>>> spring.lower_fuzzy()[:3]
(1979, 3, 1)
>>> spring.upper_fuzzy()[:3]
(1979, 5, 31)

SEASON_MONTHS_RANGE = EDTF.get(
    "SEASON_MONTHS_RANGE",
    {
        # season id: [earliest_month, last_month]
        21: [3, 5],
        22: [6, 8],
        23: [9, 11],
        # winter in the northern hemisphere wraps the end of the year, so
        # Winter 2010 could wrap into 2011.
        # For simplicity, we assume it falls at the end of the year, esp since the
        # spec says that sort order goes spring > summer > autumn > winter
        24: [12, 12],
    },
)

uncertain/approximate logic in python-edtf:

MULTIPLIER_IF_UNCERTAIN = EDTF.get("MULTIPLIER_IF_UNCERTAIN", 1.0)
MULTIPLIER_IF_APPROXIMATE = EDTF.get("MULTIPLIER_IF_APPROXIMATE", 1.0)
MULTIPLIER_IF_BOTH = EDTF.get("MULTIPLIER_IF_BOTH", 2.0)