Skip to content

pdf loses the tail in several distributions that exponentiate before dividing #453

Description

@teddytennant

While fixing #452 I checked whether other distributions share the shape that caused it. Several do.

The pattern is computing pdf as exp(...) / normalizer in value space. The numerator underflows or overflows before the density itself does, and dividing afterwards cannot recover it. Measured on main at 128c9ae:

Normal(0,1e-300).pdf(5e-299)        = 0        true = 5.40515e-244
Laplace(0,1e-300).pdf(1e-297)       = 0        true = 2.53798e-135
Exp(1e300).pdf(1e-297)              = 0        true = 5.07596e-135
Chi(160).pdf(45.0)                  = 0        true = 2.5387e-318
Levy(0,2e-297).pdf(1e-300)          = NaN
Gamma(150,1e10).pdf(1e-8)           = NaN
InverseGamma(150,1e-10).pdf(1e-12)  = NaN
Pareto(1e-200,5).pdf(1e-100)        = NaN
Gumbel(0,1).pdf(-1000.0)            = NaN

The three true values above are exact arithmetic at 120 digits. A density is never NaN, so those five rows are wrong whatever the reference value is.

These are not all one fix:

  • Laplace and InverseGamma define ln_pdf as self.pdf(x).ln(), so there is no correct log-space form to delegate to. Both need one written, the same way fix: compute FisherSnedecor pdf in log space for large freedoms #448 did for FisherSnedecor.
  • Chi and Gamma already branch to ln_pdf(x).exp() above a shape threshold (160 and 160.0). The threshold guards the shape parameter, but the overflow here comes from rate.powf(shape) and x.powf(freedom - 1.0), which blow up independently of it. Those thresholds want re-tuning rather than a new branch.
  • Gumbel fails through a double exponential overflowing to NaN, which is a cousin of this rather than the same thing.
  • Normal is probably the one worth doing first. It is the most used distribution here, its pdf_unchecked is the same expression as the LogNormal one, its ln_pdf_unchecked is already correct, and StudentsT::pdf delegates to it for freedom >= 1e8.

I did not roll any of these into #454. That would turn a one-file fix into a rewrite of a third of the distribution module. Happy to send them as separate PRs if you want them, and happy to be told some are not worth changing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions