File: leanpass/optim.py
The bias‑corrected moments are computed as self.m[i] / (1 - self.b1 ** self.t) and similarly for v. For large self.t (e.g., >1e5) self.b1 ** self.t underflows to 0, making the denominator exactly 1 and losing the bias correction. While not a crash, it deviates from the algorithm and can affect convergence.
Fix: Compute the denominator using np.power(self.b1, self.t, dtype=np.float64) or keep a running bias_correction1 *= self.b1 and use 1 - bias_correction1. This preserves precision for many steps.
File: leanpass/optim.py
Filed automatically by ai-issue-scan.
File:
leanpass/optim.pyThe bias‑corrected moments are computed as
self.m[i] / (1 - self.b1 ** self.t)and similarly forv. For largeself.t(e.g., >1e5)self.b1 ** self.tunderflows to 0, making the denominator exactly 1 and losing the bias correction. While not a crash, it deviates from the algorithm and can affect convergence.Fix: Compute the denominator using
np.power(self.b1, self.t, dtype=np.float64)or keep a runningbias_correction1 *= self.b1and use1 - bias_correction1. This preserves precision for many steps.File:
leanpass/optim.pyFiled automatically by ai-issue-scan.