Adds a cap on the maximum gain for added noise#1
Open
Arne Nix (ArneNx) wants to merge 5 commits into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an opt-in global cap (including env var support) for maximum gain during SNR-based audio mixing to prevent extreme amplification of near-silent noise segments that can corrupt training data.
Changes:
- Introduces
get_audio_mix_max_gain_db()/set_audio_mix_max_gain_db()withLHOTSE_AUDIO_MIX_MAX_GAIN_DBsupport. - Applies the configured max-gain cap inside
AudioMixerby clamping the computed SNR gain. - Exposes the new API from
lhotse.audioand the package root.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| lhotse/audio/utils.py | Adds global getter/setter and env var lookup for max mix gain (dB). |
| lhotse/audio/mixer.py | Converts max gain dB to linear factor and clamps computed mixing gain. |
| lhotse/audio/init.py | Re-exports the new getter/setter from the audio package. |
| lhotse/init.py | Re-exports the new getter/setter from the top-level package. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
34a404b to
da8c2db
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
set_audio_mix_max_gain_db()/get_audio_mix_max_gain_db()API (+ env varLHOTSE_AUDIO_MIX_MAX_GAIN_DB) to cap the amplitude gain applied during SNR-basedaudio mixing in
AudioMixerproduces high-amplitude artifacts that corrupt training data
Motivation
When using
CutSet.mix()for noise augmentation (e.g. with MUSAN), the gain applied tothe noise signal is computed as:
If a noise clip — or a random offset into one (
random_mix_offset=True) — hasnear-zero energy, the gain approaches infinity. For example, with
noise_energy ≈ 1e-12, the gain can reach~1e6, turning a near-silent clip into ahigh-amplitude artifact that completely drowns out the speech signal. The mixed audio is
then paired with the original transcript during ASR training, teaching the model to
hallucinate text from noise.
NeMo's native
NoisePerturbationhas an explicitmax_gain_dbcap (default 300 dB) forthis reason, but Lhotse's
AudioMixerhad none.API
Follows the same pattern as
set_audio_duration_mismatch_tolerance():