Fix RSLC valid data mask when L0B starts with a block of invalid data (NSDS-5279) - #344
Open
bhawkins wants to merge 1 commit into
Open
Fix RSLC valid data mask when L0B starts with a block of invalid data (NSDS-5279)#344bhawkins wants to merge 1 commit into
bhawkins wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Recently the raw data products (L0B/RRSD) began using the the
validSamplesSubSwathmetadata to mark the first N pulses of a datatake as invalid in order to reflect the ramp-up behavior of the instrument. It also uses thecalTypemetadata to mark the corresponding loopback cal data as invalid.The RSLC processor (focus.py) propagates the valid data mask forward, accounting for reskew, filter delays, presumming, etc. The L0B valid data mask is complicated by PRF dithering, so currently RSLC looks at the range timing metadata (RD, WD, WL) to figure out when the swath bounds are expected to actually change. Typically those changes can occur no faster than once every 10 seconds. The L0B valid data mask is basically consulted at the start of each block of constant range timing.
When the pulses at the start of a block are marked completely invalid in the L0B, RSLC considers the entire block invalid and emits a log message like
The first consequence is that the valid data mask of the RSLC will mark that entire block of data as invalid.
The bad behavior gets worse when the datatake is short enough that the range timing never changes. That happens whenever the first observation in a datatake is shorter than 10 seconds, and there's a chance that longer ones could be affected as well. In this case, RSLC doesn't find any valid data which leads to an unhandled exception like
Solution
The RLSC needs to better account for fully masked data regions when generating its own mask. Currently this PR accommodates fully masked regions at the beginning of an observation, which probably covers most of the cases of interest. Note that this is also necessary for SSAR data processing.
A more complete solution would accommodate blocks of invalid data that occur anywhere within the observation.