Optimize intensity matching#236
Open
minnerbe wants to merge 9 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR slightly shifts the level of caching from raw tiles to downscaled and filtered or even fully rendered tiles. The first one is a zero-tradeoff improvement: instead of the full-scale tile + mask, the downsampled versions are stored, reducing per-tile effort and cache pressure. Therefore, it is the new default. The second one trades memory for computational resources: the full rendered bounding box is stored, making the first render somewhat expensive, and the following ones essentially free. It should therefore be used cautiously (especially for ic3d processes, where the full tile needs to be rendered anyway).
@trautmane This introduces a hand-rolled cache that was modeled after
ImageProcessorCache. If you think it's worth doing that, I'm happy to try to consolidate all the different caching mechanisms. Also, imglib2 images are used instead of ImageJ image processors to enable zero-copy views for the fully-cached implementation.Previous state
IntensityMatcherrendered both tiles of every overlapping pair from scratch, on every call, viaRender.render(...)into ImageJFloatProcessor/ColorProcessorrasters sized to just the overlap box. The only cache in the path was a sharedImageProcessorCacheof raw, full-resolution source images (pre-filter, pre-downsample, pre-mesh) — so filtering, downsampling, and mesh-mapping were all redone for every pair a tile appears in, even though a tile typically overlaps many same-layer and cross-layer neighbors.Two options now
Rendering is factored into
TileRenderer, split into two independent halves:loadSource— pair-independent: load the full-res source, apply the tile's filter, downsample to its mipmap level →DownsampledSource.renderBox— pair-dependent: transform the downsampled source into the requested world-space box (using a mesh transform) →RenderedTile(image / weight / coefficient-label rasters).Two strategies decide what to cache between calls, selected by the new
--cacheRenderedTilesflag:TightBoxTileRenderer(default)CachedTileRenderer(--cacheRenderedTiles)DownsampledSourceper tileRenderedTile(whole bounding box) per tileViews.interval) of the cached renderBoth share a small generic LRU,
TileCache<T>, keyed by tile spec and weight-bounded in kilobytes by the existing--maxPixelCacheGb(adapted fromImageProcessorCache's eviction logic). As before, same-layer and cross-layer matching share one renderer/cache when their render scales are equal.New / modified classes
New:
TileRenderer— abstract base defining theloadSource/renderBoxsplit and the shared mesh-mapping logic.TightBoxTileRenderer— caches only the downsampled source; re-meshes each overlap box.CachedTileRenderer— caches the full per-tile render; serves overlaps as zero-copy crops.TileCache<T>— generic per-tile-id, kilobyte-bounded LRU cache used by both renderer strategies.DownsampledSource— the cacheable pair-independent render result (filtered/downsampled image, mask, mipmap level).RenderedTile— the image/weight/coefficient rasters as imglib2 views, replacing the previous raw ImageJ processors.Modified:
IntensityMatcher— no longer renders directly; picks a same-/cross-layerTileRendererand iterates the resulting imglib2 rasters in lockstep instead of indexing ImageJ processors by hand.AffineIntensityCorrectionBlockWorker— replaced the single sharedImageProcessorCachewith a kilobyte budget passed down toIntensityMatcher, which now owns its own per-scaleTileCacheinstances.FIBSEMIntensityCorrectionParameters— carries the newcacheRenderedTilesflag through to the matcher.AlgorithmicIntensityAdjustParameters— adds the--cacheRenderedTilesCLI flag.Validation and benchmarking
The three methods give exactly the same results, see results obtained with the:
--cacheRenderedTilesflag enabledRuntimes with the processes can be seen below, where 10 layers of the w61_s109 multi-sem stack were corrected with ic3d. Matching times were recorded, clustered into two clusters (roughly cached and uncached), and the median of both clusters is shown, as well as overall runtime and cache miss overhead. The first image shows the situation with an overly large cache (100GB), the second with a somewhat tight cache (1GB). It looks like 10GB should be sufficient for multi-sem stacks for all three methods. In the case where the cache is sufficiently large, the new methods speed up the process by ~3x and ~5x, respectively.
FYI @StephanPreibisch