fix: tolerate data-less HDUs in the database scrape - #1415
Merged
Conversation
The `HDU.hdu` setter assigned `hdu.data` straight into `Array.array`, whose setter dereferences `array.dtype` unguarded. Any HDU without data therefore raised `AttributeError: 'NoneType' object has no attribute 'dtype'` and aborted the sqlite scrape. Data-less HDUs are routine rather than exotic: the first HDU of a multi-extension FITS is conventionally an empty `PrimaryHDU`, and `AggregateFITS` emits exactly that shape (aggregate_fits.py:107). The database path could not ingest the library's own aggregated FITS output. Store a null payload for such HDUs and reconstruct them with `data=None`, so an empty `PrimaryHDU` round-trips losslessly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
HDU.hdusetter assignedhdu.datastraight into the inheritedArray.arraysetter, which dereferencesarray.dtypewithout a null check. Any HDU carrying no data therefore raisedAttributeError: 'NoneType' object has no attribute 'dtype'and aborted the whole sqlite scrape.Data-less HDUs are routine rather than exotic. The first HDU of a multi-extension FITS is conventionally an empty
PrimaryHDU, andAggregateFITSemits exactly that shape —aggregate_fits.py:107, whose own docstring at line 94 reads "The first HDU in each list is an empty PrimaryHDU". The database path could not ingest the library's own aggregated FITS output.Such HDUs now store a null payload and are reconstructed with
data=None, so an emptyPrimaryHDUround-trips losslessly.Found while reproducing a census report against
autofit_workspace_testscripts/profiling/aggregator/profile_database.py, which was filed as a test-mode env-config gap. It is neither test-mode-specific nor env-dependent — the minimal repro isdb.HDU(hdu=fits.PrimaryHDU()).Closes #1413 (library half; the workspace half is queued behind a repo conflict).
API Changes
Behaviour:
HDUround-trips data-less HDUs instead of raisingAttributeError. This only converts a hard crash into the correct result — no previously-working call changes its output.Added:
HDU.has_data, a read-only property distinguishing an HDU with an array payload from one without.See full details below.
Test Plan
test_hdu_without_data— a barefits.PrimaryHDU()round-trips, returningdata is Noneand preserving thePrimaryHDUtypetest_set_fits_with_empty_primary_hdu— the shape the scrape actually meets (empty primary + namedImageHDUextension) survivesset_fits/get_fitsAttributeErroratarray.py:56test_autofit/full suite: 1527 passed, 1 skippedautofit_workspace_testprofiling/aggregator/profile_database.pyruns end-to-end (exit 0); previously the reported crashblack --checkcleanFull API Changes (for automation & release notes)
Added
autofit.database.model.array.HDU.has_data—boolproperty;Truewhen the HDU holds an array payload. A data-less HDU stores no shape, which is how the two are distinguished.Changed Behaviour
autofit.database.model.array.HDU.hdu(setter) — an HDU whose.dataisNonenow records a null payload (_dtype,_shape,bytesleft unset) instead of raisingAttributeErrorfrom the inheritedArray.arraysetter.autofit.database.model.array.HDU.hdu(getter) — reconstructs withdata=Nonewhenhas_dataisFalse, rather than callingnp.frombufferon a null buffer.Fit.set_fits/Aggregator.add_directorynow ingest multi-extension FITS beginning with an emptyPrimaryHDU; previously any such file aborted the scrape.Migration
Generated by the PyAutoLabs agent workflow.