Skip to content

Improve validate() performance for large arrays of structures - #139

Merged
olivhoenen merged 1 commit into
iterorganization:developfrom
maarten-ic:performance/validate
Jul 27, 2026
Merged

Improve validate() performance for large arrays of structures#139
olivhoenen merged 1 commit into
iterorganization:developfrom
maarten-ic:performance/validate

Conversation

@maarten-ic

Copy link
Copy Markdown
Collaborator

Prevent an expensive _path calculation when it is not required.


Speeds up ids.validate() by a factor 5 on SDCC for the example below:

import timeit
import imas
from imas.ids_defs import CLOSEST_INTERP

uri = "imas:hdf5?path=/home/ITER/tribolp/public/imasdb/interp/105027/from_predictMagnetics_5kHz"
with imas.DBEntry(uri, "r") as entry:
    mag = entry.get_slice("magnetics", 0, CLOSEST_INTERP)

print(timeit.repeat("mag.validate()", number=10, globals=globals()))

The changes in this PR reduce the time for a single validate() from 370 ms to 66 ms on an SDCC login node with toolchain 2023b.

Prevent an expensive `_path` calculation when it is not required.
@maarten-ic maarten-ic changed the title Improve performance for large arrays of structures Improve validate() performance for large arrays of structures Jul 23, 2026

@olivhoenen olivhoenen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvement, thanks!

@olivhoenen
olivhoenen merged commit a4f5a4b into iterorganization:develop Jul 27, 2026
9 checks passed
@maarten-ic
maarten-ic deleted the performance/validate branch July 27, 2026 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants