Skip to content

Cache and extend AttributeError member-name suggestions (static members and enum values)#130

Merged
jhonabreul merged 5 commits into
QuantConnect:masterfrom
jhonabreul:bug-cache-attribute-error-suggestions
Jul 7, 2026
Merged

Cache and extend AttributeError member-name suggestions (static members and enum values)#130
jhonabreul merged 5 commits into
QuantConnect:masterfrom
jhonabreul:bug-cache-attribute-error-suggestions

Conversation

@jhonabreul

@jhonabreul jhonabreul commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

What does this implement/fix? Explain your changes.

Two related changes to the AttributeError "Did you mean ...?" suggestions (added in #124 / #129):

1. Cache the suggestions (performance fix).
Building the hint reflects over the managed type's entire member set (GetMembers with FlattenHierarchy), snake_cases every member, and runs a Levenshtein scan. This ran on every miss with no caching — and getattr(obj, name, default) / hasattr trigger it too, because the work happens on the miss path before CPython suppresses the error. A workload that probes the same missing names repeatedly — e.g. a per-bar getattr(self, "_optional", None) on a .NET-derived object — pays the full O(members) reflection + ranking cost on every call.

Two caches in ClassBase fix this:

  • _candidateNameCache (Type -> HashSet<string>): the deduplicated snake_case member names, computed once per type.
  • _suggestionCache ((Type, name) -> string): the fully-built " Did you mean: ...?" hint (empty when there is nothing to suggest), memoized per (type, missing-name).

A repeated miss is now a single dictionary lookup. Suggestions and message text are unchanged by this part.

Measured on a real Lean multi-symbol minute backtest (a Python QCAlgorithm subclass doing many per-bar optional-attribute probes), profiled with dotnet-trace: the suggestion path (GetAttrHook -> reflection + LevenshteinDistance + ToSnakeCase) accounted for ~93% of OnData CPU. With the cache the same backtest goes from not finishing (aborted, >15× slower) to ~93s, on par with the last build without the suggestion feature (~90s), with identical results.

2. Extend suggestions to type-object misses (static members and enum values).
A missing attribute on a reflected type object — a mistyped static member or enum value such as DayOfWeek.Sundey — previously raised the bare CPython type object 'X' has no attribute 'Y' with no hint, because the miss hook was installed on reflected types (which govern their instances), whereas type-object access is governed by the CLR metatype.

This installs the same miss-only __getattr__ hook on the CLR metatype (allowing the redirect when tp_getattro is type_getattro, not only the generic getattr), so a type-object miss is enriched the same way instance misses are. It stays miss-only, so hits, submodule imports and hasattr on type objects are unaffected. BuildMissingAttributeMessage resolves the target from either a CLRObject (instance) or a ClassBase (type object) and uses CPython's type object 'T' wording for the latter.

Suggestions reuse the existing ranking and the snake_case convention Python exposes members under (ToSnakeCaseMemberName): methods become lower_snake, while enum values, consts and static-readonly members become UPPER_SNAKE. Examples of the new messages:

DayOfWeek.Sundey  -> type object 'DayOfWeek' has no attribute 'Sundey' Did you mean: 'SUNDAY'?
Math.PII          -> type object 'Math' has no attribute 'PII' Did you mean: 'PI', 'min', 'pow', 'sin'?
String.Empy       -> type object 'String' has no attribute 'Empy' Did you mean: 'EMPTY', 'copy'?
Math.Sqrtt        -> type object 'Math' has no attribute 'Sqrtt' Did you mean: 'sqrt'?

Every suggested name resolves (DayOfWeek.SUNDAY, String.EMPTY, Math.sqrt, ...). Instance-attribute suggestions are unchanged (obj.lenght -> 'length').

Does this close any currently open issues?

No pythonnet issue is open. Part 1 fixes the backtest performance regression introduced with the AttributeError suggestion feature (#124 / #129); part 2 extends that feature to static/enum members.

Any other comments?

Part 1 is behavior-preserving (identical messages). Part 2 only changes the previously unhinted type-object miss path, adding suggestions and matching CPython's type object 'T' wording.

Bumps the package version to 2.0.58 (<Version>, AssemblyVersion/AssemblyFileVersion and the perf-test baseline reference).

Checklist

  • Make sure to include one or more tests for your change
  • If an enhancement PR, please create docs and at best an example
  • Ensure you have signed the .NET Foundation CLA
  • Add yourself to AUTHORS
  • Updated the CHANGELOG

Building the "Did you mean ...?" hint for a missing attribute reflected over the
managed type's full member set (GetMembers with FlattenHierarchy), snake_cased
every member, and ran a Levenshtein scan -- on every miss, with no caching.
getattr(obj, name, default) and hasattr trigger it too, since the work happens on
the miss path before CPython suppresses the error. A workload that probes the
same missing names repeatedly (e.g. a per-bar getattr(self, "_optional", None) on
a .NET-derived object) therefore paid the full O(members) reflection + ranking
cost on every access.

Add two caches in ClassBase:
- _candidateNameCache (Type -> string[]): the reflected, deduplicated snake_case
  member names, computed once per type.
- _suggestionCache ((Type, name) -> string[]): the ranked suggestion list,
  memoized so repeated misses of the same name are a dictionary lookup.

Suggestions and error messages are unchanged; only the repeated computation is
removed. On a real Lean multi-symbol minute backtest the suggestion path
dominated ~93% of OnData CPU; with this change the backtest goes from not
finishing (aborted, >15x slower) to ~93s, on par with the last build without the
suggestion feature (~90s), with identical results.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
jhonabreul and others added 2 commits July 7, 2026 09:17
Store the fully-built " Did you mean: ...?" hint (empty when there is nothing
to suggest) in _suggestionCache rather than the ranked string[]. The hint is
now assembled once inside ComputeSimilarMemberNames and memoized per
(type, missing-name); GetSuggestionHint just returns the cached string and
appends it, dropping the per-miss Count check, Select and string.Join.

Behavior and message text are unchanged; a repeated miss is now a single
dictionary lookup returning the ready-made hint.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ollection

Move the _candidateNameCache and _suggestionCache declarations up to the class
field block with the other fields. In GetCandidateMemberNames, collect the
snake_case names into a single HashSet (named names) instead of a HashSet plus a
List, and return the set directly; deduplication and storage are the same
collection. Candidate iteration order no longer matters -- suggestions are
ordered by edit distance and then by name.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…sses

A missing attribute on a reflected type object -- a mistyped static member or
enum value such as DayOfWeek.Sundey -- previously raised the bare CPython
"type object 'X' has no attribute 'Y'" with no hint, because the miss hook was
installed on reflected types (governing their instances) but type-object access
is governed by the CLR metatype.

Install the same miss-only __getattr__ hook on the CLR metatype (allowing the
redirect when tp_getattro is type_getattro, not just the generic getattr), so a
type-object miss is enriched the same way instance misses are. BuildMissing
AttributeMessage now resolves the target from either a CLRObject (instance) or a
ClassBase (type object) and uses CPython's "type object 'T'" wording for the
latter.

Suggestions reuse the existing ranking/cache and the snake_case convention
Python exposes members under (ToSnakeCaseMemberName): methods become lower_snake
while enum values, consts and static-readonly members become UPPER_SNAKE, e.g.
DayOfWeek.Sundey -> "Did you mean: 'SUNDAY'?", Math.PII -> 'PI', String.Empy ->
'EMPTY'. All suggested names resolve. Hits, imports and hasattr on type objects
are unaffected (the hook is miss-only).

Adds tests for enum, static const, static-readonly field and static method
misses, the no-similar case and hasattr, in test_enum.py and test_class.py.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jhonabreul jhonabreul changed the title Cache AttributeError member-name suggestions Cache and extend AttributeError member-name suggestions (static members and enum values) Jul 7, 2026
Bump the package <Version>, AssemblyVersion/AssemblyFileVersion and the perf-test
baseline reference to 2.0.58 for the AttributeError suggestion caching and the
static/enum suggestion extension in this PR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jhonabreul jhonabreul merged commit 65b70f3 into QuantConnect:master Jul 7, 2026
4 checks passed
@jhonabreul jhonabreul deleted the bug-cache-attribute-error-suggestions branch July 7, 2026 14:57
@jhonabreul jhonabreul restored the bug-cache-attribute-error-suggestions branch July 7, 2026 15:05
@jhonabreul jhonabreul deleted the bug-cache-attribute-error-suggestions branch July 7, 2026 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants