Skip to content

v1.3.6: Cascade reach through special-field references (RedisSet/RedisPriorityQueue of ForeignKey) - #289

Open
yedidyakfir wants to merge 16 commits into
developfrom
gsd/cascade-update-sf-pr
Open

v1.3.6: Cascade reach through special-field references (RedisSet/RedisPriorityQueue of ForeignKey)#289
yedidyakfir wants to merge 16 commits into
developfrom
gsd/cascade-update-sf-pr

Conversation

@yedidyakfir

@yedidyakfir yedidyakfir commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator

Milestone v1.3.6 — Cascade Reach Through Special-Field References

Extends the v1.3.5 TTL cascade so the shared walker also discovers and re-arms ForeignKey references held inside special-field containers — RedisSet[ForeignKey[T]] and RedisPriorityQueue[ForeignKey[T]] — which live under their own SET/ZSET keys and were never read by the inline-only (JSON.GET) traversal. Traversal-reach only; the per-child cascading-TTL-refresh apply layer is reused unchanged.

Phase 1 — Classify SF-held FK references into the cascade plan

  • New sf_container discriminator on CascadeEdge; SF-held-ref discovery pass in build_cascade_plan / _static_walk_sf_fk_edges (precedence: field > global > off; fail-fast on a missing target Meta.ttl).

Phase 2 — Traverse SF-held references server-side and re-arm children

  • library.lua: new push_sf_edge branch reads SET (SMEMBERS) / ZSET (ZRANGE) members server-side, decodes each into its target key, and feeds them through the existing push_child / next_hop / visited budget machinery — so SF-reached children re-arm to their own Meta.ttl in the same atomic FCALL as inline-reached children.
  • Model-level trigger gate (rapyer/base.py): _contains_foreign_key() now ORs in _has_cascade_enabled_sf_ref_edge() so asave / aset_ttl(cascade=True) / refresh_ttl actually fire the cascade Function for parents whose only cascade edge is SF-held (previously they silently took the native-EXPIRE fast path).
  • Serialization fix: RedisSet / RedisPriorityQueue._dump_members now validate before dumping, so a raw FK key string round-trips correctly.
  • Docs: TTL-cascade coverage matrix updated to all five cascade-eligible shapes, with a worked SF-held-ref example and the fakeredis/real-Redis divergence note.

Verification

  • Phase-goal verification: PASSED 9/9 must-haves (CASF-04..10 all traced).
  • Cycle-safety proven: self-ref in SET/PQ, mixed inline+SF diamonds, SF-only dual-edge diamonds, and children reachable both inline and via SF all terminate and re-arm per the shared budget/visited rules.
  • Dual-backend: real Redis Stack :6370 (Function path) + fakeredis root-own-EXPIRE fallback.
  • Full suites green, zero regression: 818 unit + 1623 integration passed / 205 skipped.

Follow-ups (non-blocking, from code review)

  • WR-01: an unresolvable SF-held forward ref (typo / unregistered / init_with_rapyer=False target) is silently dropped — no edge, no startup error, cascade silently off. Consider wiring into the fail-fast path.
  • WR-02: RedisSet validates members without the redis context while RedisPriorityQueue validates with it; harmless today, risks divergence for a future context-sensitive inner type.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XWA2KXKFrqLo5qnhDC3ACi

Summary by CodeRabbit

  • New Features

    • TTL cascades now follow references stored in Redis sets and priority queues.
    • Parent saves and cascading TTL updates refresh eligible referenced records.
    • Cascade handling supports nested references, collections, self-references, and shared targets.
    • Reference values in Redis containers are validated before storage.
  • Bug Fixes

    • Improved handling of forward and nested reference definitions.
    • Clarified fakeredis behavior: container TTLs refresh, but contained members are not traversed.
  • Documentation

    • Added guidance on cascade-eligible reference shapes and TTL behavior.

YedidyaHKfir and others added 11 commits July 26, 2026 01:48
…ntainer discriminator to CascadeEdge + SF-held-ref fixtures

- CascadeEdge gains sf_container: str | None = None as its last field, dropped
  by _drop_none_values/cascade_plan_json so non-SF plan bytes stay identical
- Add SF-held-ref test fixtures (RedisSet/RedisPriorityQueue holding
  Reference[T]) covering per-field, blanket, opt-out, and ttl=None fail-fast
  cases in tests/models/cascade_types.py
- Register the four ttl-carrying fixtures in ALL_CASCADE_MODELS and
  CASCADE_PLANNER_MODELS
…ef discovery pass in build_cascade_plan + edge tests

- Add _static_walk_sf_fk_edges: a dedicated pass over _special_field_names
  that emits a distinct CascadeEdge (sf_container="set"/"zset") for each
  cascade-enabled RedisSet/RedisPriorityQueue field holding a Reference[T],
  honoring field > global > off precedence via _classify_edge
- Wire the new pass into build_cascade_plan right after _static_walk_fk_edges,
  appending to the same entry.fks list
- Lazy-import RedisSet/RedisPriorityQueue inside the pass: a module-top import
  reintroduces a real cycle (priority_queue -> special -> scripts.loader ->
  planner), contrary to the plan's cycle-safety assumption
- Add tests/unit/cascade/test_cascade_sf_held_ref_plan.py covering the new
  edge shape (set/zset), coexistence with the refresh-only special suffix,
  precedence, and the None-drop hash-stability guarantee for non-SF edges
- Fix: mark the three deliberately-ttl-less SF fail-fast fixtures
  (CascadeSetRefNoTtlTarget/CascadeSetRefToNoTtl/CascadeSetRefRootNoTtl,
  added in the prior task-1 commit) init_with_rapyer=False so they are
  excluded from REDIS_MODELS and don't trip init_rapyer()'s full-model-set
  ttl validation in unrelated tests
…nested sub-model exclusion in SF-held-ref pass

Clarify that _static_walk_sf_fk_edges intentionally handles only direct
SF fields and does not recurse into nested inline sub-models (WR-01 from
code review). Nested-case traversal is deferred to the server-side work.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qot6HrjzLLCBhmHaH1CLRM
…verage matrix and worked example

- Add Cascade-Eligible Shapes coverage-matrix table (5 shapes) to ttl-cascade.md
- Add worked RedisSet cascade example mirroring CascadeSetRefParent fixture
- Extend real-Redis-7+/fakeredis divergence note for SF-held-ref cascade
… fixtures

- CascadeSetRefSelfNode/CascadePQRefSelfNode: self-ref-in-SET/PQ cycle-safety
- CascadeMixedEdgeSharedChild(Root): inline+SF shared-child max-budget-wins
- CascadeSfDiamondChild/CascadeSfDiamondRoot: SF-only dual-edge diamond
- All six registered in ALL_CASCADE_MODELS; none added to CASCADE_PLANNER_MODELS
…nch in push_edges

- push_sf_edge reads SMEMBERS/ZRANGE keyed on edge.sf_container, decodes
  each member with a guarded pcall(cjson.decode), and feeds valid
  string target keys through the existing push_child/next_hop/visited
  machinery
- push_edges now splits per-node edges into SF (dispatched immediately)
  vs inline (still batched through the single JSON.GET); SF edge paths
  never enter the inline paths array
- next_hop is called once per SF edge per node-walk, not per member
…scadeSetRefParent (RedisSet)

- Proves CASF-09's fakeredis leg for the SET-shaped SF-held-ref fixture
- Real fakeredis, real aset_ttl(cascade=True) call, no mocking
- Root main key + refs SF container key refresh; SET member (author) untouched
…scadePQRefParent (RedisPriorityQueue)

- Mirrors the RedisSet proof for the ZSET-backed SF-held-ref fixture
- Proves both SF container kinds (SET and ZSET) that Phase 1 classified
  correctly fall back to root-own-EXPIRE on fakeredis
…aversal proof (CASF-04..08)

- New test_cascade_sf_held_ref_apply.py: 8 tests (A-H) proving SET reach,
  PQ reach, dangling-count reuse, self-ref-in-SET/PQ termination, mixed
  inline+SF max-budget-wins, SF-only dual-edge diamond convergence, and
  malformed/non-string SF member tolerance -- all against real Redis :6370
- test_cascade_graph_shapes.py, test_cascade_depth_and_gate.py, and
  tests/unit/cascade/ pass unmodified (CASF-08 byte-for-byte proof)

Two auto-fixed bugs surfaced by the new self-ref-in-SET/PQ fixtures
(Rule 1/3 -- blocking, discovered exercising this plan's own fixtures for
the first time):

- [Rule 3] _unwrap_relational_target returned a raw typing.ForwardRef for
  a self-referencing FK target baked into an SF container's dynamic
  subclass generic args (RedisConverter._build_redis_subclass captures it
  at class-body time, before pydantic's model_rebuild can resolve it).
  Added _resolve_forward_ref, looking the name up in the global model
  registry, in rapyer/cascade/planner.py.
- [Rule 1] RedisSet/RedisPriorityQueue._dump_members called dump_python
  directly on raw native-Python input (e.g. a bare FK target-key string)
  without validating first; dump_python never validates, so a ForeignKey
  element reached its serializer as a plain string and crashed on
  `_target_key`. Fixed by validating via the adapter before dumping in
  rapyer/types/redis_set.py and rapyer/types/priority_queue.py -- no
  behavior change for existing scalar-typed SF fields (verified via the
  full test suite).
…r gate for SF-only parents

- Add class_declares_cascade_enabled_sf_ref_edge() to planner.py, reusing
  _static_walk_sf_fk_edges's field>global>off classification verbatim
- Add lazily-cached _has_cascade_enabled_sf_ref_edge() classmethod + OR it
  into _contains_foreign_key(), so refresh_ttl/aset_ttl now fire the
  cascade Function for SF-only cascade-enabled parents
- contains_fk_field() and __init_subclass__ left byte-for-byte unmodified
- Mock-based unit proof (test_cascade_sf_only_trigger_gate.py): run_fcall
  fires for CascadeSetRefParent/CascadePQRefParent, still plain pipe.expire
  for the cascade-disabled CascadeSetRefOptOut
…of for SF-only cascade parents

- Test A/C: asave() on CascadeSetRefParent/CascadePQRefParent re-arms an
  SF-held child's own Meta.ttl through the public save path
- Test B: aset_ttl(ttl, cascade=True) re-arms the child directly, returning
  a CascadeResult with dangling_children=0
- No _apply_cascade/run_fcall helper used anywhere in this file -- proves
  the fix through the literal public API (asave/aset_ttl), closing
  CASF-04/05/06 end-to-end (02-01 proved reach only at the direct-FCALL level)
@coderabbitai

coderabbitai Bot commented Jul 25, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: a5554f66-145f-44ee-b79a-0c9ceaa1eae4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds TTL cascade support for foreign-key references stored in RedisSet and RedisPriorityQueue, including planner metadata, Lua traversal, TTL routing, member coercion, documentation, and unit/integration coverage for cycles, budgets, dangling references, and fakeredis fallback behavior.

Changes

SF-held reference cascade

Layer / File(s) Summary
Cascade planning and fixtures
rapyer/cascade/planner.py, tests/models/cascade_types.py, tests/unit/cascade/*
Plans container-backed FK edges, resolves forward references, validates TTL requirements, and adds coverage for supported, opted-out, cyclic, mixed, and diamond-shaped graphs.
TTL routing and member serialization
rapyer/base.py, rapyer/types/redis_set.py, rapyer/types/priority_queue.py
Routes models with cascade-enabled SF edges through cascade execution and validates/coerces special-field members before JSON serialization.
Lua cascade traversal
rapyer/scripts/lua/cascade/library.lua
Reads SET/ZSET members, decodes target keys, applies traversal budgets, and preserves inline-edge batching.
Integration and documentation
tests/integration/foreign_keys/*, tests/unit/cascade/test_cascade_sf_held_ref_fakeredis_fallback.py, docs/documentation/special-fields/ttl-cascade.md
Verifies public and direct cascade application, cycle termination, deduplication, malformed members, fakeredis behavior, and eligible reference shapes.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ModelAPI
  participant CascadePlanner
  participant CascadeLua
  participant RedisContainer
  ModelAPI->>CascadePlanner: build SF-held FK cascade plan
  CascadePlanner-->>ModelAPI: return container-aware CascadeEdge
  ModelAPI->>CascadeLua: refresh TTL with cascade
  CascadeLua->>RedisContainer: read SET/ZSET members
  RedisContainer-->>CascadeLua: return referenced keys
  CascadeLua-->>ModelAPI: refresh reachable targets and report results
Loading

Possibly related PRs

Suggested reviewers: yedidyahkfir

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: TTL cascade traversal through special-field ForeignKey references in RedisSet and RedisPriorityQueue.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch gsd/cascade-update-sf-pr

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 25, 2026

Copy link
Copy Markdown

Coverage report

Total coverage: 99%

Full report
Name                                                     Stmts   Miss  Cover
----------------------------------------------------------------------------
rapyer/__init__.py                                           5      0   100%
rapyer/actions.py                                          213      0   100%
rapyer/base.py                                             869      2    99%
rapyer/cascade/__init__.py                                   3      0   100%
rapyer/cascade/planner.py                                  177      7    96%
rapyer/cascade/spec.py                                      13      0   100%
rapyer/cascade/ttl.py                                        5      0   100%
rapyer/config.py                                            45      0   100%
rapyer/context.py                                           40      0   100%
rapyer/errors/__init__.py                                   11      0   100%
rapyer/errors/base.py                                       23      0   100%
rapyer/errors/cascade.py                                     8      0   100%
rapyer/errors/delete.py                                      3      0   100%
rapyer/errors/find.py                                       15      0   100%
rapyer/fields/__init__.py                                    4      0   100%
rapyer/fields/expression.py                                108      0   100%
rapyer/fields/index.py                                      20      0   100%
rapyer/fields/key.py                                        19      0   100%
rapyer/fields/safe_load.py                                  14      0   100%
rapyer/init.py                                              67      0   100%
rapyer/links.py                                              2      0   100%
rapyer/result.py                                            31      0   100%
rapyer/scripts/__init__.py                                   5      0   100%
rapyer/scripts/constants.py                                 17      0   100%
rapyer/scripts/loader.py                                    38      0   100%
rapyer/scripts/lua/__init__.py                               0      0   100%
rapyer/scripts/lua/atomic/__init__.py                        0      0   100%
rapyer/scripts/lua/cascade/__init__.py                       0      0   100%
rapyer/scripts/lua/datetime/__init__.py                      0      0   100%
rapyer/scripts/lua/dict/__init__.py                          0      0   100%
rapyer/scripts/lua/list/__init__.py                          0      0   100%
rapyer/scripts/lua/numeric/__init__.py                       0      0   100%
rapyer/scripts/lua/sf/__init__.py                            0      0   100%
rapyer/scripts/lua/sf/redis_priority_queue/__init__.py       0      0   100%
rapyer/scripts/lua/sf/redis_set/__init__.py                  0      0   100%
rapyer/scripts/lua/string/__init__.py                        0      0   100%
rapyer/scripts/registry.py                                  63      0   100%
rapyer/types/__init__.py                                    13      0   100%
rapyer/types/base.py                                       103      0   100%
rapyer/types/byte.py                                        33      0   100%
rapyer/types/convert.py                                     53      0   100%
rapyer/types/datetime.py                                    77      0   100%
rapyer/types/dct.py                                        116      0   100%
rapyer/types/float.py                                       65      0   100%
rapyer/types/foreign_key.py                                 68      0   100%
rapyer/types/generic.py                                     83      0   100%
rapyer/types/init.py                                        10      0   100%
rapyer/types/integer.py                                     58      0   100%
rapyer/types/lst.py                                        129      0   100%
rapyer/types/priority_queue.py                             112      0   100%
rapyer/types/redis_set.py                                  206      0   100%
rapyer/types/relational.py                                  24      0   100%
rapyer/types/special.py                                     39      0   100%
rapyer/types/string.py                                      22      0   100%
rapyer/typing_support.py                                     3      0   100%
rapyer/utils/__init__.py                                     0      0   100%
rapyer/utils/annotation.py                                  62      1    98%
rapyer/utils/fields.py                                      43      0   100%
rapyer/utils/pythonic.py                                    21      0   100%
rapyer/utils/redis.py                                       77      0   100%
----------------------------------------------------------------------------
TOTAL                                                     3235     10    99%

@codspeed-hq

codspeed-hq Bot commented Jul 25, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 15.77%

❌ 17 regressed benchmarks
✅ 154 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime test_benchmark 2.4 ms 3 ms -20%
WallTime test_benchmark 1.5 ms 1.9 ms -19.34%
WallTime test_benchmark_with_ttl 1.7 ms 2.1 ms -19.13%
WallTime test_benchmark 1.2 ms 1.4 ms -17.28%
WallTime test_benchmark_with_ttl 1.8 ms 2.2 ms -17.09%
WallTime test_benchmark_with_ttl 2 ms 2.4 ms -16.52%
WallTime test_benchmark_with_ttl 1.7 ms 2 ms -16.37%
WallTime test_benchmark 14.4 ms 17.2 ms -16.17%
WallTime test_benchmark 1.2 ms 1.4 ms -15.69%
WallTime test_benchmark 1.3 ms 1.6 ms -15.67%
WallTime test_benchmark_with_ttl 1.7 ms 2 ms -15.5%
WallTime test_benchmark 1.4 ms 1.6 ms -15.04%
WallTime test_benchmark 1.1 ms 1.3 ms -14.06%
WallTime test_benchmark 1.3 ms 1.4 ms -12.77%
WallTime test_benchmark 948.1 µs 1,085.4 µs -12.65%
WallTime test_benchmark_with_ttl 1.8 ms 2.1 ms -12.25%
WallTime test_benchmark 16.3 ms 18.5 ms -11.99%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing gsd/cascade-update-sf-pr (d3a6562) with develop (67943eb)

Open in CodSpeed

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
tests/integration/foreign_keys/test_cascade_sf_held_ref_apply.py (1)

199-205: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Make the diamond test detect duplicate traversal.

The current TTL and [0, 0] assertions also pass if the shared child is walked twice. Give the shared child a dangling descendant and assert it contributes exactly one dangling count.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/foreign_keys/test_cascade_sf_held_ref_apply.py` around
lines 199 - 205, Update the diamond test around _apply_cascade to add a dangling
descendant beneath the shared child, then assert the result includes exactly one
dangling reference from that descendant. Keep the existing TTL assertions and
ensure the expected result distinguishes single traversal from duplicate visits.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/documentation/special-fields/ttl-cascade.md`:
- Around line 94-128: Convert the Python example in the TTL cascade
documentation from a fenced code block to the repository’s required indented
code-block style, preserving the example content and surrounding explanation
unchanged.

In `@rapyer/scripts/lua/cascade/library.lua`:
- Around line 226-253: Update push_sf_edge to read the SET/ZSET container with
redis.pcall instead of redis.call, preserving the existing command selection.
Treat any failed SMEMBERS or ZRANGE result as an empty member list and return
without aborting traversal, while retaining the current per-member JSON decoding
and push_child behavior for successful reads.

In `@tests/unit/cascade/test_cascade_sf_held_ref_fakeredis_fallback.py`:
- Line 38: Update the TTL assertions for persisted author keys in the fakeredis
fallback test to require exactly -1, including the assertion at the second
occurrence. Remove -2 from the accepted values so child-key deletion cannot be
treated as successful behavior.

---

Nitpick comments:
In `@tests/integration/foreign_keys/test_cascade_sf_held_ref_apply.py`:
- Around line 199-205: Update the diamond test around _apply_cascade to add a
dangling descendant beneath the shared child, then assert the result includes
exactly one dangling reference from that descendant. Keep the existing TTL
assertions and ensure the expected result distinguishes single traversal from
duplicate visits.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e1a8b9ea-a3df-408d-8af9-f6e2f1b1c59b

📥 Commits

Reviewing files that changed from the base of the PR and between 67943eb and 268a9eb.

📒 Files selected for processing (13)
  • docs/documentation/special-fields/ttl-cascade.md
  • rapyer/base.py
  • rapyer/cascade/planner.py
  • rapyer/scripts/lua/cascade/library.lua
  • rapyer/types/priority_queue.py
  • rapyer/types/redis_set.py
  • tests/integration/foreign_keys/test_cascade_sf_held_ref_apply.py
  • tests/integration/foreign_keys/test_cascade_sf_held_ref_public_api.py
  • tests/models/cascade_types.py
  • tests/unit/cascade/conftest.py
  • tests/unit/cascade/test_cascade_sf_held_ref_fakeredis_fallback.py
  • tests/unit/cascade/test_cascade_sf_held_ref_plan.py
  • tests/unit/cascade/test_cascade_sf_only_trigger_gate.py

Comment on lines +94 to +128
```python
from typing import Annotated, ClassVar

from pydantic import Field
from rapyer import AtomicRedisModel
from rapyer.cascade import CascadeTTL
from rapyer.config import RedisConfig
from rapyer.types import Reference, RedisSet


class Author(AtomicRedisModel):
name: str = "anon"

Meta: ClassVar[RedisConfig] = RedisConfig(ttl=3600)


class Library(AtomicRedisModel):
name: str = "main"
authors: Annotated[RedisSet[Reference[Author]], CascadeTTL()] = Field(
default_factory=RedisSet
)

Meta: ClassVar[RedisConfig] = RedisConfig(ttl=3600)


library = await Library(name="main").asave()
author = await Author(name="Jane").asave()
await library.authors.aadd(author.key)

# Every member of the set is re-armed to its own Meta.ttl, exactly like an
# inline collection-of-FK field, whenever the parent's TTL is (re)set:
await library.asave()
# ...or explicitly:
await library.aset_ttl(3600, cascade=True)
```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Use an indented code block for this example.

This new fenced block triggers the active MD046 warning; convert it to the repository’s required indented style.

🧰 Tools
🪛 markdownlint-cli2 (0.23.0)

[warning] 94-94: Code block style
Expected: indented; Actual: fenced

(MD046, code-block-style)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/documentation/special-fields/ttl-cascade.md` around lines 94 - 128,
Convert the Python example in the TTL cascade documentation from a fenced code
block to the repository’s required indented code-block style, preserving the
example content and surrounding explanation unchanged.

Source: Linters/SAST tools

Comment on lines +226 to +253
-- SF-held-ref edge: read the field's own SET/ZSET key (not the inline
-- JSON document) and feed each decoded member through push_child.
-- next_hop is called ONCE per edge here, never per member.
local function push_sf_edge(parent_key, edge, remaining_budget, established)
local follow, budget = next_hop(edge, remaining_budget, established)
if not follow then
return
end
if not edge.recurse_into_target then
budget = 0
end
local sf_key = special_prefix .. ':' .. parent_key .. ':' .. edge.path
local members
if edge.sf_container == 'set' then
members = redis.call('SMEMBERS', sf_key)
else
members = redis.call('ZRANGE', sf_key, 0, -1)
end
for _, raw_member in ipairs(members) do
-- A malformed (non-JSON) or non-string-decoded member is a dead
-- end for that member only -- never aborts the atomic FCALL.
local ok, target_key = pcall(cjson.decode, raw_member)
if ok and type(target_key) == 'string' then
push_child(target_key, edge, budget)
end
end
end

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

SET/ZSET reads aren't error-tolerant like the inline JSON.GET path.

push_sf_edge uses redis.call('SMEMBERS'/'ZRANGE', ...) while the inline path (read_reference_paths) deliberately uses redis.pcall('JSON.GET', ...) so a WRONGTYPE/corrupt target becomes "a dead end for further traversal, not an aborted cascade." A WRONGTYPE (or any Redis-level error) on an SF container key here raises and aborts the whole atomic FCALL during the read phase — before any EXPIRE runs — so nothing gets refreshed at all, not even the root. That breaks the same resilience guarantee the rest of the file is careful to uphold.

🔒️ Proposed fix: mirror read_reference_paths' pcall tolerance
         local sf_key = special_prefix .. ':' .. parent_key .. ':' .. edge.path
-        local members
+        local members
         if edge.sf_container == 'set' then
-            members = redis.call('SMEMBERS', sf_key)
+            members = redis.pcall('SMEMBERS', sf_key)
         else
-            members = redis.call('ZRANGE', sf_key, 0, -1)
+            members = redis.pcall('ZRANGE', sf_key, 0, -1)
+        end
+        if type(members) ~= 'table' or members.err then
+            return  -- WRONGTYPE/corrupt SF key: dead end for this edge only
         end
         for _, raw_member in ipairs(members) do
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
-- SF-held-ref edge: read the field's own SET/ZSET key (not the inline
-- JSON document) and feed each decoded member through push_child.
-- next_hop is called ONCE per edge here, never per member.
local function push_sf_edge(parent_key, edge, remaining_budget, established)
local follow, budget = next_hop(edge, remaining_budget, established)
if not follow then
return
end
if not edge.recurse_into_target then
budget = 0
end
local sf_key = special_prefix .. ':' .. parent_key .. ':' .. edge.path
local members
if edge.sf_container == 'set' then
members = redis.call('SMEMBERS', sf_key)
else
members = redis.call('ZRANGE', sf_key, 0, -1)
end
for _, raw_member in ipairs(members) do
-- A malformed (non-JSON) or non-string-decoded member is a dead
-- end for that member only -- never aborts the atomic FCALL.
local ok, target_key = pcall(cjson.decode, raw_member)
if ok and type(target_key) == 'string' then
push_child(target_key, edge, budget)
end
end
end
-- SF-held-ref edge: read the field's own SET/ZSET key (not the inline
-- JSON document) and feed each decoded member through push_child.
-- next_hop is called ONCE per edge here, never per member.
local function push_sf_edge(parent_key, edge, remaining_budget, established)
local follow, budget = next_hop(edge, remaining_budget, established)
if not follow then
return
end
if not edge.recurse_into_target then
budget = 0
end
local sf_key = special_prefix .. ':' .. parent_key .. ':' .. edge.path
local members
if edge.sf_container == 'set' then
members = redis.pcall('SMEMBERS', sf_key)
else
members = redis.pcall('ZRANGE', sf_key, 0, -1)
end
if type(members) ~= 'table' or members.err then
return -- WRONGTYPE/corrupt SF key: dead end for this edge only
end
for _, raw_member in ipairs(members) do
-- A malformed (non-JSON) or non-string-decoded member is a dead
-- end for that member only -- never aborts the atomic FCALL.
local ok, target_key = pcall(cjson.decode, raw_member)
if ok and type(target_key) == 'string' then
push_child(target_key, edge, budget)
end
end
end
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@rapyer/scripts/lua/cascade/library.lua` around lines 226 - 253, Update
push_sf_edge to read the SET/ZSET container with redis.pcall instead of
redis.call, preserving the existing command selection. Treat any failed SMEMBERS
or ZRANGE result as an empty member list and return without aborting traversal,
while retaining the current per-member JSON decoding and push_child behavior for
successful reads.

assert result == CascadeResult(dangling_children=0, dangling_special=0)
assert await fake_redis_client.ttl(parent.key) > 0
assert await fake_redis_client.ttl(refs_key) > 0
assert await fake_redis_client.ttl(author.key) in (-1, -2)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Do not accept deletion as successful fallback behavior.

Each author key was persisted, so its unchanged TTL must be -1. Allowing -2 masks an erroneous child-key deletion.

Proposed fix
-    assert await fake_redis_client.ttl(author.key) in (-1, -2)
+    assert await fake_redis_client.ttl(author.key) == -1

Also applies to: 63-63

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/cascade/test_cascade_sf_held_ref_fakeredis_fallback.py` at line
38, Update the TTL assertions for persisted author keys in the fakeredis
fallback test to require exactly -1, including the assertion at the second
occurrence. Remove -2 from the accepted values so child-key deletion cannot be
treated as successful behavior.

Source: Path instructions

Comment on lines +20 to +29
async def _apply_cascade(real_redis_client, root):
return await real_redis_client.fcall(
type(root).Meta.cascade_function_name,
1,
root.key,
type(root).__name__,
SPECIAL_FIELD_KEY_PREFIX,
type(root).Meta.ttl,
1,
)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be more common function, in conftest for upper level, I think many can use it

Comment on lines +52 to +55
assert await real_redis_client.ttl(parent.key) > 0
assert await real_redis_client.ttl(author1.key) > 0
assert await real_redis_client.ttl(author2.key) > 0

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you only check above 0? cant you check more specific to see the ttl is correct?

await real_redis_client.persist(key)

# Act
await _apply_cascade(real_redis_client, parent)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you check with apply and not actions that shuld require cascade instead?

yedidyakfir added a commit that referenced this pull request Jul 31, 2026
… asserts

Addresses PR #289 review comments: the fcall helper was duplicated across
4 integration test modules; hoisted to conftest.py as apply_cascade for
reuse. Also tightens the bare `ttl > 0` asserts in
test_cascade_sf_held_ref_apply.py to `0 < ttl <= CASCADE_FIXTURE_TTL_SECONDS`,
matching the precise-bound pattern already used in test_cascade_ttl_apply.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
YedidyaHKfir and others added 4 commits July 31, 2026 15:13
…into _contain_fk

RedisSet/RedisPriorityQueue.contains_fk_field() now introspects the member
type (mirroring GenericRedisType), so an SF-held FK field flows into _contain_fk
via the existing __init_subclass__ path — the same detection predicate as inline
FK fields. The planner's _contain_fk walk branches by read-shape (nested inline
sub-model / RedisSet-or-PQ sf_container edge / inline collection), with a
top_level guard preserving the deferral of nested SF-held-ref traversal.

_contains_foreign_key() collapses back to `_relational_field_names or
_contain_fk`; the bolted-on _has_cascade_enabled_sf_ref_edge gate, its
_cascade_sf_ref_edge_flag cache, and planner.class_declares_cascade_enabled_sf_ref_edge
are deleted. build_cascade_plan emits byte-identical plans (verified).

Reverses Phase-1 decision D-02 (SF edge now lives in _contain_fk as well as
_special_field_names). A cascade-opt-out SF-only parent now gates like a normal
opt-out FK: it takes the FCALL path with an empty-edge plan (no child re-arm)
instead of a plain-EXPIRE special case — removing the SF-vs-inline asymmetry.

Tests: invert the D-02 guard; add plain-SF-container and nested-deferral
regressions; update the opt-out gate test to the unified behavior. Full suites
green (unit 820, integration 1623/205 skipped, zero regression).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XWA2KXKFrqLo5qnhDC3ACi
…K or special field

Rename AtomicRedisModel._contains_foreign_key -> _needs_cascade_script and broaden
it to `_relational_field_names or _contain_fk or _special_field_names or _contain_sf`.
Any model with an FK edge (direct or containing) OR a special field (direct or nested
via a link) now refreshes through the cascade Redis Function; only plain scalar models
keep the native-EXPIRE fast path. Both fast-path call sites updated.

Correct because the script refreshes the same keys the EXPIRE path did (root JSON +
special-field keys via the plan's special_suffixes), following no edge when there are
none. Tradeoff: plain-SF models now issue an FCALL per refresh instead of pipelined
EXPIRE.

Tests: reverse the plain-SF gate test (a RedisSet[str] model now needs the script),
rename the freeze-test entry, update a docstring reference. Full suites green
(unit 820, integration 1623/205 skipped, zero regression).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XWA2KXKFrqLo5qnhDC3ACi
…ionale as one-line # comments

Trim reasons-dump function/class docstrings to a single "what" line and move
implementation rationale to one-line # comments above the specific code block,
per the project comment style. No logic change (diff is comments/docstrings
only; cascade unit suite green).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XWA2KXKFrqLo5qnhDC3ACi
… asserts

Addresses PR #289 review comments: the fcall helper was duplicated across
4 integration test modules; hoisted to conftest.py as apply_cascade for
reuse. Also tightens the bare `ttl > 0` asserts in
test_cascade_sf_held_ref_apply.py to `0 < ttl <= CASCADE_FIXTURE_TTL_SECONDS`,
matching the precise-bound pattern already used in test_cascade_ttl_apply.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@yedidyakfir yedidyakfir left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do all the tests for cascade use apply cascade and dont use actions to cause ttl update (like += or read action)
Is this case tested?

Comment on lines +24 to +28
with (
patch("rapyer.base.ensure_pipeline", fake_ensure_pipeline),
patch("rapyer.base.scripts_registry.run_fcall") as mock_run_fcall,
):
# Act

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

patches should be extaracted to fixtures so many tests can use the same patch

yedidyakfir added a commit that referenced this pull request Jul 31, 2026
…t patch

Addresses PR #289 review comments.

CodeRabbit (major): push_sf_edge used redis.call for SMEMBERS/ZRANGE, unlike
the inline read_reference_paths path's redis.pcall -- a WRONGTYPE on an SF
container key aborted the whole atomic FCALL before any EXPIRE ran, breaking
the resilience guarantee the rest of the file upholds. Now pcall + treat an
error as an empty member list; added the SF-container counterpart of the
existing inline WRONGTYPE regression test.

yedidyakfir: the ensure_pipeline/run_fcall patch-and-assert boilerplate was
duplicated across test_cascade_sf_only_trigger_gate.py,
test_refresh_ttl_cascade_branch.py, and test_aset_ttl_cascade_flag.py.
Hoisted to a shared fcall_pipeline_spy fixture in tests/unit/cascade/conftest.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…t patch

Addresses PR #289 review comments.

CodeRabbit (major): push_sf_edge used redis.call for SMEMBERS/ZRANGE, unlike
the inline read_reference_paths path's redis.pcall -- a WRONGTYPE on an SF
container key aborted the whole atomic FCALL before any EXPIRE ran, breaking
the resilience guarantee the rest of the file upholds. Now pcall + treat an
error as an empty member list; added the SF-container counterpart of the
existing inline WRONGTYPE regression test.

yedidyakfir: the ensure_pipeline/run_fcall patch-and-assert boilerplate was
duplicated across test_cascade_sf_only_trigger_gate.py,
test_refresh_ttl_cascade_branch.py, and test_aset_ttl_cascade_flag.py.
Hoisted to a shared fcall_pipeline_spy fixture in tests/unit/cascade/conftest.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants