Skip to content

Add paginated object-dependencies endpoint for users, avoid OOM - #2011

Open
jcPimcore wants to merge 12 commits into
2025.4from
fix/308-paginated-user-object-dependencies
Open

Add paginated object-dependencies endpoint for users, avoid OOM#2011
jcPimcore wants to merge 12 commits into
2025.4from
fix/308-paginated-user-object-dependencies

Conversation

@jcPimcore

@jcPimcore jcPimcore commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Changes in this pull request

Resolves: pimcore/platform-version#308 Studio UI
Resolves: pimcore/admin-ui-classic-bundle#1106 Studio UI

GET /user/{id} OOMs / times out when the user is referenced by a very large number of DataObjects via a User-type class field, because ObjectDependenciesService::getDependenciesForUser() unconditionally hydrated every matching object and embedded the full, unbounded list in every response (same class of bug as pimcore/admin-ui-classic-bundle#1106, just not yet reported for Studio).

  • Add ObjectDependenciesRepository/Interface: reproduces the class/field discovery, but counts matches per class first (cheap, non-hydrating) and only ever hydrates the slice of objects that falls inside a requested offset/limit window - bounded regardless of total match count or page depth.
  • Add a new paginated GET /user/{id}/object-dependencies endpoint (GetObjectDependenciesController), modeled on the existing generic element Dependencies tab's CollectionController, to browse the full list. Enforces a MAX_PAGE_SIZE of 100 (matching the largest page size selectable in the Studio UI) before offset/limit are computed, and resolves/authorizes the target user first (404 if unknown, 403 if it's an admin and the caller isn't) instead of silently returning 200 with an empty collection.
  • No schema-breaking change, but a real behavioral impact worth knowing about: objectDependencies on the main User schema keeps its existing dependencies/hasHidden fields exactly as they were (same names, same types) - it only gains a new field, totalItems, alongside them. totalItems is marked optional in the OpenAPI schema (even though it's always present in the actual response) specifically because studio-ui-bundle is an actively-published npm package and this type is re-exported through its public SDK entry point - making it required would generate a non-optional TS field and break any existing consumer code constructing/mocking a value of that type. However, dependencies previously contained the user's complete list of referencing objects, and now stops at 20. Any existing integration that assumed dependencies was exhaustive (rather than checking totalItems) will now silently see only the first 20 - such integrations must start reading totalItems and, if it exceeds dependencies.length, call the new paginated endpoint for the rest. hasHidden is affected the same way: it now only reflects permission-denied objects within that 20-item window, not across the full list as before. getPreviewForUser() is a new service method used only by UserHydrator for this embedded preview, kept separate from getPaginatedDependenciesForUser() (used by the new endpoint, returns the generic Collection type).
  • Multi-User-field classes: dependency matching now joins conditions with OR instead of AND - an object referencing the user via any one of several User-type fields was previously excluded from both the preview and the endpoint unless every field matched.
  • Deterministic ORDER BY id on the underlying listings, so offset/limit paging is stable across requests (verified: two adjacent pages against real data have zero overlap and no gap).
  • Upgrade note added/refined across a few passes to accurately describe the above.

Additional info

Verified against a real 50,000-object repro (a backend user referenced by 50,000 DataObjects via a user field, via the real DI container against the live DB, not just unit-mocked):

  • The new endpoint returns exactly the requested page (tested page 1, a mid page, and the true last page) with correct totalItems; memory stays flat across page depth (no growth as offset increases).
  • UserHydrator's embedded preview returns class=ObjectDependencies, hasHidden=false, dependencies=20, totalItems=50000 in ~0.2s / ~90MB peak (bounded, no OOM).
  • Schema/User.php diffs as byte-for-byte identical to its pre-change state; ObjectDependencies.php diffs as purely additive (only totalItems + its getter added, dependencies/hasHidden untouched).
  • A focused unit test (ObjectDependenciesRepositoryTest) covers the cross-class pagination windowing math directly (class-boundary pages, deep offsets, exhausted budget, exact boundaries), since the surrounding dynamic class-discovery/listing code has no fixtures in this bundle's unit suite.

Related: pimcore/studio-ui-bundle's fix/308-paginated-user-object-dependencies (frontend for this endpoint), pimcore/ee-admin-ui-classic-bundle's fix/308-cap-user-object-dependencies (short-term cap for the classic admin UI, same root issue).

🤖 Generated with Claude Code

…rray

ObjectDependenciesService::getDependenciesForUser() and the resolver it
called (DataObject\Service::getObjectsReferencingUser()) unconditionally
hydrate every DataObject referencing a user via a User-type field, and
the result was embedded in every GET /users/{id} response. This has the
same unbounded-hydration OOM risk as admin-ui-classic-bundle#1106, just
not yet reported for Studio.

Add ObjectDependenciesRepository, which reproduces the same class/field
discovery but counts matches per class first and only ever hydrates the
slice of objects that falls inside the requested page's offset/limit
window, regardless of total match count. Expose it via a new paginated
GET /user/{id}/object-dependencies endpoint (GetObjectDependenciesController),
modeled on the generic element Dependencies tab's CollectionController.
Drop the objectDependencies field from the main user payload/schema and
UserHydrator, since it's no longer embedded.

Refs: pimcore/platform-version#308
Note the removal of the embedded objectDependencies field from GET
/users/{id} and point at the new paginated GET /user/{id}/object-
dependencies endpoint, following the existing per-version bullet
convention in this file.

Refs: pimcore/platform-version#308
Avoid the BC break of dropping objectDependencies entirely: keep it on
the User schema, but as a small, deliberately-capped 20-item preview
(ObjectDependenciesPreview: totalItems + dependencies) built by calling
the same paginated ObjectDependenciesService added earlier with
page=1/pageSize=20, instead of the old unbounded hydration. Consumers
that need more than the preview use GET /user/{id}/object-dependencies.

Update the upgrade note accordingly: this is a schema shape change
(hasHidden removed, totalItems added, capped at 20), not a removal.

Refs: pimcore/platform-version#308
Copilot AI balanced review requested due to automatic review settings August 20, 2026 13:06
@jcPimcore jcPimcore self-assigned this Aug 20, 2026
@jcPimcore jcPimcore added this to the 2026.2.7 milestone Aug 20, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds bounded previews and paginated retrieval for user object dependencies to prevent excessive memory usage.

Changes:

  • Adds a paginated object-dependencies endpoint and repository.
  • Caps embedded user dependency previews at 20 items.
  • Updates tests, API documentation, configuration, and upgrade guidance.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
translations/studio_api_docs.en.yaml Adds endpoint documentation strings.
tests/Unit/User/Service/ObjectDependenciesServiceTest.php Tests dependency permission filtering.
src/User/Service/ObjectDependenciesServiceInterface.php Defines the paginated service API.
src/User/Service/ObjectDependenciesService.php Hydrates paginated, permitted dependencies.
src/User/Schema/User.php Uses the preview schema.
src/User/Schema/ObjectDependenciesPreview.php Defines the bounded preview response.
src/User/Schema/ObjectDependencies.php Removes the previous unbounded schema.
src/User/Repository/ObjectDependenciesRepositoryInterface.php Defines paginated repository access.
src/User/Repository/ObjectDependenciesRepository.php Implements cross-class counting and pagination.
src/User/Hydrator/UserHydrator.php Builds the 20-item preview.
src/User/Controller/GetObjectDependenciesController.php Exposes the paginated endpoint.
doc/02_Installation_and_Configuration/05_Upgrade.md Documents the response schema change.
config/users.yaml Registers the new repository.
Suppressed comments (3)

src/User/Repository/ObjectDependenciesRepository.php:100

  • Joining multiple User-field predicates with AND only finds objects where every User field references this user. An object that references the user in just one of several User fields is omitted from both the page and totalItems; these predicates need OR semantics.
        $list->setCondition(implode(' AND ', $conditionParts), array_fill(0, count($conditionParts), $userId));

src/User/Controller/GetObjectDependenciesController.php:83

  • No lookup validates $id before querying class listings, so an unknown user ID returns 200 with an empty collection even though this endpoint advertises NOT_FOUND. It also bypasses the admin-user visibility check used by UserService::getUserById(). Resolve and authorize the target user before returning dependencies.
        $collection = $this->objectDependenciesService->getPaginatedDependenciesForUser($id, $parameters);

src/User/Repository/ObjectDependenciesRepository.php:50

  • The object listing is paged with OFFSET/LIMIT but has no ORDER BY. Database row order is not guaranteed, so repeated page requests can overlap or omit dependencies. Apply a deterministic unique order (for example, the object ID) before loading each slice.
                        $list->setOffset($localOffset);
                        $list->setLimit($localLimit);
                        $items = array_merge($items, $list->load());

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/User/Controller/GetObjectDependenciesController.php
Comment thread src/User/Repository/ObjectDependenciesRepository.php
@jcPimcore jcPimcore modified the milestones: 2026.2.7, 2025.4.12 Aug 20, 2026
- ObjectDependenciesRepository: join multi-field User conditions with OR,
  not AND - an object referencing the user via any one of several
  User-type fields was previously excluded from both the page and
  totalItems unless every field matched. Also add a deterministic
  ORDER BY id, since OFFSET/LIMIT paging without one isn't guaranteed
  stable across requests (verified against real data: two adjacent
  pages now have zero overlap and no gap).
- Extract the cross-class windowing math into a pure resolveClassWindow()
  method and add a focused unit test for it (class-boundary pages, deep
  offsets, exhausted budget, exact boundaries) - the surrounding
  class-discovery/listing code still has no unit fixtures in this
  bundle's suite, but the arithmetic that determines pagination
  correctness now does.
- GetObjectDependenciesController: enforce a MAX_PAGE_SIZE of 100
  (matching the largest selectable page size in the Studio UI) before
  offset/limit are calculated, since CollectionParameters has no upper
  bound and an unbounded pageSize would reintroduce the OOM this
  endpoint exists to prevent. Also resolve and authorize the target
  user (NotFoundException / admin-only-viewable-by-admins, mirroring
  UserService::getUserById()) before querying dependencies, instead of
  querying an arbitrary/nonexistent id and silently returning 200 with
  an empty collection.
- PageSizeParameter gains an optional maxSize to surface the cap in the
  OpenAPI schema; existing call sites are unaffected (defaults to no
  maximum).

Refs: pimcore/platform-version#308
@jcPimcore

Copy link
Copy Markdown
Contributor Author

Addressed all 5 points from the Copilot review in 2ca20d3:

  • ObjectDependenciesRepository: multi-field User conditions now joined with OR instead of AND (an object referencing the user via any one of several User-type fields was previously excluded unless every field matched).
  • Added a deterministic ORDER BY id so offset/limit paging is stable across requests — verified against the real 50k-row repro data: two adjacent pages now have zero overlap and no gap.
  • GetObjectDependenciesController now resolves and authorizes the target user (404 if missing, 403 if it's an admin and the caller isn't, mirroring UserService::getUserById()) before querying dependencies, instead of silently returning 200 with an empty collection for an unknown/unauthorized id.
  • Added a MAX_PAGE_SIZE = 100 cap (matching the largest selectable page size in the Studio UI), enforced before offset/limit are computed, since CollectionParameters has no upper bound on its own and an unbounded pageSize would reintroduce the OOM this endpoint exists to prevent.
  • Extracted the cross-class windowing arithmetic into a pure resolveClassWindow() method and added ObjectDependenciesRepositoryTest covering class-boundary pages, deep offsets, exhausted budget, and exact boundaries — the surrounding dynamic class-discovery/listing code still has no unit fixtures in this bundle's suite, but the math that determines pagination correctness is now directly tested.

🤖 Generated with Claude Code

The success-response description for GET /user/{id}/object-dependencies
copied this bundle's standard "with total count as header param"
phrasing, but PaginatedResponseTrait::getPaginatedCollection() also
serializes totalItems in the JSON body via the Collection DTO - true for
every endpoint using that trait, not just this one. Only fixing the one
description this PR introduced; the same incomplete phrasing predates
this change on ~15 other endpoints across the bundle and is out of scope
here.

Refs: pimcore/platform-version#308, review feedback on
pimcore/studio-ui-bundle#4008
The bounded preview (previous commit) still replaced ObjectDependencies
(dependencies + hasHidden) with a differently-shaped ObjectDependenciesPreview
(totalItems + dependencies), dropping hasHidden. That was never necessary -
adding totalItems doesn't require removing hasHidden.

Restore the original ObjectDependencies class name and its dependencies/
hasHidden fields exactly as they were, and add totalItems alongside them
as a new, additive property. GET /users/{id} keeps returning the same
shape it always has, just with dependencies now capped at 20 (down from
unbounded) and one new field. hasHidden's scope narrows to the visible
preview window rather than scanning every referencing object, which is
an unavoidable, documented consequence of no longer scanning everything -
but the schema itself has zero breaking changes: nothing was removed or
renamed, nothing changed type.

Add ObjectDependenciesServiceInterface::getPreviewForUser(), used only by
UserHydrator, alongside the existing getPaginatedDependenciesForUser()
used by the new paginated endpoint - keeps the preview's original return
type separate from the generic Collection type the real pagination uses.

Refs: pimcore/platform-version#308
Purely cosmetic: ObjectDependenciesServiceInterface had moved from its
original 3rd position to last. Autowiring resolves by type so this
never affected anything functionally, but restoring the exact original
order removes even a theoretical positional-instantiation concern.

Refs: pimcore/platform-version#308
Listing::load() is typed to return the broader DataObject[], not
Concrete[], even though a per-class dynamic Listing (e.g.
Issue1106\Listing) only ever loads Concrete instances of that class.
Narrow it with an instanceof filter so the inferred return type matches
the interface's declared array{items: Concrete[], totalItems: int} -
a no-op at runtime (verified: still returns the same 50/50000 against
the real repro data, and every item is genuinely Concrete already).

Verified with a real PHPStan run (installed standalone, not part of
this bundle's own composer deps) against this file and the rest of
src/User at this bundle's configured level 6: no errors.

Refs: pimcore/platform-version#308
CI failed with PHPUnit's ClassIsFinalException: Dependency is a
`final readonly class`, so makeEmpty()/mock generators can't create a
test double for it. It's a trivial constructor-only DTO anyway -
instantiate a real one instead of doubling it.

Verified the fixed test logic directly (same code path, real classes,
not just reasoning): items=1, totalItems=2 as expected.

Refs: pimcore/platform-version#308

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (2)

doc/02_Installation_and_Configuration/05_Upgrade.md:6

  • The affected endpoint is singular (GET /user/{id} in GetUserController), so this upgrade note currently directs readers to a nonexistent /users/{id} route. Use the actual route name here.
- [User Management] Fixed: `GET /users/{id}` could time out or run out of memory when the user was referenced by a very large number of DataObjects (e.g. via a `User`-type class field), because the full, unbounded list of referencing objects was hydrated and embedded in every response. The `objectDependencies.dependencies` array on the `User` schema is now capped at 20 entries, and a new paginated `GET /user/{id}/object-dependencies` endpoint was added to browse the full list.

doc/02_Installation_and_Configuration/05_Upgrade.md:8

  • Calling this “not a breaking change” is misleading: although the response shape is additive, dependencies previously contained the complete collection and now silently stops at 20. Existing API clients that rely on the full array must migrate to the new endpoint, so the upgrade note should explicitly identify that behavioral compatibility impact.
> **Note:** this is not a breaking change. `objectDependencies` keeps its existing `dependencies`/`hasHidden` fields; it only gains a new `totalItems` field alongside them. `hasHidden` now only reflects permission-denied objects within the 20-item preview window, rather than scanning every referencing object as before - use `totalItems` and `GET /user/{id}/object-dependencies?page=&pageSize=` to see the full picture beyond the preview.

- The upgrade note said GET /users/{id} (plural); the actual route is
  GET /user/{id} (singular, GetUserController).
- "Not a breaking change" was an overclaim. The objectDependencies
  *schema* genuinely has no breaking change (dependencies/hasHidden
  keep their names, types, meaning; totalItems is purely additive),
  but dependencies previously contained the complete list and now
  caps at 20 - a real behavioral compatibility impact for any consumer
  that assumed completeness rather than checking totalItems. State
  that plainly instead of a blanket "not breaking" claim.

Refs: pimcore/platform-version#308
@jcPimcore

Copy link
Copy Markdown
Contributor Author

Addressed both points from this review in d8d01a8:

  • Fixed the route typo: the upgrade note said GET /users/{id} (plural); the actual route is GET /user/{id} (singular, GetUserController).
  • Reworded the "not a breaking change" note - it was an overclaim. To be precise: the objectDependencies schema genuinely has no breaking change (dependencies/hasHidden keep their names, types, and meaning; totalItems is purely additive). But you're right that there's a real behavioral compatibility impact: dependencies previously contained the complete list and now caps at 20. Any consumer that assumed completeness rather than checking totalItems will now silently see a partial view. The note states that plainly now instead of a blanket "not breaking" claim. Also updated the PR description to match.

🤖 Generated with Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

studio-ui-bundle is a real, actively-published npm package
(@pimcore/studio-ui-bundle), and UserObjectDependencies is re-exported
through its public SDK entry point (sdk/api/user/index.ts). Marking
totalItems as OpenAPI-required generated it as a non-optional TS field,
which is source-breaking for any SDK consumer constructing or mocking
a value of that type without it.

Drop totalItems from the schema's required list. The PHP property
itself is unaffected and always set - this only changes the generated
contract to totalItems?: number, which is backward compatible with
existing consumer code while still giving new consumers the real value
in every actual response.

Verified against real data: hasHidden=false, dependencies=20,
totalItems=50000 - unchanged at runtime, only the schema's required
list changed.

Refs: pimcore/platform-version#308

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (2)

src/User/Controller/GetObjectDependenciesController.php:108

  • The controller now performs user lookup and admin authorization directly in addition to calling the dependency service. This duplicates UserService::getUserById() (src/User/Service/UserService.php:157-165) and breaks the User-domain convention that controllers delegate to one service, making alternate service callers bypass these checks. Move target-user resolution and authorization into ObjectDependenciesServiceInterface/its implementation so this controller only invokes that service.
        $targetUser = $this->userRepository->getUserById($id);

        if ($targetUser->isAdmin() && !$this->securityService->getCurrentUser()->isAdmin()) {
            throw new ForbiddenException('Only admins can view other admins');

tests/Unit/User/Repository/ObjectDependenciesRepositoryTest.php:63

  • This data provider is non-static, but the repository requires PHPUnit 12.5/13.1, where data providers must be public and static. PHPUnit will therefore reject or skip these cases, leaving the pagination math effectively untested.
    public function windowProvider(): array

Per Copilot review feedback: totalItems is a raw match count that
never subtracts permission-denied objects, since accurately excluding
them would require hydrating and checking every matching object
across the whole dataset - exactly the unbounded cost this endpoint
exists to avoid. That's a legitimate tradeoff, but was previously
undocumented: a page (or the embedded preview) can come back shorter
than requested, or empty, while totalItems stays the same.

Documented in the ObjectDependencies schema description, the
totalItems property description, the paginated endpoint's operation
description, and the upgrade note.

Refs: pimcore/platform-version#308
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
1 New Major Issues (required ≤ 0)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@jcPimcore
jcPimcore requested a review from martineiber August 21, 2026 10:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants