Skip to content

Advanced expression time-bucket merge fails when OR criteria reuse the same key #79

Description

@Maksym-0xp

Disclosure: ChatGPT 5.6 Sol was used to format this issue. As well as generating/writing the GO related suggestions, as I don't know GO it might be complete BS..

The non stacking when using AND(OR(originalFileName, originalFileName), localDateTime) is present, but can be worked around, so its not a high priority issue.

Problem

Assets that have the same normalized filename and are within the configured localDateTime delta are not stacked when:

  • Advanced expression mode is used.
  • An OR contains multiple criteria for the same key, such as originalFileName.
  • The timestamps fall into adjacent fixed time buckets.

The affected assets are only 145 ms apart with a configured delta of 1000 ms, but they remain unstacked without an error.

Relationship to #40 and #46/#65

This appears related to #40 and #46, which was fixed by #65, but it is a distinct advanced-expression edge case rather than a direct regression of either fixed path.

#40 added the sliding-window time-bucket merge for legacy and advanced-expression processing. #65 extended that behavior to convertible advanced groups configurations.

This report already uses the advanced expression format, but its OR alternatives reuse the originalFileName criterion key.

Setup

Item Value
Installation Docker Compose
immich-stack version 0.2.42
Image revision 4530f75602bb7d2dd1e2a99289ac61c7a2cc4355
Image digest sha256:83f02b8a9329b0ea6b30d3ffbf049d66b7a34b5d1c17eef9fc6c8b2c4841588e
Immich version v3.1.0
Run mode cron
Cron interval 300 seconds
Dry run false
Replace existing stacks false

Criteria

{
  "mode": "advanced",
  "expression": {
    "operator": "AND",
    "children": [
      {
        "operator": "OR",
        "children": [
          {
            "criteria": {
              "key": "originalFileName",
              "regex": {
                "key": "(?i)^(PXL_[0-9]{8}_[0-9]+)\\.RAW-[0-9]+(\\.COVER|\\.ORIGINAL)?\\.(jpe?g|jpe|dng)$",
                "index": 1
              }
            }
          },
          {
            "criteria": {
              "key": "originalFileName",
              "regex": {
                "key": "(?i)^(.+)\\.(jpe?g|jpe|3fr|ari|arw|cap|cin|cr2|cr3|crw|dcr|dng|erf|fff|iiq|k25|kdc|mrw|nef|nrw|orf|ori|pef|raf|raw|rw2|rwl|sr2|srf|srw|x3f)$",
                "index": 1
              }
            }
          }
        ]
      },
      {
        "criteria": {
          "key": "localDateTime",
          "delta": {
            "milliseconds": 1000
          }
        }
      }
    ]
  }
}

Reproduction

The following two assets normalize to the same filename through the Pixel-specific regex:

Filename localDateTime
PXL_20260826_114529052.RAW-02.ORIGINAL.dng 2026-08-26T11:45:28.907Z
PXL_20260826_114529052.RAW-01.jpg 2026-08-26T11:45:29.052Z

The timestamps differ by only 145 ms, which is within the configured 1000 ms delta.

They fall into adjacent initial buckets:

  • 2026-08-26T11:45:28.000Z
  • 2026-08-26T11:45:29.000Z

After multiple cron executions, both assets remain unstacked. The job completes without reporting an error or creating a stack for the pair.

Assets with equivalent names and timestamps inside the same truncated second do stack successfully, suggesting the failure occurs specifically in the adjacent-bucket merge.

Expected behavior

One stack containing both assets should be created because:

  1. Both filenames produce PXL_20260826_114529052 through the Pixel regex.
  2. Their timestamps differ by 145 ms.
  3. The configured delta is 1000 ms.

Actual behavior

No stack is created, and both assets remain as unstacked singletons.

Possible cause

From reading the current implementation, there appears to be a positional mismatch between the flattened criteria and the generated grouping key:

  1. flattenCriteriaFromExpression preserves every leaf and produces:

    [originalFileName, originalFileName, localDateTime]
    
  2. The grouping-key construction deduplicates repeated field keys, producing a key shaped like:

    [originalFileName, localDateTime]
    
  3. mergeTimeBasedGroups derives the time position from the non-deduplicated criteria and therefore records index 2.

  4. In the actual grouping key, the time component is at index 1. extractNonTimeComponents consequently does not recognize it as the time component and retains the time bucket as part of the non-time grouping key.

  5. Because the adjacent buckets now appear to have different non-time keys, they are never compared by the sliding-window merge.

Relevant implementation files:

The duplicate-key OR shape is also similar to the documented Mixed Camera Setups. The corresponding example test appears to use identical timestamps, so it does not exercise an adjacent-bucket merge.

Suggested regression test

Use an expression shaped like:

AND(
  OR(originalFileName regex A, originalFileName regex B),
  localDateTime delta 1000 ms
)

Provide two assets that:

  • Normalize to the same filename.
  • Have timestamps 28.907 and 29.052.
  • Therefore cross a fixed one-second boundary while remaining only 145 ms apart.

The result should be one group containing both assets.

Possible fixes could either:

  • Deduplicate criteria consistently before calculating time-component positions, or
  • Identify time components by their criterion key instead of by a positional index.

Configuration workaround

Moving the localDateTime child before the filename OR changes the flattened order to:

[localDateTime, originalFileName, originalFileName]

The generated key then begins with:

[localDateTime, originalFileName]

This keeps the time component at index 0 in both representations and should avoid the mismatch. However, the ordering of children under an AND expression should not change its behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions