Possible use-after-unmap in buildSmallBlockMatcherProto during hs_compile_multi

[xqjcool]

Summary

We are seeing a deterministic crash inside Hyperscan 5.4.2 while calling
hs_compile_multi() with 2787 expressions.

The crash occurs later in buildIncludedIdMap() while reading the first
hwlmLiteral from fproto->hwlmProto->lits. Kernel-side munmap tracing
shows that the backing address range was previously released from
buildSmallBlockMatcherProto() through the ASan allocator.

This appears to be a possible ownership/lifetime issue involving a temporary
MatcherProto and the final LitProto/HWLMProto.

Environment

• Hyperscan: 5.4.2
• Architecture: x86_64 Linux
• Application compiler: Clang 20.1.8
• libstdc++ headers/runtime: GCC 14.3.0
• Application build: optimized release build with AddressSanitizer
• Application ASan flags: -fsanitize=address
• jemalloc disabled
• libhs.so.5 itself is not ASan-instrumented
• libhs.so.5 uses the normal dynamic C/C++ allocators:
  malloc, free, operator new, and operator delete

The application calls hs_compile_multi() synchronously. The expression,
flag, ID, platform, output database, and error-pointer arrays remain valid
for the duration of the call.

Crash stack

The relevant stack is:

#10 std::unordered_map<...>::find(...)
#11 ue2::contains(...)
    src/util/container.h:51
#12 ue2::buildIncludedIdMap(...)
    src/rose/rose_build_bytecode.cpp:2885
#13 ue2::findInclusionGroups(...)
    src/rose/rose_build_bytecode.cpp:2910
#14 ue2::buildLiteralPrograms(...)
    src/rose/rose_build_bytecode.cpp:3061
#15 ue2::RoseBuildImpl::buildFinalEngine(...)
    src/rose/rose_build_bytecode.cpp:3715
#16 ue2::RoseBuildImpl::buildRose(...)
#17 ue2::generateRoseEngine(...)
#18 ue2::build(...)
#19 ue2::hs_compile_multi_int(...)
    src/hs.cpp:255
#20 hs_compile_multi(...)
    src/hs.cpp:438

At the fault:

fproto = 0x7bc6eb356ea0
drproto = NULL
eproto = NULL
sbproto = NULL
elements = 2787

&lit.id = 0x7b96e49bc820

GDB cannot read lit.id because its mapping has already been removed.

### munmap evidence

We added temporary tracing around the kernel munmap syscall and captured
the successful munmap that covers the fault address:

tgid=9749 tid=9749
addr=0x7b96e4978000
len=0x6a000
end=0x7b96e49e2000

The fault address lies inside this range:

0x7b96e49bc820 - 0x7b96e4978000 = 0x44820

The captured userspace stack associated with that munmap resolves to:

__sanitizer::internal_munmap(void *, unsigned long) + 7
__sanitizer::UnmapOrDie(void *, unsigned long, bool) + 32
__asan::asan_free(...) + 471
___interceptor_free.part.0
ue2::buildSmallBlockMatcherProto(...) + 434
ue2::RoseBuildImpl::buildFinalEngine(...) + 6036
ue2::RoseBuildImpl::buildRose(...)
ue2::generateRoseEngine(...)
ue2::build(...)
ue2::hs_compile_multi_int(...)
hs_compile_multi(...)

Therefore, the mapping containing lit.id was released through a normal
C++/ASan free path originating from buildSmallBlockMatcherProto().

### Reproducibility

We have observed this multiple times with different ASLR layouts.

For example, in another run:

munmap start = 0x7be5c02e9000
munmap length = 0x6a000
fault address = 0x7be5c032d820

Again:

fault address - munmap start = 0x44820

The following properties are stable across runs:

- 2787 expressions
- crash at buildIncludedIdMap() line 2885
- release originates from buildSmallBlockMatcherProto() + 434
- munmap length is 0x6a000
- fault offset inside the unmapped allocation is 0x44820

This makes random address corruption less likely.

### Additional observations

The userspace stack memory also contains addresses resolving to:

std::__introsort_loop<... ue2::AccelString ...>
__interceptor_memcmp

These are not necessarily active unwind frames, but they indicate that the
same function previously processed/sorted a temporary AccelString vector.

Disassembly around buildSmallBlockMatcherProto() suggests that the release
may occur during cleanup of a local MatcherProto, possibly around an early
return or a branch handling a MatcherProto with one literal.

Our current hypothesis is:

1. buildSmallBlockMatcherProto() creates a temporary MatcherProto.
2. Some vector storage is copied/moved or otherwise referenced by the final
   LitProto/HWLMProto.

3. Cleanup of the temporary object frees the storage.
4. buildIncludedIdMap() later iterates fproto->hwlmProto->lits, whose
   storage still points into the freed mapping.

This is a hypothesis, not yet a confirmed source-level root cause.

### Questions

1. Is there any known ownership or move/copy issue between MatcherProto,
   LitProto, and HWLMProto in buildSmallBlockMatcherProto()?

2. Is the early-return/single-literal path expected to destroy storage that
   may already have been transferred to the resulting LitProto?

3. Are there known compatibility issues when a non-ASan Hyperscan shared
   library is loaded into an ASan-instrumented executable?

4. Would you recommend any specific internal assertions or address logging
   to verify ownership of mp.lits, mp.accel, fproto->accel_lits, and
   fproto->hwlmProto->lits?

We can provide the exact expression set privately if needed.

[xqjcool]

Description

We are investigating a deterministic SIGSEGV in Hyperscan 5.4.2 while
compiling a multi-pattern database with hs_compile_multi().

The crash occurs in ue2::buildIncludedIdMap() while reading the first
hwlmLiteral from the floating LitProto.

The address is no longer mapped at the time of the access.

Environment

• Hyperscan: 5.4.2

• Architecture: x86_64 Linux

• Compiler: Clang 20.1.8

• C++ standard library: libstdc++ 14.3.0

• Build type: Release / -O3

• Shared libraries: enabled

• FAT_RUNTIME=OFF

• Sanitizer flags:

  -fsanitize=address -fno-omit-frame-pointer -fno-optimize-sibling-calls -g

• hs_compile_multi():

  • elements: 2787
  • mode: HS_MODE_BLOCK

Crash stack

The relevant part of the stack is:

ue2::buildIncludedIdMap()
ue2::findInclusionGroups()
ue2::buildLiteralPrograms()
ue2::RoseBuildImpl::buildFinalEngine()
ue2::RoseBuildImpl::buildRose()
ue2::generateRoseEngine()
ue2::build()
ue2::hs_compile_multi_int()
hs_compile_multi()

The fault occurs at:

} else if (lit.included_id != INVALID_LIT_ID) {

At the time of the crash:

litProto             = 0x7bc04c456e70
litProto->hwlmProto  = 0x7c404c283d20
&lit                 = 0x7b9045a9a800
&lit.id              = 0x7b9045a9a820
&lit.included_id     = 0x7b9045a9a828

GDB cannot access the literal storage:

(gdb) x/16gx &lit
0x7b9045a9a800: Cannot access memory at address 0x7b9045a9a800

## Lifetime instrumentation

We added temporary logging around:

- buildFloatingMatcherProto()
- buildSmallBlockMatcherProto()
- buildIncludedIdMap()

For each vector, we log its owner, data pointer, end pointer, capacity
pointer, size, capacity, and element size.

### Reproduction 1

floating.result.accel_lits
  begin=0x7bce17615800
  end=0x7bce17656120
  cap=0x7bce17656120
  size=2755 capacity=2755 elem_size=96

floating.result.hwlm_lits
  begin=0x7bce17658800
  end=0x7bce17696e28
  cap=0x7bce17696e28
  size=2457 capacity=2457 elem_size=104

small.mp.lits
  begin=0x7bce17615800
  end=0x7bce17651040
  cap=0x7bce1767d800
  size=2344 capacity=4096 elem_size=104

The capacity range of small.mp.lits overlaps both live vectors in the
floating result.

In particular:

small.mp.lits.begin == floating.result.accel_lits.begin

### Reproduction 2

floating.result.accel_lits
  begin=0x7b3e6a83c800
  end=0x7b3e6a87d120
  cap=0x7b3e6a87d120

floating.result.hwlm_lits
  begin=0x7b3e6a87f800
  end=0x7b3e6a8bde28
  cap=0x7b3e6a8bde28

small.mp.lits
  begin=0x7b3e6a83c800
  end=0x7b3e6a878040
  cap=0x7b3e6a8a4800

The same overlap occurs under a different ASLR layout.

Later, the same floating LitProto is passed to
buildIncludedIdMap():

included.enter
  litProto=0x7b6e71156e40
  hwlmProto=0x7bee70f83d20

included.hwlm_lits
  begin=0x7b3e6a87f800

included.lit
  lit=0x7b3e6a87f800
  id_addr=0x7b3e6a87f820

The first literal is then accessed and the process receives SIGSEGV.

## Current interpretation

The logs appear consistent with one of the following:

1. storage still referenced by the floating LitProto is freed too early
   and then reused by small.mp.lits; or

2. two live vectors unexpectedly share/overlap the same allocation.

We understand that the logs alone do not yet distinguish an upstream
Hyperscan lifetime issue from an allocator/integration issue.

Could buildSmallBlockMatcherProto() legitimately invalidate or reuse
storage owned by the floating LitProto?

Are there any known ownership or move/copy issues involving
MatcherProto, LitProto, HWLMProto, or AccelString in this path?

## Diagnostic patch

We can provide the small logging patch used to collect the owner and
vector storage ranges.

We are also testing whether temporarily skipping
buildSmallBlockMatcherProto() prevents the crash.