memory: bound cache admission and eviction

[ravi-pplx]

Problem

The old memory tier evicted at capacity by collecting and selection-sorting
every cached object while holding one process-wide write lock. Admission cost
grew quadratically with cache cardinality, so a miss burst could block unrelated
hits. Incomplete writers and reader-pinned generations also did not share one
enforceable memory budget.

Change

• Shard entries and single-key operations across 16 locks.
• Replace the global scan and sort with fixed-work CLOCK admission.
• Scan at most 64 entries per shard and commit at most 64 victims per trim.
• Revalidate planned victims with hit generations before removal.
• Account retained entries, incomplete writers, replacement buffers, and
  reader-pinned generations against one ceiling.
• Allocate declared-length bodies only as bytes arrive and never grow their
  buffers beyond the declared length.
• Use a 4 KiB starter capacity for small unknown-length bodies and reserve only
  net buffer growth.
• Reject negative, overflowing, or finite inflight limits that leave no retained
  capacity.
• Distinguish inflight sub-limit exhaustion from hard-limit exhaustion and only
  run eviction when it can make the reservation succeed.
• Abort completed and late tier writers when parallel creation is cancelled or
  one backend fails; release successful writer contexts on close.
• Restore io.WriterTo on memory hits so response writes avoid a 32 KiB
  intermediate buffer.
• Emit cachew.memory.admission_declines_total with a bounded reason label.
• Decline only the optional memory-tier copy when bounded admission cannot make
  room. Authoritative-tier writes and client delivery continue.

Capacity semantics

limit-mb bounds Cachew's accounted memory, not process RSS. Accounted memory
includes retained buffer capacity, estimated entry metadata, and retired buffers
still pinned by readers. Every retained entry and incomplete writer carries a
minimum 4 KiB charge, so a 1 GiB cache can hold at most roughly 262,000 objects
even when their payloads are smaller. limit-mb = 0 means unlimited retention.

inflight-limit-mb defaults to zero for compatibility. When both limits are
finite, it must be smaller than limit-mb and is a sub-limit rather than extra
capacity. With limit-mb = 32768 and inflight-limit-mb = 1024, retained data
is trimmed toward 31 GiB and retained plus incomplete-writer accounting cannot
exceed 32 GiB. An unlimited cache may still use a positive inflight limit to
bound incomplete writes independently. Inflight exhaustion declines that
optional writer without trimming retained objects, because eviction cannot free
inflight capacity.

Declared bodies grow lazily but stop exactly at their promised length.
Unknown-length bodies grow geometrically; spare capacity is charged rather than
hidden, avoiding another full-body copy at publication. Growing a writer reserves
only the net capacity increase before copying. The allocator can briefly retain
both buffers, so RSS may transiently exceed accounted memory by at most the old
buffer capacity.

Before and after

Measured on an Apple M4 Max (darwin/arm64). The public comparison runs the same
Create + 16 KiB Write + Close operation against pre-PR commit 410ebdf and
the core bounded-eviction commit cb06519; values are medians of three one-second
runs. Follow-up commits preserve the same sharding and CLOCK algorithm.

Approximate entries	Before	Core implementation	Speedup	Before B/op	Core B/op
256	130 us/op	3.57 us/op	36.5x	83,888	18,823
1,024	1.80 ms/op	3.50 us/op	514x	214,859	18,815
4,096	29.4 ms/op	3.42 us/op	8,610x	1,484,649	18,820

Final-head cardinality checks at ec489cb:

Workload	1K entries	10K entries	100K entries
Admission only	0.41 us/op	0.44 us/op	0.48 us/op
Configured Create + Write + Close	2.28 us/op	2.27 us/op	2.38 us/op

The restored 1 MiB memory-hit WriteTo path improves from 4.95 us/op and
33,448 B/op at cb06519 to about 0.24 us/op and 680 B/op at ec489cb, about a
20x speedup while removing the 32 KiB scratch allocation. This benchmark writes
to a discard sink and therefore reports latency and allocations, not synthetic
byte-copy throughput. Parallel hot Open/Close remains about 0.27 us/op.
Saturated parallel admission completes in about 0.51 us per attempt and accepts
about 58% of optional memory copies; bounded declines do not shed client
requests.

Correctness coverage

• Post-plan hits invalidate older CLOCK eviction plans.
• Accounting is reconstructed through create, grow, publish, replace, abort,
  delete, reader-pinned retirement, and shutdown.
• Declared lengths remain lazy and finish with exact buffer capacity.
• Equivalent unknown-length streams remain cacheable across write chunk sizes.
• Invalid finite inflight configurations fail construction.
• Unlimited retained entries survive a fully occupied inflight budget.
• Cancellation before victim commit leaves existing entries intact.
• Tiered cancellation and backend errors abort both completed and late writers;
  successful close releases the derived writer context.
• Every silent memory-tier decline path records a bounded reason.
• A declined memory copy still writes the complete object to the authoritative
  tier.