fix(endpoint-posts): look up posts by id so older posts don’t 404

[rmdes]

Fixes #924.

• getPostProperties now reads the posts collection by _id (getObjectId from @indiekit/util). Invalid ids and a missing database return false as before, so the 404 path is unchanged.
• Regression test seeds 41 posts and fetches the oldest; it fails on main with 404 !== 200.
• The five tests that relied on the mock Micropub ?q=source listing now seed a post in the test database; the unused interceptor is removed from the mock agent.
• Unit tests for getPostProperties (found, not found, invalid id, no database).

Verified locally: node --test packages/endpoint-posts/test/**/*.js 37/37, eslint and prettier clean.

[paulrobertlloyd]

@rmdes The reason for the current implementation is so that endpoint-posts (hopefully) works if you’ve configured Indiekit to use a third-party Micropub server (see application.micropubEndpoint).

This PR ties the posts endpoint directly to the database, which is what I was trying to avoid. Should Indiekit support third-party Micropub servers? Some (many?) may not provide all the features needs to power the posts interface, and makes Indiekit all the more complex. But supporting third party endpoints feels like we’re following the ethos of the IndieWeb. It’s a good question, and open to thoughts!

[rmdes]

You're right about the intent, and digging into the history I think this is a regression rather than a design disagreement.

Until 32be7f47 ("feat(endpoint-posts): query posts using uid", Oct 2023), getPostProperties did this:

micropubUrl.searchParams.append("q", "source");
micropubUrl.searchParams.append("url", getPostUrl(id));

One post, fetched by URL. That's exactly the query the Micropub spec defines — "include the URL of the post in the url parameter" — and servers that support updating posts MUST support it, so it works against any conforming third-party endpoint. The spec defines no pagination and no way to list posts via q=source, and no way to fetch a post by any identifier other than its URL.

Since that commit we ask for ?q=source with no url and no limit, take getCursor's default page of 40 (util/lib/mongodb.js:23) and .find(item => item.uid === uid) within it — so anything below the newest 40 is a 404. That's the bug, and it isn't really about the database: it's the url-less form of a query the spec only defines with a url.

The awkward part is the identifier. jf2ToMf2 injects uid: [_id] (endpoint-micropub/lib/mf2.js:49), so the uid these routes key on is Indiekit's own ObjectId — something a third-party server would never return. The per-post routes are therefore already tied to our database through the id, which is the thing you were trying to avoid.

So I'd rather ask than assume: what drove the move to uid in 32be7f47? If it was to get base64url blobs out of the URLs, that makes sense — but the url-keyed version was the portable one, and restoring &url= is what would make third-party endpoints real rather than nominal.

Happy to rework this either way:

1. Restore the url-keyed query — getPostUrl is still in utils.js and still unit-tested; needs the getPostId encoder back and the route to carry a URL.
2. Keep uid, add &url= — resolve id → url once, then query by url; keeps the current route shape.
3. This PR as-is — simplest, but only if third-party Micropub servers are out of scope for the posts UI.

One caveat on (2), so it isn't oversold: resolving id → url needs something that maps an ObjectId to a post URL, and against a third-party endpoint that's the same chicken-and-egg — the only way to learn the URL for an unknown id is the paginated listing we're trying to stop relying on. So (2) fixes the 404 and restores spec compliance for Indiekit's own store, but only (1) actually decouples the per-post routes from our database. (2) is the smaller, safer step; (1) is the one that delivers the goal.

I'd default to (2) to keep your 2023 decision about route shape intact, but if third-party support is the priority then (1) is the honest answer.

[paulrobertlloyd]

Looking into this, I think one of the primary reasons to use ObjectID was that it’s date sortable, whereas an encoded URL isn’t.

I’m wondering if the way forward is for Indiekit to include properties.uid on posts and files which is a UUIDv7 string, and support querying posts by uid as well as url.

This could then also be the first step to removing support for MongoDB, as we could index on that UUID instead of ObjectIDs.

We’d possibly need a way to migrate from _id: ObjectID to properties.uid: UUIDv7 for existing servers, too.

I wonder if a Micropub server doesn’t return posts with properties.uid whether there would be a way to fall back to properties.url, but that might involve needless complexity.

Thoughts?

[rmdes]

UUIDv7 keeps the sortability ObjectId was giving you — 48-bit millisecond prefix, so lexicographic order is time order.

It also removes the awkward part of going backend-agnostic. In #917 the only seam the collection interface needed was id casting — const castId = collection.castId ?? getObjectId in getCursor, with the SQLite side supplying Number. A UUIDv7 is a string in either backend, so that seam stops being needed rather than being papered over.

One migration detail worth catching: post-type-count.js:42-43 does getObjectId(properties.uid) to exclude the post being updated. new ObjectId() throws a BSONError on anything that isn't a 24-character hex string, so the first update of a post carrying a UUID uid would 500 rather than degrade. Easy to move across in the same change, but it wants moving deliberately.

On the url fallback, I'd skip it: Indiekit writes a uid for every post it stores, so it would only fire for foreign data, and dual-key lookup is easier to add than to withdraw.

Happy to take the migration — UUIDv7 for new posts, backfilled from the ObjectId timestamp so ordering survives, post-type-count moved across. As a separate PR ahead of #925 if you'd prefer.

[paulrobertlloyd]

Yeah, let's do a separate PR. I started investigating this myself, but would like to see your take. Should make it easier for me to review too, having now dug into the code a bit.