fix(pchronicle): non-loopback serve and lazy Directory discovery - #128
Open
reiase wants to merge 5 commits into
Open
fix(pchronicle): non-loopback serve and lazy Directory discovery#128reiase wants to merge 5 commits into
reiase wants to merge 5 commits into
Conversation
Stop recursive object-store listing that hit max_files on large prefixes, classify Directory vs Dataset from shallow markers, and let catalog serve bind/listen before discovery finishes. Co-authored-by: Cursor <cursoragent@cursor.com>
Consolidated multiple lines of code into single lines for improved clarity and readability in the `discovery.rs` and `generic.rs` files. This change enhances the overall code structure without altering functionality.
…ry logic Added a new `Directory` variant to the `Candidate` enum to represent navigational directories in the catalog. Updated the discovery logic to classify immediate child directories and dataset sources separately, ensuring that loose files are not registered as sources. Enhanced the sorting and counting of sources in the catalog to accommodate the new directory type. Updated documentation to reflect these changes.
…ehavior Added support for writing and reading relative bytes in the DatasetLocation, enabling recursive listing of importable JSON objects. Improved the import process to clear existing prefixes in object stores before writing new data, ensuring a clean slate for imports. Updated documentation to clarify the behavior of the replace mode for object-store datasets, emphasizing that it clears the destination prefix before writing. Added tests to verify the new import behavior and ensure correct handling of existing data.
… modules Enhanced the formatting of code in the `location.rs`, `exchange.rs`, `sync.rs`, and `tests.rs` files for better clarity. This includes consolidating multiple lines into single lines where appropriate and ensuring consistent indentation. No functional changes were made, focusing solely on code aesthetics and maintainability.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
catalog://pins so ECS/LAN listeners and pins work.chronicle.manifest/CURRENT/ events markers → Dataset; plain prefixes only probe immediate children (no loose-file Sources).servediscovery in the background so--listencan accept connections before large prefixes finish classifying; update RFCs 0013/0015 accordingly.Test plan
just test persisting-pchroniclejust test persisting-pchronicle-cli(loopback/pin tests flipped to accept non-loopback)pchronicle serve --catalog-config … --listen 127.0.0.1:8081should bind withoutmax_filesrecursive-list failurecatalog://<non-loopback-ip>:portand confirm it is accepted*/events.lancestill appears in catalog/projection inventoryMade with Cursor