Expose OPTICS's reachability graph - #366
Merged
josevalim merged 2 commits intoAug 30, 2026
Merged
Conversation
compute_optics_graph already builds reachability, core_distances, ordering and predecessor, but fit/2 only kept labels and discarded the rest at the end of the function. These are what a reachability plot, the standard way to read an OPTICS run, is drawn from. core_distances is flattened before being returned: it carries a trailing size-1 axis from the neighbor search it is sliced out of, which the internal code accommodates through broadcasting but which has no reason to leak into the public field. No test file existed for this module, so one was added: doctest plus values checked against scikit-learn 1.6.1 on the moduledoc's own data.
Contributor
Author
|
Would it make sense to cut a new release soon? maybe v0.5.0 |
Contributor
|
We can do a new release soon. For this PR, can we improve the tests? They are converting too many values and asserting on shapes. Ideally we want to assert on values and rely Nx.all_close if possible. |
Contributor
|
Sorry, assert_all_close! |
RicardoSantos-99
force-pushed
the
add-optics-reachability-fields
branch
from
August 28, 2026 18:33
e7ba261 to
0db755b
Compare
Contributor
Author
Done |
RicardoSantos-99
added a commit
to RicardoSantos-99/verdict
that referenced
this pull request
Aug 29, 2026
OPTICS does not hand back clusters so much as an ordering with a distance attached to every point. The reachability plot is how that ordering is read: a valley is a cluster, the walls either side say how separated it is, and one picture replaces a run at several cut-offs. Both the distances and the labels are read through the model's ordering. They are stored in the order the points were given, and only mean something in the order OPTICS reached them, which is what puts each point beside the one it was reached from. Verified against scikit-learn 1.6.1, whose reachability_[ordering_] is the same arrangement and whose raw arrays Scholar matches value for value. A point OPTICS could not reach carries an infinite distance, which no axis can hold and which Jason encodes as the string "infinity" if it is left alone. Those are drawn as full-height dashed rules rather than bars, so the wall stays visible without a number being invented for it. The first point in the ordering is always one of them. The cut-off the clusters were extracted at is drawn as a reference, and takes a value of its own, which is how the next eps to try gets chosen without fitting again. It is left out when the model's own eps is infinite, since that has no position on the axis either. Needs Scholar's OPTICS to expose reachability, ordering and labels, which is elixir-nx/scholar#366.
Contributor
|
💚 💙 💜 💛 ❤️ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scholar.Cluster.OPTICS.fit/2computes the full reachability graph but only keeps the labels.compute_optics_graphbuildscore_distances,reachability,predecessorandordering, thenfit_pfeeds them intocluster_optics_dbscanand discards them at the end of the function.That graph is not an internal detail.
orderingagainstreachabilityis the reachability plot, the standard way OPTICS results are read and the way clusters are told apart from noise visually, and it is whatpredecessorandcore_distancessupport (tracing how a point was reached, and at what radius it became a core point). None of it survives pastfit/2today, so nothing built on Scholar can draw one.This adds
:reachability,:core_distances,:orderingand:predecessorto the struct, populated the same way:labelsalready is.