continuous-depth

Here are 2 public repositories matching this topic...

PeterJemley / Continuous-Depth-Transformers-with-Learned-Control-Dynamics

Hybrid transformer architecture replacing discrete layers with Neural ODE blocks, enabling continuous semantic control (sentiment, style) at inference time via learned steering signals

nlp deep-learning transformers text-generation pytorch interpretability neural-ode controllable-generation continuous-depth

Updated Jan 30, 2026
Jupyter Notebook

zaydabash / ContinuumLM

Star

Neural ODE Transformer in Julia, trained via adjoint sensitivity methods. Matched-architecture vs. discrete Transformer on Penn Treebank: 119.9 val perplexity vs 113.7 discrete. Single-seed result; discrete wins at this scale as expected, continuous-depth advantage expected to emerge at larger scale.

machine-learning natural-language-processing deep-learning julia text-generation transformer neural-networks tensorboard differential-equations language-model continuous-time adjoint-method neural-ode sciml kv-cache autoregressive-generation flux-jl continuous-depth

Updated Jun 30, 2026
Julia

Add this topic to your repo

To associate your repository with the continuous-depth topic, visit your repo's landing page and select "manage topics."

Learn more