Neural Algorithmic Reasoning: Architectures, Continuous Data Structures, and Length Generalization

Full report

On this page

The Transition from Bounded Correlations to Executable Algorithms

Historically, the success of deep learning has been predicated on the ability of continuous neural networks to interpolate within the highly structured, high-dimensional distributions of their training data. By mapping noisy inputs to statistical correlations, these models achieve remarkable performance on tasks ranging from computer vision to natural language processing. However, traditional neural architectures exhibit a catastrophic vulnerability: they reliably fail at out-of-distribution (OOD) extrapolation. When tasked with processing inputs that are structurally more complex or significantly larger than those encountered during training, models dependent on bounded correlations suffer immediate performance collapse1. Classical rule-based algorithms, conversely, operate on an entirely different paradigm. An algorithm—whether sorting an array, finding a shortest path, or traversing a tree—possesses perfect length generalization and provable correctness across arbitrary input scales1. The limitation of classical algorithms is their rigidity; they require strictly formalized, discrete inputs and cannot natively process raw, noisy, or ambiguous data. For instance, converting a complex, dynamic real-world environment like urban traffic congestion into a set of discrete scalar edge weights for a shortest-path algorithm results in massive information loss, rendering the perfectly optimal algorithmic solution practically useless for the original noisy environment2. Neural Algorithmic Reasoning (NAR) has emerged as the synthesis of these two diametrically opposed paradigms. The discipline seeks to train continuous neural networks to internalize and mimic the step-by-step execution of classical algorithms2. Rather than learning bounded statistical correlations, an algorithmic reasoner learns executable operations—such as pointer manipulation, graph traversal, memory allocation, and dynamic programming2. By remaining in a continuous, high-dimensional space, the network avoids the information loss associated with manual discretization while gaining the ability to extrapolate its learned logic to vastly larger problem instances2. The standard implementation of this paradigm compares a learned or heuristic graph controller against exact classical algorithms. During this process, the framework records the exact algorithmic invariants—intermediate computational states—to supervise the controller. To prove that actual executable logic has been acquired, the heuristic controller is subsequently tested on much larger inputs than those used during development, a process formally known as test-small/train-large extrapolation2. To realize this vision, the field has evolved from basic recurrent networks toward sophisticated neural execution engines equipped with differentiable memory tapes, continuous data structures, topology-aware message-passing mechanisms, and adaptive computation times.

Architectures for Universal Computation: Neural Execution Engines

The foundational requirement for neural algorithmic execution is the decoupling of a network's computational logic from its working memory. Standard Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks are theoretically Turing-complete, possessing the mathematical capacity to simulate arbitrary procedures8. In practice, however, their distributed hidden states act as entangled bottlenecks. When forced to execute long-running algorithms, these networks suffer from catastrophic forgetting, numerical drift, and compounding errors8. A true neural execution engine requires a compartmentalized, explicitly addressable memory system.

The Neural Turing Machine and Differentiable Tapes

The Neural Turing Machine (NTM) introduced a fully differentiable analogue to the von Neumann architecture, enabling networks to interact with external memory resources. The architecture couples a neural network controller (typically an LSTM or a feedforward network) to an external memory matrix [Figure omitted from source export], where [Figure omitted from source export] represents the number of memory locations and [Figure omitted from source export] represents the vector dimensionality at each location8. Because discrete memory addressing is non-differentiable and blocks gradient descent, the NTM utilizes "blurry" attention distributions to interact with all memory locations simultaneously. During a read operation, the read head produces a continuous read vector [Figure omitted from source export] as a convex combination of all memory rows, weighted by an attention distribution [Figure omitted from source export]: [Figure omitted from source export] Writing is decomposed into a differentiable erase step and an add step. The controller emits an erase vector [Figure omitted from source export] and an add vector [Figure omitted from source export], modifying the memory matrix dynamically: [Figure omitted from source export] [Figure omitted from source export] The NTM employs both content-based addressing—retrieving information based on cosine similarity to a search key—and location-based addressing, allowing it to iterate through memory locations sequentially, akin to a Turing machine reading and writing along a tape8. Preliminary tests proved that NTMs could infer algorithms like sequence copying, sorting, and associative recall purely from input-output examples9. However, the NTM struggled with complex sequential tracking, dynamic memory de-allocation, and resolving addressing conflicts over long computational horizons14.

The Differentiable Neural Computer and Memory Linkage

The Differentiable Neural Computer (DNC) advanced the NTM framework by introducing mechanisms for temporal memory linkage and dynamic allocation16. Algorithms frequently require traversing data structures in the exact order they were constructed, regardless of where the data resides in physical memory. To achieve this, the DNC maintains a temporal linkage matrix [Figure omitted from source export], which tracks the transition probabilities between consecutively written memory locations. If location [Figure omitted from source export] was written to immediately after location [Figure omitted from source export], [Figure omitted from source export] approaches [Figure omitted from source export]15. This associative matrix allows the network to project address distributions forward or backward in time, natively recovering sequences irrespective of the elapsed time between writes18. Furthermore, the DNC includes dynamic memory allocation. The controller computes a "usage" level for each memory location, allowing it to identify and overwrite the least-used addresses. A free gate dictates when memory should be de-allocated, preventing the memory matrix from filling to capacity and enabling the continuous execution of long-running algorithms16. Despite these architectural improvements, large differentiable memory matrices remain susceptible to noise accumulation. The lack of strict key-value separation in content-based addressing often results in flat, noisy attention distributions, because the stored value inadvertently influences the similarity score calculation intended only for the key15. Additionally, chained memory reads via the temporal linkage matrix can exponentially degrade addressing precision over time, limiting the reliable depth of algorithm execution15.

Differentiable Data Structures and the Chomsky Hierarchy

To move beyond unstructured memory matrices and model specific algorithmic patterns accurately, neural architectures must adopt the inductive biases of classical data structures such as stacks, queues, and deques. A continuous, differentiable stack must support recursive computations and Last-In-First-Out (LIFO) memory retrieval while remaining fully accessible to end-to-end backpropagation10. The incorporation of these structures maps directly to the Chomsky hierarchy, effectively upgrading an RNN to the equivalent of a Pushdown Automaton, thereby enabling the parsing of context-free languages10.

Stratification versus Superposition

Differentiable stacks and queues generally follow two design philosophies: superposition and stratification. In the superposition approach, stack operations (push, pop, no-op) are treated as mutually exclusive probabilities. The network computes three separate potential stack state matrices corresponding to each specific action and interpolates among them using the predicted operation probabilities25. The stratification approach, conversely, treats fractional weights as physical "thicknesses" or degrees of presence. In this continuous stack formulation, a controller emits a continuous push value [Figure omitted from source export] and a pop value [Figure omitted from source export] alongside a value vector [Figure omitted from source export]22. The strength of the element at index [Figure omitted from source export] at time [Figure omitted from source export], denoted as [Figure omitted from source export], is updated by traversing the stack from top to bottom and subtracting the pop scalar [Figure omitted from source export] from the topmost elements until the deletion quantity is mathematically exhausted: [Figure omitted from source export] A read operation retrieves a weighted sum of the stack's values scaled by these remaining continuous strengths. A differentiable queue operates via identical principles, except the pop operation consumes the lowest-indexed (oldest) strengths from the front of the sequence rather than the highest-indexed ones22.

Nondeterministic and Tree-Structured Stacks

Standard differentiable stacks are strictly deterministic; they simulate a single sequence of stack operations at each time step. The Nondeterministic Stack RNN (NS-RNN) breaks this limitation by adapting Lang's algorithm for simulating nondeterministic pushdown automata. The NS-RNN explicitly maintains an exponential number of possible stack configurations simultaneously, utilizing cubic time and quadratic space complexity10. By assigning arbitrary unnormalized positive weights instead of strict probability distributions to stack actions, the NS-RNN effectively models highly complex context-free languages and algorithmic patterns that deterministic structures fail to recognize10. Furthermore, recursive algorithms that manipulate hierarchical structures require tree-structured memory. Architectures such as the Tree Stack Memory Unit (Tree-SMU) embed a differentiable stack into every individual node of a recursive neural network. Each node learns soft push and pop operations that gate the combined states and stacks of its children, passing integrated historical contexts upward through the tree topology27. Similar mathematical motivations drive the Recursive Transformer, which augments standard self-attention layers with differentiable stack operations. By explicitly incorporating hidden state stacks between Transformer layers, the model routes token-level representations through push and pop mechanisms, granting Large Language Models the ability to recognize deterministic context-free grammars—a known failure point for standard attention mechanisms23.

Heuristic Graph Controllers and Algorithmic Alignment

While recurrent controllers equipped with external stacks or tapes effectively simulate sequential Turing machines, a vast majority of classical algorithms—ranging from dynamic programming and greedy search to divide-and-conquer—are most naturally expressed as operations over sets of objects and their topological relations. Consequently, Graph Neural Networks (GNNs) have become the de facto processor architectures, serving as heuristic graph controllers for Neural Algorithmic Reasoning29. The efficacy of a GNN in executing an algorithm is governed by the principle of "algorithmic alignment." The core hypothesis dictates that a neural network will exhibit lower sample complexity and superior test-small/train-large extrapolation if its internal architectural components map directly to the computational subroutines of the target algorithm31.

GNNs as Dynamic Programmers

Theoretical frameworks utilizing category theory and abstract algebra have proven that message-passing GNNs strictly align with dynamic programming (DP) algorithms32. Dynamic programming breaks a global problem into a graph of localized subproblems. The Bellman-Ford algorithm for single-source shortest paths perfectly illustrates this algebraic alignment. The Bellman-Ford algorithm operates over a graph [Figure omitted from source export] by iteratively relaxing edge weights. This process is formalized mathematically within the tropical "min-plus" semiring [Figure omitted from source export]. In this semiring, the standard addition operation is replaced by the [Figure omitted from source export] operator, and standard multiplication is replaced by addition33. The shortest path distance update for a node [Figure omitted from source export] from its neighbors [Figure omitted from source export] is given by: [Figure omitted from source export] A Message-Passing Neural Network (MPNN) performs an analogous operation in the real-number space. The message function generates continuous representations corresponding to the edge additions, and the permutation-invariant aggregation function (e.g., element-wise maximum or minimum) aligns directly with the semiring's [Figure omitted from source export] operation33. By demonstrating that both GNN computations and DP algorithms can be diagrammatically abstracted into pullbacks and pushforwards over latent spaces, researchers have established a rigorous theoretical foundation explaining why GNNs effortlessly extrapolate dynamic programming logic across varying graph sizes32.

Overcoming Expressivity Limits: The 1-WL Boundary

Despite their near-perfect alignment with dynamic programming, standard GNNs suffer from fundamental expressivity limits. The distinguishing power of any standard MPNN is strictly upper-bounded by the 1-dimensional Weisfeiler-Lehman (1-WL) graph isomorphism test38. A standard GNN cannot learn to distinguish certain symmetric graph structures (e.g., distinguishing a cycle of six nodes from two disjoint cycles of three nodes). This mathematically limits its ability to execute algorithms requiring complex, non-local topological understanding.

Pointer Graph Networks (PGNs)

To overcome the 1-WL limits and execute algorithms that rely on dynamic pointer manipulation—which inherently violate strict local message-passing paradigms—the Pointer Graph Network (PGN) was introduced40. GNNs typically operate on a static graph structure presumed to be known upfront. PGNs dynamically infer latent graph structures by augmenting the node set with adaptive pointers. At each computational step, each node dynamically points to another node, creating an adjacency matrix [Figure omitted from source export]. Message passing is subsequently executed over these inferred, asymmetric pointer edges30. PGNs are directly supervised on the intermediate execution steps of pointer-based data structures, enabling the network to simulate complex, non-local algorithms like disjoint-set unions (DSU) and link/cut trees30. This dynamic, data-driven rewiring grants PGNs significant OOD generalization advantages over standard static GNNs, particularly on dynamic connectivity tasks30.

Triplet-GMPNNs and Edge-Aware Reasoning

For highly complex reasoning—such as string matching algorithms (e.g., Knuth-Morris-Pratt), computational geometry, and complex graph logic like the Floyd-Warshall algorithm—reasoning over nodes and single edges is insufficient45. Standard local message passing (1-WL bounded) catastrophically fails on string matching due to the inability to contextualize long-range sub-string invariants. The Triplet-GMPNN architecture extends the MPNN framework to perform edge-aware attention and message passing across triplets of vertices. By computing edge latents and aggregating multi-hop triplet messages (effectively operating on a [Figure omitted from source export] tensor structure), the network bypasses 1-WL limitations45. This edge-level reasoning significantly improves the extraction of intermediate node effects, yielding state-of-the-art results on the most difficult subsets of algorithmic tasks45.

Adaptive Computation Time and Neural Halting

A fundamental characteristic of classical algorithms is that their execution time is a dynamic function of the input complexity, not merely the input size. Standard deep neural networks deploy a fixed computational depth (a static number of layers or recurrent unrolls) regardless of whether the problem is trivial or requires extensive iterative routing49. In Neural Algorithmic Reasoning, fixing the number of recurrent GNN steps artificially restricts the model's ability to complete algorithms on larger graphs, directly undermining length generalization and test-small/train-large extrapolation.

The Limits of ACT and the PonderNet Solution

Early attempts to implement dynamic depth introduced Adaptive Computation Time (ACT), which attached a halting unit to RNNs. ACT estimated a halting probability and penalized "ponder time" to encourage the network to solve problems quickly50. However, ACT computed the final prediction as a weighted average across all computational steps. This caused representation blurring, destroying the solver's precision, and frequently caused the network to fail entirely due to extreme sensitivity to hyperparameter tuning50. PonderNet resolves these issues by reformulating adaptive computation as a probabilistic halting process rather than a weighted average. At each step [Figure omitted from source export], the network produces a prediction [Figure omitted from source export], a hidden state [Figure omitted from source export], and a conditional probability of halting [Figure omitted from source export]50. PonderNet defines a Bernoulli random variable for halting, converting the conditional probabilities into an unconditioned probability distribution [Figure omitted from source export] indicating that the network halts exactly at step [Figure omitted from source export]: [Figure omitted from source export] Crucially, during inference, the final prediction is simply the output from the exact step at which the network decides to halt, completely avoiding the destructive averaging of intermediate states50. The training objective enforces efficient resource allocation without aggressively suppressing computation. The loss function is a combination of a reconstruction loss and a regularization loss: [Figure omitted from source export] [Figure omitted from source export] The regularization term computes the Kullback-Leibler (KL) divergence between the network's learned halting distribution and a geometric prior distribution parameterised by [Figure omitted from source export]. This mechanism encourages the network to explore different step counts and acts as a form of Occam's razor. By effectively decoupling computation time from fixed architectures, networks augmented with PonderNet-style halting achieve near-perfect extrapolation accuracy on longer sequences by autonomously increasing their pondering steps49.

Deep Equilibrium Models for Algorithmic Reasoning

An alternative to step-by-step halting is recognizing that many classical algorithms converge to a steady state. For algorithms such as single-source shortest paths, minimum spanning trees, and sorting, further iterations beyond the optimal solution do not alter the final output values—the algorithm has reached an equilibrium53. Instead of forcing a recurrent GNN to align precisely with the step-by-step trajectory of an algorithm, recent methodologies employ Deep Equilibrium (DEQ) models. DEQ models search directly for the fixed-point equilibrium of the network's transformation function. By aligning the network to the algorithm's termination properties rather than its intermediate loop executions, the model avoids matching each GNN iteration with a step of the algorithm. This fundamentally reduces the required number of forward iterations and circumvents the vanishing gradient issues associated with deep unrolling53.

Benchmarking Extrapolation: The CLRS-30 Framework

The unification of algorithmic reasoning research required a standardized testbed capable of isolating inductive biases and verifying strict out-of-distribution length generalization. The CLRS Algorithmic Reasoning Benchmark (CLRS-30) fulfills this role, providing an exhaustive suite of 30 classical algorithms adapted from the canonical Introduction to Algorithms text (Cormen, Leiserson, Rivest, and Stein)6.

The Encode-Process-Decode Paradigm and Invariant Recording

The CLRS-30 benchmark evaluates algorithms divided into eight core categories: Sorting, Searching, Graph Algorithms, Dynamic Programming, String Matching, Computational Geometry, Greedy, and Divide & Conquer6. All tasks are mapped to a universal graph-based representation. Elements like array values or string characters are encoded as node features [Figure omitted from source export], while structural relationships (e.g., adjacency or predecessor pointers) are encoded as edge features [Figure omitted from source export]6. Models operating on the benchmark follow a strict encode-process-decode architecture29:

1. Encoder: Linear layers map raw input features (scalars, categoricals, pointers) into high-dimensional latent vectors representing nodes, edges, and global graph states.

2. Processor: A heuristic graph controller (usually a GNN, PGN, or Triplet-GMPNN) performs sequential message passing across the graph, iteratively updating latent representations based on topological context.

3. Decoder: Linear readouts extract algorithmic predictions from the final or intermediate states.

To enforce algorithmic alignment and ensure the model learns true executable logic rather than mere input-output mapping, CLRS-30 employs trajectory-based supervision. The framework records the exact algorithmic invariants—intermediate computational states—at every step of the algorithm's execution. These invariants are exposed to the neural network as "hints." For example, during insertion sort, hints record the dynamic rewiring of predecessor pointers at each iteration; during Bellman-Ford, hints track the evolving distance matrices6. The network is explicitly supervised to predict these hints, mathematically binding the continuous weights of the controller to the discrete logic of the exact classical algorithm29.

Test-Small, Train-Large Extrapolation Metrics

The defining challenge of CLRS-30 is testing length extrapolation. Models are trained on small, 16-node graphs (in-distribution) and evaluated on structurally diverse 64-node graphs (out-of-distribution)7. Evaluation is primarily measured via the micro-averaged F1 score for discrete classification and pointer tracking probes, and mean squared error for regression targets6. When transitioning from in-distribution validation to OOD test sets, the performance of neural architectures degrades. However, analysis reveals that specific inductive biases dictate where models succeed or fail across the algorithmic taxonomy.

Note: Data synthesized from CLRS-30 baseline benchmarking37. The benchmarking indicates that MPNNs naturally align with standard graph and greedy algorithms due to the algebraic parallels between message passing and dynamic programming44. Pointer Graph Networks (PGNs) emerge as highly robust generalists for OOD tasks, achieving broad superiority on Divide & Conquer tasks by actively modifying connectivity to match the algorithm's control flow, tying or winning on 14 out of 30 algorithms44. Conversely, traditional architectures fail almost completely on string-matching tasks, an issue later rectified by the Triplet-GMPNN. By leveraging multi-hop, edge-based triplet reasoning, the Triplet-GMPNN breaks the 1-WL expressivity barrier, achieving up to a 30% improvement in string algorithm OOD performance compared to standard baselines45. Furthermore, recurrent NAR (RNAR) architectures exploiting natural node ordering have successfully mastered historically resistant sorting tasks like Heapsort and Quickselect59.

Enhancing OOD Generalization and Invariant Mechanisms

Even with highly aligned processors like Triplet-GMPNNs, neural reasoners exhibit a dangerous tendency to memorize implicit correlations within the training size distribution. The network learns unintended shortcut features—such as graph size artifacts or specific degree distributions—rather than the intended algorithmic logic. When the input size scales up during test-large extrapolation, these spurious correlations break, collapsing the model's accuracy5.

Hint-ReLIC and Causal Regularization

To suppress size-dependent memorization without altering the GNN architecture, researchers introduced Hint-ReLIC (Representation Learning via Invariant Causal mechanisms). Hint-ReLIC leverages the mathematical reality that, across a vast distribution of different algorithmic inputs, intermediate computations often converge to identical substates61. For instance, adding linear transformations to edge weights in a shortest path problem does not alter the ultimate Dijkstra traversal path5. By analyzing the causal graph of an algorithm's execution, Hint-ReLIC utilizes targeted data augmentations that produce different raw inputs guaranteed to yield the exact same next-step execution trajectory. The network is then trained with a contrastive self-supervised objective, enforcing that the latent representations of these differing inputs must remain invariant if their resulting algorithmic steps are identical5. This technique forces the neural network to ignore graph-size artifacts and focus solely on relative ordering and local topology, yielding up to a 3x improvement in OOD micro-F1 scores on the CLRS benchmark61.

Modeling Multiple Correct Solutions and Tie-Breaking

Another significant bottleneck in training continuous execution engines is the assumption of deterministic trajectories. Standard CLRS data sets generate training traces using deterministic tie-breaking. However, many classical algorithms possess multiple correct topological execution paths. In Depth-First Search (DFS) or shortest-path problems, randomized tie-breaking among edges of equal weight yields different, yet entirely valid, search trees4. When a neural network is penalized for outputting a valid solution simply because it differs from the single deterministic ground-truth trace in the training set, the loss landscape becomes artificially rugged, trapping models in local minima4. Advanced NAR frameworks address this by adapting the model input to reflect multiple correct solutions, converting the deterministic target into a probability distribution over valid predecessor-successor relationships4. The network is trained to predict the distribution of all valid solutions, and sophisticated extraction algorithms are applied to the output space to sample distinct, mathematically sound solution trees. This diversification is highly critical for real-world applications, such as cyberphysical routing systems, where returning a single rigid solution is brittle compared to providing a flexible distribution of valid routes4.

Open-Book Reasoning and Language Integration

The frontier of Neural Algorithmic Reasoning is currently expanding beyond isolated execution toward contextual, multi-task learning. Traditional NAR feeds isolated graph instances to the model. New frameworks introduce "open-book" neural reasoning, where the network is augmented with a supplementary memory component storing representations of various algorithms and instances from the training set66. During execution, an attention mechanism allows the network to dynamically query and aggregate logic from related tasks—for instance, retrieving sorting logic while attempting to solve a minimum spanning tree. This mimics a mathematician referencing a textbook of formulas, significantly reducing memory burden and providing a robust tool for interpretable multi-task training66. Furthermore, as the machine learning industry pivots heavily toward Large Language Models (LLMs), NAR principles are being mapped into textual domains. Benchmarks such as CLRS-Text translate high-dimensional graph execution traces into procedural text prompts. This bridging allows researchers to evaluate whether LLMs can learn rigorous, step-by-step length extrapolation rather than relying purely on semantic heuristics68. It fundamentally marries the robust invariances and provable correctness of algorithmic execution with the broad semantic reasoning capabilities of foundation models.

Conclusion

The pursuit of Neural Algorithmic Reasoning represents a fundamental shift in machine intelligence, moving beyond the brittle interpolation of bounded correlations toward the acquisition of true, executable logic. By explicitly designing architectures that align with the formal operations of theoretical computer science—employing continuous stacks, temporal linkage matrices, tropical semiring message-passing, and probabilistic halting—deep neural networks are graduating into robust neural execution engines. Architectures such as Pointer Graph Networks and Triplet-GMPNNs prove that topological flexibility and edge-aware reasoning are mandatory for escaping the expressivity bounds of standard models, allowing the execution of complex string matching and pointer manipulation. Meanwhile, probabilistic mechanisms like PonderNet decouple fixed-depth computation from input complexity, endowing networks with the adaptive pondering time necessary to solve harder problems. Evaluated rigorously through the exact trajectory supervision of the CLRS-30 benchmark, and stabilized through causal invariance techniques like Hint-ReLIC, these models conclusively demonstrate that test-small/train-large extrapolation is achievable. This fusion promises a future where neural networks can safely, predictably, and optimally govern complex, real-world cyberphysical systems without sacrificing the high-dimensional capabilities of deep learning.

Works cited

1. \[2105.02761\] Neural Algorithmic Reasoning \- arXiv, https://arxiv.org/abs/2105.02761

2. Neural algorithmic reasoning \- PMC \- NIH, https://pmc.ncbi.nlm.nih.gov/articles/PMC8276006/

3. arXiv:2211.00692v2 \[cs.LG\] 18 Mar 2023, https://arxiv.org/pdf/2211.00692

4. Neural Algorithmic Reasoning with Multiple Correct Solutions \- arXiv, https://arxiv.org/html/2409.06953v4

5. Neural Algorithmic Reasoning Without Intermediate Supervision, https://arxiv.org/html/2306.13411v1

6. CLRS-30: Algorithmic Benchmark \- Emergent Mind, https://www.emergentmind.com/topics/clrs-30-algorithmic-benchmark

7. CLRS Benchmark: Neural Algorithmic Reasoning \- Emergent Mind, https://www.emergentmind.com/topics/clrs-benchmark

8. Neural Turing Machines | alphaXiv, https://www.alphaxiv.org/abs/1410.5401

9. Neural Turing Machines arXiv:1410.5401v2 \[cs.NE\] 10 Dec 2014, https://arxiv.org/pdf/1410.5401

10. Learning Hierarchical Structures with Differentiable ... \- ResearchGate, https://www.researchgate.net/publication/354400894\_Learning\_Hierarchical\_Structures\_with\_Differentiable\_Nondeterministic\_Stacks

11. Paper page \- Neural Turing Machines \- Hugging Face, https://huggingface.co/papers/1410.5401

12. Evolving Neural Turing Machines for Reward-based Learning, https://sebastianrisi.com/wp-content/uploads/greve\_gecco16.pdf

13. Adaptive Computation with Elastic Input Sequence \- arXiv, https://arxiv.org/html/2301.13195v2

14. \[1410.5401\] Neural Turing Machines \- arXiv, https://arxiv.org/abs/1410.5401

15. improving differentiable neural computers through memory masking, https://openreview.net/pdf?id=HyGEM3C9KQ

16. Differentiable neural computer \- Wikipedia, https://en.wikipedia.org/wiki/Differentiable\_neural\_computer

17. Differentiable neural computers \- Google DeepMind, https://deepmind.google/blog/differentiable-neural-computers/

18. ARTICLE \- C. Lee Giles, https://clgiles.ist.psu.edu/IST597/materials/slides/papers-memory/2016-graves.pdf

19. Robust and Scalable Differentiable Neural Computer for Question, https://aclanthology.org/W18-2606.pdf

20. Improving Differentiable Neural Computers Through Memory ... \- arXiv, https://arxiv.org/abs/1904.10278

21. Memory Augmented Recursive Neural Networks \- arXiv, https://www.arxiv.org/pdf/1911.01545v3

22. arXiv:1506.02516v3 \[cs.NE\] 3 Nov 2015, https://arxiv.org/pdf/1506.02516

23. Recursive Transformer: Boosting Reasoning Ability with State Stack, https://neurips.cc/virtual/2025/poster/120124

24. Learning Context-free Languages with Nondeterministic Stack RNNs, https://aclanthology.org/2020.conll-1.41.pdf

25. Learning Hierarchical Structures with Differentiable ... \- OpenReview, https://openreview.net/references/pdf?id=O9Mk1uqXra

26. Learning Hierarchical Structures with Differentiable ... \- OpenReview, https://openreview.net/pdf?id=5LXw\_QplBiF

27. arXiv:1911.01545v5 \[cs.LG\] 16 Oct 2020, https://arxiv.org/pdf/1911.01545

28. Recognizing and Verifying Mathematical Equations using, https://ojs.aaai.org/index.php/AAAI/article/view/16634/16441

29. google-deepmind/clrs \- GitHub, https://github.com/google-deepmind/clrs

30. Pointer Graph Networks \- arXiv, https://arxiv.org/pdf/2006.06380

31. Graph Neural Networks are Dynamic Programmers \- alphaXiv, https://www.alphaxiv.org/abs/2203.15544v3

32. Graph Neural Networks are Dynamic Programmers \- OpenReview, https://openreview.net/references/pdf?id=UGivKi\_cV

33. Graph Neural Networks are Dynamic Programmers, https://proceedings.neurips.cc/paper\_files/paper/2022/file/8248b1ded388fcdbbd121bcdfea3068c-Paper-Conference.pdf

34. arXiv:2209.11142v2 \[cs.LG\] 3 Dec 2022, https://arxiv.org/pdf/2209.11142

35. \[2203.15544\] Graph Neural Networks are Dynamic Programmers, https://arxiv.org/abs/2203.15544

36. The Algebraic Path Problem for Graph Metrics, https://proceedings.mlr.press/v162/sanmarti-n22a/sanmarti-n22a.pdf

37. The CLRS Algorithmic Reasoning Benchmark, https://proceedings.mlr.press/v162/velickovic22a/velickovic22a.pdf

38. Weisfeiler–Leman at the margin: When more expressivity matters, https://arxiv.org/pdf/2402.07568

39. N \-WL: A New Hierarchy of Expressivity for Graph Neural Networks, https://iclr.cc/virtual/2023/poster/11218

40. \[2006.06380\] Pointer Graph Networks \- arXiv, https://arxiv.org/abs/2006.06380

41. Pointer Graph Networks \- Lune, https://luneresearch.com/papers/f77fbf52-511f-4d52-9e07-7adc07ff847c

42. Pointer Graph Networks, https://grlplus.github.io/papers/90.pdf

43. Pointer Graph Networks \- arXiv, https://arxiv.org/html/2006.06380v2

44. The CLRS Algorithmic Reasoning Benchmark \[Quick Review\] \- Liner, https://liner.com/review/clrs-algorithmic-reasoning-benchmark

45. \[2312.05611\] Triplet Edge Attention for Algorithmic Reasoning \- arXiv, https://arxiv.org/abs/2312.05611

46. Triplet Edge Attention for Algorithmic Reasoning \- arXiv, https://arxiv.org/html/2312.05611v1

47. Triplet Edge Attention for Algorithmic Reasoning \- OpenReview, https://openreview.net/pdf/8167595987388b8bdd96f444bc3bcddb2eae66e5.pdf

48. A Context-Enhanced Framework for Sequential Graph Reasoning, https://www.ijcai.org/proceedings/2024/0542.pdf

49. arXiv:2107.05407v2 \[cs.LG\] 2 Sep 2021, https://arxiv.org/pdf/2107.05407

50. PonderNet: Learning to Ponder \- arXiv, https://arxiv.org/html/2107.05407

51. End-to-end Algorithm Synthesis with Recurrent Networks \- arXiv, https://arxiv.org/html/2202.05826v3

52. PonderNet: Learning to Ponder \- alphaXiv, https://www.alphaxiv.org/abs/2107.05407

53. The Deep Equilibrium Algorithmic Reasoner, https://marworkshop.github.io/cvpr24/pdf/13.pdf

54. arXiv:2402.06445v2 \[cs.LG\] 9 Apr 2024, https://arxiv.org/pdf/2402.06445

55. Graph Neural Networks Are Dynamic Programmers \- ResearchGate, https://www.researchgate.net/publication/401462635\_Graph\_Neural\_Networks\_Are\_Dynamic\_Programmers

56. The CLRS Algorithmic Reasoning Benchmark \- Semantic Scholar, https://www.semanticscholar.org/paper/The-CLRS-Algorithmic-Reasoning-Benchmark-Velivckovi%27c-Badia/9332dd9afa389d8d273e1713a6e1ca98797c6e5f

57. The CLRS Algorithmic Reasoning Benchmark \- Zeta Alpha, https://www.zeta-alpha.com/post/competitive-coding-for-graph-neural-nets-and-beyond-the-clrs-algorithmic-reasoning-benchmark

58. Neural Algorithmic Reasoning Without Intermediate Supervision, https://proceedings.neurips.cc/paper\_files/paper/2023/file/a2370db7c99791ad5d9f3ef48ad6d464-Paper-Conference.pdf

59. \[2409.07154\] Recurrent Aggregators in Neural Algorithmic Reasoning, https://arxiv.org/abs/2409.07154

60. \[PDF\] Recurrent Aggregators in Neural Algorithmic Reasoning, https://www.semanticscholar.org/paper/Recurrent-Aggregators-in-Neural-Algorithmic-Xu-Veli%C4%8Dkovi%C4%87/50125145b0e4cfe42fc9b15822f1b3b0a5f978cc

61. Neural Algorithmic Reasoning with Causal Regularisation \- arXiv, https://arxiv.org/abs/2302.10258

62. Importance of hint usage in the final performance. The table shows, https://www.researchgate.net/figure/Importance-of-hint-usage-in-the-final-performance-The-table-shows-mean-and-stderr-of-the\_tbl1\_368688077

63. Neural Algorithmic Reasoning with Causal Regularisation, https://proceedings.mlr.press/v202/bevilacqua23a/bevilacqua23a.pdf

64. Neural Algorithmic Reasoning with Causal Regularisation \- arXiv, https://arxiv.org/pdf/2302.10258

65. Neural Algorithmic Reasoning with Multiple Correct Solutions \- arXiv, https://arxiv.org/html/2409.06953v1

66. Open-Book Neural Algorithmic Reasoning \- arXiv, https://arxiv.org/html/2501.00072v1

67. Open-Book Neural Algorithmic Reasoning \- arXiv, https://arxiv.org/pdf/2501.00072

68. The CLRS-Text Algorithmic Reasoning Language Benchmark \- arXiv, https://arxiv.org/abs/2406.04229

69. (PDF) The CLRS-Text Algorithmic Reasoning Language Benchmark, https://www.researchgate.net/publication/381227056\_The\_CLRS-Text\_Algorithmic\_Reasoning\_Language\_Benchmark

70. The CLRS-Text Algorithmic Reasoning Benchmark \- ICML 2026, https://icml.cc/virtual/2024/36420