From Interface to Inference:
Eliciting Any-Order Inference from Any-Order Models

When does an any-order prediction interface lead to genuinely any-order inference?

Seunggeun Kim*1, Jaeyeon Kim*2, Taekyun Lee*1, Yuyuan Chen*2, Yilun Du2, Sham Kakade2, Sitan Chen2
*Co-first authors 1University of Texas at Austin 2Harvard University
Paper Code

Introduction

Many discrete reasoning tasks, such as code generation, are inherently non-causal. Programmers move between high-level structure and local details, revise earlier decisions, and insert missing pieces. Autoregressive language models decode left-to-right and therefore require hand-designed mechanisms to support these behaviors.

Masked diffusion models (MDMs) are natural candidates for any-order inference: given a partially masked sequence, they can predict clean tokens at arbitrary masked positions. However, we show that this any-order prediction interface does not automatically yield genuinely any-order inference. We identify the bottleneck and propose two complementary ways to overcome it.

Overview of the interface-to-inference gap and our two solutions

The Interface-to-Inference Gap

Confidence-based decoding is the dominant inference strategy for MDMs: at each step, it reveals the positions where the model is most confident. Although this policy is not explicitly left-to-right, we find that its final programs are structurally similar to programs generated by strict left-to-right decoding.

  • We parse generated Python programs into abstract syntax trees (ASTs) and compare their global structure.
  • Across DiffuCoder and Dream variants, confidence-based policies find programs much closer to the left-to-right reference set than random decoding does.
  • The any-order interface therefore often collapses to a familiar causal generation strategy.

AST-based structural similarity to left-to-right decoding

Confidence-based decoding yields code structurally close to left-to-right decoding.

Positional Uncertainty

We identify positional uncertainty as a fundamental bottleneck of token-level, fixed-canvas MDMs. A model may know what semantic component should appear next while remaining uncertain about where to place it. For example, the model may know that a program needs a fallback return False, but its correct position can remain ambiguous until the surrounding structure is resolved.

  • High aggregate mass: a token is globally likely across the remaining masked canvas.
  • Low localization: that probability is dispersed across several plausible positions.
  • Because confidence decoding ranks individual positions, it favors locally easy continuations over globally meaningful but positionally dispersed components.

Mass and localization plots showing positional uncertainty

Many high-mass tokens remain undecoded because their probability is dispersed across positions.

1) Insertion-Based Masked Diffusion

Our first solution builds on FlexMDM, which alternates between revealing masked tokens and inserting new masks. Unmasking no longer commits a token to one final absolute position, because later insertions can shift its location in the completed sequence.

What Is New in Our FlexMDM?

  • Flexible inference schedules. A time-reparameterization method allows the insertion schedule to be changed at inference time without retraining the model.
  • More stable insertion prediction. We reformulate insertion-length prediction in log space, which provides a more natural geometry and improves training stability.
  • General-purpose code generation. Instead of task-specific LoRA fine-tuning, we fully fine-tune Dream-Coder 7B on 2.2M Python sequences.
Metrics showing FlexMDM generation across program-tree nodes
  • FlexMDM generates drafts across non-contiguous program regions and moves back and forth between unfinished AST nodes.
  • We fine-tune Dream-Coder 7B on 2.2M Python sequences to obtain a general-purpose Python FlexMDM.
  • FlexMDM outperforms Dream-Coder at Pass@16 on HumanEval and HumanEval+, although the gain is not uniform across every benchmark.

FlexMDM Generation Traces

Supplementary video. The generated sequence and its AST-tree trace evolve together, showing how FlexMDM moves across non-contiguous program regions.

BenchmarkModelPass@1Pass@4Pass@8Pass@16
HumanEvalDream-Coder58.6581.3486.9490.85
FlexMDM50.6578.6986.8692.07
HumanEval+Dream-Coder53.8975.9681.5385.37
FlexMDM46.4673.3981.4887.20

2) Latent-Space Masked Diffusion

Our second solution, LatentMDM, moves masked diffusion from token space to a continuous latent space. Rather than selecting individual token positions, the model chooses which masked semantic segment to decode next, enabling search over latent generation orders.

LatentMDM architecture with segment encoders, a latent masked diffusion model, and autoregressive segment decoders

LatentMDM applies masked diffusion to encoded semantic segments and decodes selected segments with a lightweight autoregressive decoder.

LatentMDM accuracy-latency results and positional uncertainty analysis

  • LatentMDM's confidence policy outperforms left-to-right and random segment-selection policies by more than 6 percentage points.
  • At matched wall-clock time, LatentMDM outperforms both token-level MDM and an autoregressive model by 5.5 percentage points.
  • One LatentMDM inference pass takes roughly half the latency of the autoregressive model, even with KV caching.
  • Moving to semantic segments relaxes positional uncertainty and enables a distinct form of genuinely any-order inference.

Takeaways

  1. An any-order training objective does not guarantee genuinely any-order inference.
  2. Fixed-canvas token models suffer from positional uncertainty: they must decide both what to generate and where to place it.
  3. Insertion-based MDMs enable generation across non-contiguous program structure.
  4. Latent-space MDMs enable search over semantic generation orders.

These two solutions are complementary rather than competing. FlexMDM stays in token space and can be retrofitted from a pretrained MDM, while LatentMDM enables segment-level order search but typically requires training from scratch. Together, they show that a model's interface only makes any-order inference possible; the inference procedure determines whether that possibility is realized.

BibTeX

@article{kim2026interface,
  title  = {From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models},
  author = {Kim, Seunggeun and Kim, Jaeyeon and Lee, Taekyun and Chen, Yuyuan and Du, Yilun and Kakade, Sham and Chen, Sitan},
  year   = {2026}
}