Skip to main content

Local LLMs Can Generate Structurally Valid X07 Programs

· 5 min read

Run a small local code model for a while and you notice the failures are rarely about logic. They are about structure. The model gets close, but the output does not parse, the imports drift, a block never closes, or the file shape just is not valid for the target language.

X07 sidesteps that. The canonical source format is x07AST JSON, and the toolchain can export both a JSON Schema and a grammar bundle describing that structure. The docs call that export surface Genpack.

The payoff for local models: constrained decoding can target the language's actual source form, not a best-effort textual approximation of it.

Why X07 works well with constrained decoding​

The target is not "some text that probably looks like code." It is a concrete JSON tree with a published schema, stable versions, and deterministic tooling around it. The docs expose:

  • x07 ast schema
  • x07 ast grammar --cfg
  • Python and TypeScript SDK surfaces under x07-genpack and @x07/genpack

So a local-model workflow splits cleanly:

  1. constrain the decoder so it can only emit structurally valid x07AST
  2. run X07's deterministic formatting and linting passes
  3. apply quickfixes where the toolchain can repair the result automatically

Recipe A: export the X07 schema and compile it with XGrammar​

note

These commands and host-language snippets are for generating X07 programs. Python is only the glue code around the decoder; the generated language is still X07.

# Export the canonical X07 AST schema to a local file.
x07 ast schema > /tmp/x07ast.schema.json
import pathlib  # Read the exported X07 schema from disk.
import xgrammar as xgr  # Compile decoding constraints for your local model.

schema_str = pathlib.Path("/tmp/x07ast.schema.json").read_text(encoding="utf-8")  # Load the X07 schema bytes as text.
compiler = xgr.GrammarCompiler(tokenizer_info=...)  # Bind the grammar compiler to your model's tokenizer.
compiled = compiler.compile_json_schema(schema_str)  # Produce a decoder constraint that only accepts valid x07AST structure.

Recipe B: export the X07 grammar bundle and use the min variant​

The grammar bundle is useful when you want the exact shipped grammar variants rather than schema compilation at runtime.

# Export the X07 grammar bundle with both shipped variants.
x07 ast grammar --cfg > /tmp/x07ast.grammar_bundle.json
import json  # Decode the X07 grammar bundle JSON.
import pathlib  # Read the exported bundle from disk.
import xgrammar as xgr  # Compile the selected grammar for your decoder.

bundle = json.loads(pathlib.Path("/tmp/x07ast.grammar_bundle.json").read_text(encoding="utf-8"))  # Load the full X07 grammar bundle.
min_cfg = next(v["cfg"] for v in bundle["variants"] if v["name"] == "min")  # Select the compact grammar variant for smaller or weaker models.
compiler = xgr.GrammarCompiler(tokenizer_info=...)  # Bind the grammar compiler to your tokenizer.
compiled = compiler.compile_grammar(min_cfg)  # Compile the X07 grammar into runtime decoding constraints.

The current X07 docs recommend the min variant for smaller models and higher-throughput decoding, while pretty is mainly for readability and debugging.

Recipe C: drive the same X07 schema through Outlines​

If your local workflow already uses Outlines, the same schema export works there too.

# Export the canonical X07 schema once for the Outlines generator.
x07 ast schema > /tmp/x07ast.schema.json
import pathlib  # Read the exported X07 schema from disk.
import outlines  # Build a schema-constrained generator.

schema_str = pathlib.Path("/tmp/x07ast.schema.json").read_text(encoding="utf-8")  # Load the X07 schema text.
generate = outlines.generate.json(model, schema_str)  # Ask Outlines to constrain generation to valid x07AST JSON.
ast = generate("Generate a valid x07AST module that echoes its input")  # Produce one structurally valid X07 program.

What the model is actually generating​

note

This tiny example is a real X07 entry program in canonical x07AST JSON form. The comments explain the structure for readers who are new to the language.

{
  "schema_version": "x07.x07ast@0.8.0", // Match the current X07 AST schema version.
  "kind": "entry", // Emit a runnable entry program.
  "module_id": "main", // Name the module that owns this entrypoint.
  "imports": [], // This minimal program does not need imported modules.
  "decls": [], // There are no helper declarations in this tiny example.
  "solve": ["view.to_bytes", "input"] // Return the input bytes unchanged.
}

JSON is what the decoder constrains against, so JSON is what stays on disk. If you want a readable view of the same program, x07 ast to-text projects it losslessly to x07text — here that solve becomes (view.to_bytes input).

The example is tiny on purpose, and it makes the point: the target is explicit structure, not free-form source text.

Converge deterministically after generation​

Constrained decoding gets you valid structure; it does not get you canonical, lint-clean output. That last step is deterministic.

# Canonicalize the generated X07 AST so later diffs are stable.
x07 fmt --input generated.x07.json --write --json

# Collect machine-readable diagnostics for any remaining issues.
x07 lint --input generated.x07.json --json

# Apply any JSON Patch quickfixes that the X07 toolchain can repair automatically.
x07 fix --input generated.x07.json --write --json

Inside a full X07 project, x07 run, x07 build, and x07 bundle already apply that repair loop automatically by default.

Why this matters more for local models​

Local models have less margin: fewer tokens to waste, less room for syntax retries. Targeting x07AST directly helps on every front:

  • the output grammar is narrower than a general-purpose text language
  • the source form is explicitly machine-editable
  • the toolchain returns structured diagnostics and quickfixes
  • the repair loop is deterministic

That is the whole idea behind Genpack. Not "prompt harder until the syntax looks right," but "make the language a first-class structured target for the decoder."