Repository navigation
Generate Term encoders for Rust types read from source - #1
Merged
Merged
Conversation
RustQ.Syn.Enum gains variant_shapes, a RustQ.Syn.Variant per variant with its kind (unit, tuple, or named), fields, and doc comments, so generators can follow enum payload types from real Rust source. RustQ.Syn.Struct and RustQ.Syn.Enum also report lifetimes and type_parameters, matching function and method metadata. The new fields are defaulted; variants still lists variant names.
Field access, struct literal and struct pattern fields, and macro item arguments built their identifiers with format_ident!, which cannot produce a keyword, so value.type failed to render. They now go through ident_from_part, which emits r#type, and the Elixir renderer escapes macro arguments the same way. ident_from_part also accepts names that are already spelled r#type.
RustQ.Rustler.Term.encoders_from_source/3 follows field and variant types from root types through a RustQ.Syn index and builds one encoder function per reachable struct or enum, so crates can encode types they do not own and that do not implement serde. encoder_atoms_from_source/3 returns the matching atom declarations. Wrapper roles, external helpers, the variant tag, and per-type renames and exclusions are explicit options. A reachable type that is not indexed, a wrapper, a scalar, or external fails generation with a list of the unmapped types. Also add Syn.Index.structs/1, struct/2, struct!/2 and PatternBuilder.tuple/1. The public consumer fixture generates encoders for a small IR and its package test now runs clippy.
Found while generating vize_ex's Vapor IR encoders:
- A variant tag that would replace a payload field with the same key
(CreateComponentIRNode has its own kind field) now fails generation
instead of silently overwriting the field. The public consumer
fixture had this bug and now renames the field.
- transparent: true encodes a struct with exactly one field as that
field, so IREffect { operations } can stay a list.
- Unmapped and unsupported types are reported once, with every type
that uses them.
A literal atom such as :if_node in a map field was emitted as a Rust type named after the atom (pub kind: if_node), which does not compile. Lowercase atoms that are not Rust primitive names are Elixir literals, so they now become Atom, like a union of atoms. Rust type names written as atoms (u32, MyType) are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generates Rustler
Termencoders for Rust types read from source. This covers types owned by another crate that don't implement serde, such asvize_atelier_vapor's IR, which vize_ex currently encodes by hand.RustQ.Rustler.Term.encoders_from_source/3Starting from root types, it follows field and variant payload types through a
RustQ.Synindex and builds one free function per reachable struct or enum:tag:, data-carrying variants also carry the variant atom.OptionencodesNoneasnil, sequences and sets as lists, and maps as maps.Vec/Box),external:helpers, and per-typeexcept:, key and variant renames, andwith:helpers.tag:that would replace a payload field with the same key fails generation instead of silently overwriting it.transparent: trueencodes a struct with exactly one field as that field.encoder_atoms_from_source/3returns the atom declarations from the same source.The feature is documented in a new "Encoders for types from Rust source" section of the Rustler generation guide.
Supporting changes
RustQ.Syn.Enum.variant_shapesgives oneRustQ.Syn.Variantper variant, with its kind, fields, and docs.lifetimesandtype_parametersare added onSyn.StructandSyn.Enum. All are defaulted, andvariantsis unchanged.RustQ.Syn.Index.structs/1,struct/2, andstruct!/2.RustQ.Rust.AST.PatternBuilder.tuple/1, for the existingPatTuplenode.required(:kind) => :if_node) is typed asAtom. It used to be emitted as a Rust type namedif_node.value.type), keyword struct literal and pattern fields, and keyword macro arguments now render as raw identifiers (r#type) in both renderers. Previously field access failed to render. The existingr#typespelling keeps working.Verification
mix cipasses: 802 tests, credo, dialyzer, ExDNA, Reach, format,rust.fmt,rust.check,rust.clippy,rustq.gen --check, corpus, and templates.Syn.Indexinput, and every error path. Direct render tests cover the keyword fix andPatternBuilder.tuple/1.Box,Vec,Option,HashMap, a keyword field, a newtype, and all variant kinds. Its package test now also runscargo clippy -- -D warnings, which is clean.First real consumer
vize_ex's Vapor IR encoders were generated with this branch, replacing its handwritten
vapor_irencoders. The generated code compiles clippy-clean with-D warnings. Across 18 templates covering every IR operation, the output kept every existing key and value, with 0 removed keys and 0 changed values, and it now also includes the IR fields the handwritten code dropped. The tag collision check andtransparent:came out of that migration.