Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
d1947f7
VM exec loop
antoniosarosi Jun 19, 2025
1088232
Merge branch 'canary' into antonio/vm
antoniosarosi Jun 20, 2025
f45e7a5
Add compiler crate
antoniosarosi Jun 20, 2025
b639121
Merge branch 'canary' into antonio/vm
antoniosarosi Jun 24, 2025
6315297
Call functions
antoniosarosi Jun 27, 2025
db931e8
Recursively compile exprs
antoniosarosi Jun 27, 2025
32a6ca0
Fix instruction pointer
antoniosarosi Jun 27, 2025
ce28002
Limit allocations to `Vm::objects` only
antoniosarosi Jun 27, 2025
0625258
Compiler stuff
antoniosarosi Jun 27, 2025
1fdc7ee
Implement if else statements
antoniosarosi Jun 30, 2025
ed2af60
Reuse `ast`
antoniosarosi Jun 30, 2025
191e567
Add debugger
antoniosarosi Jul 1, 2025
cbd6ae8
Refactor compiler
antoniosarosi Jul 1, 2025
eec9310
Fix linebreaks
antoniosarosi Jul 1, 2025
2cafe79
Param name
antoniosarosi Jul 1, 2025
5cef042
Document skip else patch jump
antoniosarosi Jul 1, 2025
a2395c5
Add bytecode display function
antoniosarosi Jul 1, 2025
46b1fe4
Implement array constructors
antoniosarosi Jul 2, 2025
8e7d515
Merge branch 'canary' into antonio/vm
antoniosarosi Jul 2, 2025
f490c35
Docs for frame copy
antoniosarosi Jul 2, 2025
7cebfe4
Parse numbers correctly
antoniosarosi Jul 2, 2025
23fcb79
Refactor compiler tests
antoniosarosi Jul 2, 2025
57bbc75
Change some names
antoniosarosi Jul 2, 2025
9465065
Refactor VM tests
antoniosarosi Jul 2, 2025
247d174
Add bytecode dissassembly
antoniosarosi Jul 2, 2025
d4e4c82
Add fn name on dissassembly
antoniosarosi Jul 2, 2025
48b3042
Impl classes, instances and spread
antoniosarosi Jul 8, 2025
3a5f614
TODO comment
antoniosarosi Jul 8, 2025
0ba991b
store strings in the vm (#2114)
imalsogreg Jul 9, 2025
7e6ad37
Compile expr blocks
antoniosarosi Jul 9, 2025
cf81627
Refactor indexes
antoniosarosi Jul 11, 2025
e234b66
Restore if.baml
antoniosarosi Jul 11, 2025
32cad57
Merge branch 'canary' into antonio/vm
ba11b0y Jul 14, 2025
9209c65
Fix compiler not compiling
antoniosarosi Jul 15, 2025
9ae0876
add source line numbers when displaying bytecode (#2132)
ba11b0y Jul 15, 2025
f686af2
checkpoint
ba11b0y Jul 15, 2025
338a18c
Merge remote-tracking branch 'refs/remotes/origin/antonio/vm' into an…
antoniosarosi Jul 16, 2025
9cf2cf1
pretty print bytecode (#2139)
ba11b0y Jul 16, 2025
3c1ca3f
Refactor `display_bytecode` and add line breaks
antoniosarosi Jul 16, 2025
5f13da5
Merge branch 'canary' into antonio/vm
antoniosarosi Jul 16, 2025
a5491ec
Merge branch 'canary' into antonio/vm
antoniosarosi Jul 16, 2025
c3ecdb7
Remove for loops from top level assignments
antoniosarosi Jul 16, 2025
5bcc967
Merge branch 'canary' into antonio/vm
antoniosarosi Jul 16, 2025
927e48d
for loops implementation (#2117)
ba11b0y Jul 16, 2025
0b9c1b2
Merge `canary`
antoniosarosi Jul 16, 2025
980a1ca
Reorder instructions by type
antoniosarosi Jul 16, 2025
3226e68
Fix order
antoniosarosi Jul 16, 2025
9e73a13
checkpoint
ba11b0y Jul 16, 2025
e37705e
add a separator to maps
ba11b0y Jul 16, 2025
d642cd5
use existing COLON keyword
ba11b0y Jul 16, 2025
fdc7821
fix: allow newlines after colon
ba11b0y Jul 16, 2025
232f91e
Merge branch 'canary' into maps/grammar-update
ba11b0y Jul 16, 2025
384e397
Merge branch 'antonio/vm' into rahul/map-constructors
ba11b0y Jul 16, 2025
ab74cc9
Merge branch 'maps/grammar-update' into rahul/map-constructors
ba11b0y Jul 16, 2025
85c39f9
map constructor implementation
ba11b0y Jul 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions engine/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions engine/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
[workspace]
resolver = "2"
members = [
"baml-compiler",
"baml-lib/*",
"baml-lsp-types",
"baml-rpc",
"baml-ids",
"baml-runtime",
"baml-schema-wasm",
"baml-vm",
"bstd",
"cli",
"language_client_codegen",
Expand All @@ -21,6 +23,7 @@ members = [
"zed", "llm-response-parser",
]
default-members = [
"baml-compiler",
"baml-lsp-types",
"baml-lib/*",
"baml-rpc",
Expand All @@ -31,6 +34,7 @@ default-members = [
# and it's OK to not build this by default because we have to build ts separately (this may
# also have something to do with resolver=2 and how duplicate dep builds are handled)
# "baml-schema-wasm",
"baml-vm",
"bstd",
"cli",
"language_client_cffi",
Expand Down
117 changes: 117 additions & 0 deletions engine/baml-compiler/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Overview

The baml-compiler is a Rust crate that compiles BAML (Basically a Made-up Language) source code into bytecode for execution by the BAML VM. It's part of a larger monorepo that includes the BAML runtime, language clients, and tooling.

## Common Development Commands

### Building and Testing the Compiler

```bash
# Build the compiler
cargo build

# Run compiler tests
cargo test

# Format code
cargo fmt

# Run linter
cargo clippy

# Build with watch mode (from repository root)
../../tools/build . --watch

# Run tests with watch mode
../../tools/build . --test
```

### Working with the Full BAML Stack

From the repository root (`/home/greghale/code/baml/`):

```bash
# Build all Rust components
cd engine && cargo build

# Run all Rust tests
cd engine && cargo test --lib

# Build and test specific component
./tools/build engine/baml-compiler --test

# Run integration tests
cd integ-tests && ./run-tests.sh
```

## Architecture

### Compiler Structure

The baml-compiler transforms BAML source code → AST → Bytecode:

1. **Parser** (from parser-database) → AST
2. **Validation** (from baml-core) → Validated IR
3. **Compiler** (this crate) → VM Bytecode

Key files:
- `src/lib.rs` - Main compiler entry point and bytecode generation
- Dependencies on:
- `internal-baml-core` - IR and validation
- `internal-baml-parser-database` - Parsing and AST
- `baml-vm` - Bytecode definitions

### Bytecode Instructions

The compiler generates stack-based bytecode with instructions like:
- `LoadConst`, `LoadVar`, `LoadGlobal` - Load values
- `Call`, `Return` - Function calls
- `Jump`, `JumpIfFalse` - Control flow
- `AllocArray`, `Pop` - Data structures

### Current Limitations

Several features are marked as `todo!()` in the implementation:
- Raw strings
- Maps
- Class constructors
- Lambdas
- Loops (for, while)

## Working with BAML Code

BAML is a domain-specific language for building reliable AI workflows. When modifying the compiler:

1. Changes to bytecode generation affect the VM execution
2. New language features require updates to:
- Parser (in parser-database)
- IR/validation (in baml-core)
- Compiler (this crate)
- VM (in baml-vm)

3. Test changes using integration tests in `/integ-tests/`

## Testing Individual Components

```bash
# Run a specific test
cargo test test_name

# Run tests with output
cargo test -- --nocapture

# Run only compiler tests (not workspace)
cargo test --package baml-compiler
```

## Related Components

When working on the compiler, you may need to understand:
- `baml-lib/parser-database/` - How BAML is parsed
- `baml-lib/baml-core/` - The IR and type system
- `baml-vm/` - How bytecode is executed
- `baml-runtime/` - How the runtime orchestrates execution
14 changes: 14 additions & 0 deletions engine/baml-compiler/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
[package]
edition = "2021"
name = "baml-compiler"
version.workspace = true
authors.workspace = true
description = "BAML Bytecode Compiler"
license-file.workspace = true

[dependencies]
internal-baml-core = { path = "../baml-lib/baml-core" }
internal-baml-diagnostics = { path = "../baml-lib/diagnostics" }
internal-baml-parser-database = { path = "../baml-lib/parser-database" }
baml-vm = { path = "../baml-vm" }
anyhow = { workspace = true }
Loading