Info
It originally started as an excuse to learn more about compilers and language design in Rust, but over time I became interested in whether a dedicated language could make certain data-processing workflows easier to express and reason about.
The result is a small DSL that compiles `.xzz` scripts into Polars LazyFrame execution plans through a Rust compiler pipeline.
Working on it ended up touching a lot of areas I wanted to learn more about:
* parsing and AST design
* code generation
* Rust workspace organization
* type systems
* data processing backends
Along the way I implemented things like:
* a typed DSL with `Option` for null-safe columns
* CSV schema inference
* code generation targeting Polars LazyFrames
* a multi-crate workspace with dependency isolation
* a small visual editor that I built as a separate experiment
One thing I spent a surprising amount of time on was the workspace architecture.
The compiler and CLI don't link Polars directly. Instead, execution happens through a separate runner executable that contains the heavy dependencies. I wanted to see how small I could keep the CLI binary while still keeping the overall structure reasonably clean.
I also probably spent more time reorganizing crates than actually adding language features.
A small example:
.xzz
type AirQuality = {
station: string,
pm10: Option<float>,
}
v data = load("data.csv") :: AirQuality
|> filter(pm10 > 50)
|> groupBy("station")
|> mean("pm10")
Repository:
This is by far the largest Rust project I've worked on so far, so I'd be interested in hearing what more experienced Rust developers would do differently.
In particular, I'd appreciate feedback on:
* compiler architecture
* workspace structure
* code generation
* general Rust design choices
Any suggestions, criticism, or things that look obviously wrong are welcome.
And if anyone ends up trying it, I'd be especially interested in hearing what parts of the language feel unintuitive or awkward. That's something that's hard to evaluate when you're also the person designing it.
Thanks for taking a look.
1 post - 1 participant
🏷️ Rust_feed