I've been researching and working on a CLI tool called sem that diffs code at the entity level (functions, classes, structs) instead of raw lines. Written in Rust, open source.

The motivation came from feeding diffs to LLMs. Line-level diffs are optimized for human eyes scanning a terminal, but when you feed a git diff to a model, most of those tokens are context lines, hunk headers, and unchanged code. I ran some attention score experiments and found the model concentrates significantly better on semantic diffs than raw git diffs.

sem uses tree-sitter to extract entities and diffs at that level. Instead of lines with +/- noise, you get exact entity changes: which struct changed, which function was added, which ones were modified. It also builds a cross-file dependency graph with petgraph and does transitive impact analysis via BFS.

Commands:

• sem diff - entity-level diff with word-level inline highlights
• sem entities - list all entities in a file with their line ranges
• sem impact - show what breaks if an entity changes
• sem blame - git blame at the entity level
• sem log - track how an entity evolved over time
• sem context - token-budgeted context for LLMs

Parsers for Rust, Python, TypeScript, Go, Java, C, C++, C#, Ruby, Bash, Swift, Kotlin, plus JSON, YAML, TOML, Markdown, CSV.

There's also an extension of this research study towards a git merge driver called Weave that plugs into git via .gitattributes and resolves conflicts at the entity level. Two developers editing different methods in the same class get a clean merge instead of a conflict.

I wrote up the algorithmic decisions (structural hashing with xxHash64, three-phase entity matching, BFS vs DFS for impact analysis, Rayon for parallel parsing) here: Building Semantic Version Control in Rust

Install: brew install sem-cli

GitHub: https://github.com/Ataraxy-Labs/sem

Happy to answer any questions about the tree-sitter integration or the graph approach. Always eager for constructive criticism.

1 post - 1 participant

Read full topic