Exlex: A "Lawless" DOD Config Parser (Zero-copy, Arena Mutation, no_std)

I recently started learning Rust by building a project called Exlex which is a Human readable configuration parser BUT I used DOD-based zero copy parser with a minimalistic syntax that actually works well with my parser. (JSON and TOML are heavier for my goal and possibly very complex).

NOTE:

• Exlex is not complete (8-9 days of development)
• The Docs are incomplete
• interface has a lot of work to do
• While not a game engine itself, it uses SoA (Structure of Arrays) patterns common in high-performance engines to maximize cache efficiency

Exlex offers a unique combination of:

• Zero copy immutable parser
• Native no_std support
• SIMD byte search via memchr on specific functions
• Supports modifying data and dumping it back into string (Arena mutator)
• Human readable format
• Low memory usage even on mutations (15,000 allocation (toml_edit) vs 13)

Stability of parser and mutator

• Proptested (I wrote the Exlex by myself but used AI to generate the heavy testing/benchmark boilerplate).
• for more details look at TESTING.md file

~/Projects/exlex_bench main*
❯ PROPTEST_CASES=10000 cargo test proptest_mutator_engine --release

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/proptest_fuzz.rs (target/release/deps/proptest_fuzz-6b0455884b22cdc2)

running 1 test
test proptest_mutator_engine ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 2 filtered out; finished in 5.87s

What it is and What it does not aim to be:

• Built for hardware constraint environment.
• Built to be Cache-friendly and Memory friendly as much as it can.
• Built for overall speed in lifecycle of a program (Parse -> Read -> Mutate -> Save).
• Syntax specifically designed to make parser fast while maintaining human readability
• It is NOT a feature rich or highly flexible syntax (Use json or toml if you need dynamic typing or complex data structures).

Hardware

I measured the hardware execution on an Intel i3-6006U (Skylake, 2C/4T, 2.0GHz):

• Instructions Per Cycle (IPC): 1.7. This confirms the CPU's pipeline is nearly always fed and rarely waiting for memory stalls.
• L1 Cache Locality: By using flat parallel vectors instead of a standard node tree, I achieved a very high cache hit rate, mathematically evidenced by the 0.07% TLB miss rate.

I have benchmarked over 10 scenarios/6 data topologies.
For the interactive criterion benchmarking, see Benchmarks.html in the repo

Trade-offs

• Rigid syntax
• O(N) Linear scan - In usual configuration parsing, Linear scans outperforms Hashmap because of lack pointer chasing and cache misses (in Intel Core i3 6006U, approx upto 65-75 properties). For continuous arrays of numbers, a linear search on a modern processor is incredibly fast.

For more info read the README.md, Any suggestions/bug reporting are warmly welcomed!

Github repo: GitHub - cychronex-labs/Exlex: Zero-copy, Performance-first,no_std, DOD-based, Arena based Mutation Parser · GitHub
All benchmarks/tests are publicly available at GitHub - cychronex-labs/Exlex-Benchmark: Benchmark code heavily written by AI for benchmarking Project Exlex · GitHub

Thank you

3 posts - 2 participants

Read full topic