Hi,

I've been working on this for a bit - picojson

The gist is: a streaming parser that does not allocate, recurse or panic - e.g. fully deterministic behavior ( some test/demo results here ). The caller will always get an error if we are out of buffers or the input is broken.

Motivation is mostly usage in really resource constrained env, e.g microcontrollers with kilobytes of memory, that aren't allowed to crash.

I'd be looking for more feedback on whether the toplevel API makes sense, and any other improvements for the internal implementations as well of course. I know i need to work on code de-duplication between the Slice / Stream parsers, and the longer i wait the harder it gets

The absolute minimal use looks like this:

let json = r#"{"name": "value"}"#;
let mut parser = SliceParser::new(json);

while let Some(event) = parser.next() {
    match event.expect("Parse error") {
        Event::Key(key) => println!("Found key: {}", key),
        _ => {}
    }
}

In a more real world use case target you'd want to pass in scratch buffer allow dealing with string escapes:

let mut scratch = [0u8; 16]; // Just enough to hold the longest string
let parser = SliceParser::with_buffer(r#"{"msg": "Hello\nWorld"}"#, &mut scratch);

But parsing from a slice isn't that interesting, more useful ( and complex ) is the StreamParser:

let json = br#"{"large": "document with lots of data..."}"#;
// Simulate reading only 4 bytes at a time
let reader = ChunkReader::new(json, 4);
let mut buffer = [0u8; 128];
let mut parser = StreamParser::new(reader, &mut buffer);

Of course - this is yet another demo reading from a slice. In real uses anything that implements the simple Reader trait can be used as a source ( UART, network socket, file etc )

The parsing loop looks similar as above.

Here's a more full featured example how to selectively extract real data from a more complex document.

There's a lot more code in the crate than i anticipated that this would need ( parsers are never easy huh ? ) so I'm not looking for anyone to dig through it all. Would just appreciate any comments as to how to make this better / ergonomic / usable.

One specific bit that i don't know how to really do better: i have int-8 / 32 / 64 feature selection in Config.toml, to prevent 64-bit math primitives to be included on say, 8-bit AVR targets. serde-json-core can solve this much more conveniently as the destination de-serialization type is known at compile time:

Deserialization of integers doesn’t go through u64; instead the string is directly parsed into the requested integer type. This avoids pulling in KBs of compiler intrinsics when targeting a non 64-bit architecture.

With pull parser I'm not sure there's a good way to make that work. Appreciate any thoughts!

1 post - 1 participant

Read full topic