Hi everyone,

I've been working on a terminal-based log viewer called LogLens, and today I'm open sourcing the core parsing and query engine as a standalone library: loglens-core.

What is it? It's a library designed to handle the "messy middle" of log analysis—detecting formats, parsing structured data, and filtering it efficiently—without requiring a predefined schema.

Key Features:

• Zero-Config Parsing: It heuristically detects whether a line is JSON or Logfmt and parses it accordingly.
• Custom Query Syntax: I implemented a simple query language (level == "error" && latency > 500) that operates directly on the structured data.
• Time-Aware:It tries to automatically identify and parse timestamp fields for range filtering.

Why I built it: I found that existing tools often required too much configuration (defining schemas upfront) or were too slow for ad-hoc investigation of large log files. I wanted something that could "just work" on a messy production.log file.

example usage:

use loglens_core::{evaluate, parsers::parse_log_line, LogEntry};

let line = r#"{"level": "error", "latency": 502, "msg": "Timeout"}"#;

// 1. Parse: The engine detects this is JSON
let entry = parse_log_line(line);

// 2. Evaluate: Run the query only if it's structured data
if let LogEntry::Structured(value) = entry {
let match_found = evaluate(&value, line, "latency > 500").unwrap();
println!("Match: {}", match_found); // Prints: Match: true
}

Links:

• Crate: crates.io: Rust Package Registry
• Docs: loglens_core - Rust
• Repo: GitHub - Caelrith/loglens-core: The core engine for the LogLens tool.

I'd love any feedback on the API design or the query parser logic. Thanks!

1 post - 1 participant

Read full topic