Hey folks! I'm using tree-sitter to parse some Rust source code. I'm trying to write a function that produces an iterator over all the children module items of a node. I have a function that more or less looks something like

pub fn get_node_children_modules<'tree>(
    tree: &tree_sitter::Node<'tree>,
    source_bytes: &[u8],
) -> impl StreamingIterator<Item = ModuleItem<'tree>> {

    const MOD_QUERY_STRING: &str =
        "(mod_item name: (identifier) @ident body: (declaration_list)) \
         @item_mod_body \n(mod_item name: (identifier) @ident) @item_mod";

    let ast_query =
        Query::new(&tree_sitter_rust::LANGUAGE.into(), MOD_QUERY_STRING)
            .unwrap();

    let mut cursor = QueryCursor::new();

    // ... do things with `cursor` ...

    let mod_items = cursor.matches(&ast_query, *tree, source_bytes);

    let capture_unpack = |qm: &QueryMatch| {
        // return objects bound to the lifetime of
        // `cursor` and `ast_query`
    };

    mod_items.map(capture_unpack)
}

Unsurprisingly, this does not work as ast_query and cursor do not live long enough. Instead, I've tried to return a type that also contains cursor and ast_query:

struct ModuleItemIterator<'tree> {
    ast_query: tree_sitter::Query,
    cursor: tree_sitter::QueryCursor,
    iterator: Box<dyn StreamingIterator<Item = ModuleItem<'tree>> + 'tree>,
}

impl<'tree> StreamingIterator for ModuleItemIterator<'tree> { /* */ }

The issue is more or less the same:

error[E0515]: cannot return value referencing local variable `cursor`
   --> src\project_structure.rs:229:5
    |
179 |       let mod_items = cursor.matches(&ast_query, *tree, source_bytes);
    |                       ------ `cursor` is borrowed here
...
229 | /     ModuleItemIterator {
230 | |         ast_query,
231 | |         cursor,
232 | |         iterator: Box::new(mod_items.map(capture_unpack)),
233 | |     }
    | |_____^ returns a value referencing data owned by the current function

error[E0515]: cannot return value referencing local variable `ast_query`
   --> src\project_structure.rs:229:5
    |
179 |       let mod_items = cursor.matches(&ast_query, *tree, source_bytes);
    |                                      ---------- `ast_query` is borrowed here
...
229 | /     ModuleItemIterator {
230 | |         ast_query,
231 | |         cursor,
232 | |         iterator: Box::new(mod_items.map(capture_unpack)),
233 | |     }
    | |_____^ returns a value referencing data owned by the current function

error[E0505]: cannot move out of `ast_query` because it is borrowed
   --> src\project_structure.rs:230:9
    |
143 |   pub fn get_node_children_modules<'tree>(
    |                                    ----- lifetime `'tree` defined here
...
163 |       let ast_query =
    |           --------- binding `ast_query` declared here
...
179 |       let mod_items = cursor.matches(&ast_query, *tree, source_bytes);
    |                                      ---------- borrow of `ast_query` occurs here
...
229 | /     ModuleItemIterator {
230 | |         ast_query,
    | |         ^^^^^^^^^ move out of `ast_query` occurs here
231 | |         cursor,
232 | |         iterator: Box::new(mod_items.map(capture_unpack)),
233 | |     }
    | |_____- returning this value requires that `ast_query` is borrowed for `'tree`

error[E0505]: cannot move out of `cursor` because it is borrowed
    --> src\project_structure.rs:231:9
     |
 143 |   pub fn get_node_children_modules<'tree>(
     |                                    ----- lifetime `'tree` defined here
...
 174 |       let mut cursor = QueryCursor::new();
     |           ---------- binding `cursor` declared here
...
 179 |       let mod_items = cursor.matches(&ast_query, *tree, source_bytes);
     |                       ------ borrow of `cursor` occurs here
...
 229 | /     ModuleItemIterator {
 230 | |         ast_query,
 231 | |         cursor,
     | |         ^^^^^^ move out of `cursor` occurs here
 232 | |         iterator: Box::new(mod_items.map(capture_unpack)),
 233 | |     }
     | |_____- returning this value requires that `cursor` is borrowed for `'tree`

It wouldn't be an issue to instead eagerly evaluate the iterator and just return a vector. I'd just like to ask whether something like this is possible for educational purposes. Is this even a good idea? Would it be possible to implement it by pinning the cursor and ast_query objects, and defining the iterator field to be MaybeUninit<Box<_>>?

4 posts - 2 participants

Read full topic