How do you organize a cache that is shared for read and write between multiple threads? Any crate that you'd recommend?

What I'm trying to make: some geospatial data is sharded by territories and served as tiles. A web server (based on Axum) will get the requests. Usually a (human) user focuses the map on one area, the browser reuqests the tiles visible in the window simultaneously. Most of the tiles will be computed from the same shard, but have different content, and the client won't request them again in days, because of their own cache. If the user zooms in or out, the client requests the tiles at the different level, that must be built again anew. That's why I think the best way is to cache not the output, but the raw data.

The simplest way would be caching some chunks of raw bytes (maybe file pointers or DB connections, but that's harder). If I implement this cache with RwLocks, I'll get either global locks or race conditions.

Here's a scenario: user drags the map into a new area. There's 10..30 HTTP requests that should be served from the same shard.

• If I try to avoid a write lock (by making a read lock, checking and then either using the data or fetching it), in my case multiple threads will check for the same data, find out it's absent, and all of them will go fetching it, then each of them will get a write lock to store the same thing, waiting in queue and then overwriting. Double-checking before writing makes things expensive too.

• If the threads get a write lock from the very beginning, instead of race condition, I get threads waiting in a long queue for others, and those in the end served a lot longer.

I may try a different shard approach, making finer shards, but that has serious costs in storage, and extra complexity in the code. I contemplated a single server thread, that would get requests from the other threads over channels, but I'm sure I just know too little, and there's much more complexity there.

I guess, I'm not competent enough to implement this myself. Please, recommend me a solution.

4 posts - 4 participants

Read full topic