Warning
Info
I have an axum async handler as follows (simplified):
async fn put_handler(Path(method_id): Path<u16>, Query(params): Query<Params>, header_map: HeaderMap, body: Bytes) -> impl IntoResponse {
if CACHES.put(method_id, params.key.unwrap(), params.value,unwrap()).is_ok() {
StatusCode::OK
}
else {
StatusCode::INTERNAL_SERVER_ERROR
}
}
'CACHES.put' is ultrafast, in the nanoseconds range. That's why is sync. It only performs a 'put' in a cache and sets the params.value content to a in-memory buffer.
I also have the CACHE.get(key), which is async, because it involves to send a message to a worker (spawn_blocking) and await the response (in a oneshot channel) that comes from disk. This is in the microseconds range (usually < 500us) and, occasionally, few milliseconds.
async fn get_handler(Path(method_id): Path<u16>, Query(params): Query<Params>) -> Response<Body> {
match CACHES.get(method_id, params.k.unwrap()).await {
Ok((delay, maybe_document)) => {
if let Some(document) = maybe_document {
return (StatusCode::OK, document).into_response();
}
// 1. Not found and no 'delay'.
if delay == 0 {
return StatusCode::NOT_FOUND.into_response()
}
// 2. Not found yet, but it will be there soon --> return 'delay'.
let header = [(constants::DELAY_SLEEP_HEADER_NAME, delay)];
(StatusCode::NOT_FOUND, header).into_response()
},
Err(CacheError::CacheNotFound(_)) => {
StatusCode::NOT_FOUND.into_response()
},
Err(err) => {
StatusCode::INTERNAL_SERVER_ERROR.into_response()
}
}
}
When I perform my tests, if my requests are mainly 'get', everything works fine. But every time I increment the number of 'put', the latency in notifying the oneshot (from the spawn_blocking to the async get that is awaiting the response) increases more and more.
The more 'puts' I have, the greater the latency is.
I suspect this increasing number of sync 'puts' are starving the tokio scheduler, so every time is harder for Tokio to wake up tasks that are awaiting a notify.
So, is there any way to make these 'put' not to interfere in the tokio scheduler?
I read something about block_in_place, but not sure if this is the best solution.
Thanks.
1 post - 1 participant
🏷️ Rust_feed