In Need of Architecture Strategy for Real Time Audio Transfer

⚓ rust    📅 2025-05-18    👤 surdeus    👁️ 5      

surdeus

Warning

This post was published 42 days ago. The information described in this article may have changed.

Hi,

tldr: I need to sum up lot's of float value on the go with relay network and transmit it.

Story: I'm building an audio chat application. I constantly get sound data from clients. Sound data type is f32 simply. Here is a simple analogy (numbers are not real): A microphone creates 8 float values per second and a speaker consumes 8 float values per second. So if a person listens just one person there is no problem but let's say 2 people speaking and a person is listening this means that I have 16 float values per second but listener is only able to consume 8 per second. It means delay. So I need to mix them. Basically I'm going to sum up float values.

I come up with this kind of thing.

#[derive(Debug)]
struct User {
    audio_buffer: Vec<f32>,
}
impl User {
    async fn new() {
        let audio_buffer = vec![];
        let user = Self { audio_buffer };
        ONLINE_USERS.write().await.push(user);
    }
}

static ONLINE_USERS: LazyLock<RwLock<Vec<User>>> = LazyLock::new(|| RwLock::new(vec![]));
static GLOBAL_TIMER: LazyLock<broadcast::Sender<Timer>> = LazyLock::new(|| broadcast::channel(1).0);

#[derive(Debug, Clone, Copy)]
enum Timer {
    LocalBuffer,
    GlobalBuffer,
}

async fn start_global_timer() {
    loop {
        GLOBAL_TIMER.send(Timer::LocalBuffer);
        sleep(GLOBAL_TIMER_LOCAL_BUFFER_TIME).await;

        GLOBAL_TIMER.send(Timer::GlobalBuffer);
        sleep(GLOBAL_TIMER_GLOBAL_BUFFER_TIME).await;
    }
}

async fn receive_sound_data_and_save_to_online_user_buffer() {
//receive remote data for sleep time
//save gathered data to online_users buffer
}
async fn sum_up_sound_data_from_all_online_users_except_himself_and_transmit_to_him() {
//If I sum his sound data. It means he will hear himself too.

//iterate all users data except user himself and sum up.
//send data to remote client
//sleep so buffers can reload.
}

But this architecture feels wrong. I had couple ideas too but just something doesn't fit in my mind. I need help about strategy. I hope I explained in a understandable way :confused: .

1 post - 1 participant

Read full topic

🏷️ rust_feed