Let's say you are recording and processing audio in realtime. You have one function available which runs on a separate thread and provides you with the samples (fixed number every call). As I understand this article (and similar ones), you are not supposed to malloc on that thread (as that thread hanging could result in dropped samples). But if you want to save the audio, how else are you going to save it without appending to some array (for which you need to malloc)?
One common solution is to have another thread continuously prepare a bunch of empty buffers in advance, the audio thread just uses one of them, and after that is filled another thread writes it to disk. No locking is required, and no malloccing/disk IO happens on the audio thread.
Couldn't you just spin through buffers alloc'd at startup? Why do you need another thread for buffer management? You could do something like a ring buffer to change pointers.
That said, I've just started working on my first audio pipeline app, and I'm having sampling rate issues making voices sound super deep.
> But if you want to save the audio, how else are you going to save it without appending to some array (for which you need to malloc)?
I don't know why everybody is making this so hard: You use a lock-free statically allocated circular buffer for all communication to or from a real-time audio thread. Period. Full stop. The audio thread wins under all contention scenarios. The audio thread ONLY ships data from the circular buffer to the hardware system or vice versa--IT DOES NO OTHER TASK. Everything else talks to the circular buffers.
Nothing else works.
The big questions are even if you do that--1) how do you keep the circular buffer from starving even with your other threads working to fill the circular buffer? and 2) if the circular buffer starves, what do you do?
The audio thread ONLY ships data from the circular buffer to the hardware system or vice versa--IT DOES NO OTHER TASK.
All you have done in this case is push the problem up one level. The interface with the hardware already works this way.
The interesting problem is actually doing something more substantial - eg. deciding which audio to play, or processing it in some way - in a way that lets you keep feeding that circular buffer on time, every time.
> All you have done in this case is push the problem up one level.
Actually, using a circular buffer has transferred a "hard, real time" problem into a "soft real time" problem. As long as what you want to do has an average time shorter than the time between audio packets and you have enough audio packets buffered, you can ride across the occasional schedule miss because the system was off doing something else (like GC).
For example, VoIP quite often uses a "jitter buffer" for exactly this task.
Now, you still need to keep the circular buffer fed and that may not be easy. However, it's a lot easier than being 1:1 with a hard audio thread.
> The interface with the hardware already works this way.
Most of the time when I'm interacting with low-latency audio threads, they generally don't allow me to specify the buffer semantics with very much flexibility.
This is not right; this can get you priority inversion if your audio thread is waiting for a lower priority thread, and for many tasks it just adds overhead.
As an example, consider some software I wrote recently: https://www.jefftk.com/bass-whistle It reads in audio of someone whistling, interprets it, and resynthesizes it as bass. It needs to be very low latency, or else it won't feel fluent to play. Roughly, on each audio callback it needs to:
* For every sample in the input buffer determine (a) is there whistling and if so (b) what pitch it is.
* Use those decisions to synthesize audio and populate an output buffer.
It does all of this on the audio thread, and there's absolutely no reason to move this processing elsewhere.
> This is not right; this can get you priority inversion if your audio thread is waiting for a lower priority thread
No. It can't invert because the circular buffer is lock free--that's the whole point. The audio thread controls the resources and if the audio thread doesn't relinquish then the other threads can't add to the data structure--the audio thread, however, never, ever sees a point of contention.
Now, the audio thread can theoretically starve because it's holding resources so long that the other threads can't transfer a buffer in. However, as audio threads tend to be hard, real-time periodic with the highest priority, so that problem is of the programmers own making.
> for many tasks it just adds overhead.
Only if your task is audio in->audio out with very little processing.
The moment you have to pull from network or disk, change in response to a UI event, or anything else which touches something non-audio, you either eat the overhead of a circular buffer data structure or you risk a glitch.
You only need one result (the latest) at any given time, so you allocate room before your loop, set an atomic indicating that a calculation is in progress, then have exclusive rw access to that memory.
It’s a really old trick, lots of unixy APIs returning strings or objects that you are not expected to free do a similar thing (function local static variables to storing results) at the cost of not being thread-safe.
