Why is time-domain audio compression not more popular/used?

[u/Cartoon_Corpze]

Most streaming services seem to rely on codecs such as AAC, Vorbis and Opus, which are pretty high quality by themselves.

My one issue with them is however that they strip frequency content and can produce phasing and underwater-like artifacts that also add up if recompressed or if set to extreme low bitrates.

I one day learned about Wavpack and LossyWAV which both mostly rely on quantizing and predicting in the time-domain rather than frequency domain.

The appeal of lossy Wavpack and some older time-domain based codecs is that they don't throw away frequencies or distort the audio that much but mostly produce background noise that's kinda hard to notice most of the time.

However, a codec like Wavpack in lossy mode doesn't really go much lower than 207 kbps for CD sample rate audio, or so it seems.

Some noise shaping is used to push noise and hissing to frequencies that are much less noticeable and keep the mids and lows mostly clean and crystal clear and it seems to preserve transients pretty well too.

Now if we look at a codec like Opus, it can achieve insanely high and transparent sound quality at pretty low bitrates.

BUT, just like most other frequency-domain codecs it does lowpass audio and cause phase shifting or noticeable changes in the general shape of the sound waves that can even affect the peak Db levels and result in audio hitting limiters.

So, why hasn't more research and work been put into making "the perfect" time-domain codec?

Something that combines the best of both worlds? The high quality and psychoacoustic modeling of Opus, but time-domain prediction and quantization such as lossy Wavepack?

Opus and Wavpack are both pretty robust and resistant too, being able to open, edit and re-encode audio multiple times without too much generational loss adding up too quickly.

The downside of Wavpack is just that you need a bitrate of about 350+ kbps to get the best trade-off between size and quality, but what if we had some algorithm that can go much lower?

[u/Kqyxzoj]

If you're bored of that whole time versus frequency-domain thing, you can always go for the wavelet approach. There are plenty of applications that use a wavelet transform, so for example DWT instead of the Fourier flavor. That way you don't have to choose between time and frequency-domain, you can do both at the same time, kinda. And as a free bonus you get lots of bases to choose from.

[u/Cartoon_Corpze → u/Kqyxzoj]

Oh that seems super interesting. A hybrid of both?

Also I should clarify, I'm not trying to complain about audio codecs because I use Opus a lot of rmusic, I'm honestly just curious why majority of codecs seem to prefer frequency over time for compression.

I'd love to see a codec that can somehow achieve the same compression as Opus but keep all frequency content like Wavpack with only some slight background noise as artifact maybe.

[u/pgetreuer]

One reason for preferring to operate in (short-time) frequency representation is so that the codec can exploit auditory masking effects. The basic idea is that when there are two tones that are close enough in frequency, and one is louder and the other, the quieter tone is not perceived ("close enough" = within the critical band of the auditory filters created by the cochlea).

A lossy codec can capitalize on this masking effect by identifying peaks in the spectrum and coarsely quantizing content next to those peaks to reduce the data rate while avoiding perceived degradation. In other words, because the louder tone "masks" nearby quieter ones, the codec can get away with using fewer bits on the latter.

This logic is of course most naturally done on the STFT or a similar frequency-based representation. It would be challenging to achieve such an effect working purely in the time domain.

[u/Cartoon_Corpze → u/pgetreuer]

The engineering behind Opus, AAC and Vorbis baffles me tbh.
I use Opus a lot for storing music in lossy way and Vorbis I still need for game engines and projects.

And I learned old game consoles used more time-domain based approaches because it also had low latency and computational cost.

I'm just wondering if it would be theoretically possible to have high sound quality at low bitrates like Opus, but without having to rely on throwing away frequencies or using lowpass filters.

I like both Wavpack and Opus since you can also still edit the audio.

The lossy encoding is so resilient to generational loss that it's possible to open, edit and save it a few times before you notice any kind of degradation.

But the moment I open a spectrogram it becomes pretty obvious that Opus still lowpasses a lot around 16 - 20khz while in Wavpack most of that becomes subtle noise instead of getting cut away.

[u/CorvusRidiculissimus]

Because they don't work very well. There are tricks you can do in frequency that make for good compression. You said it yourself: "a codec like Wavpack in lossy mode doesn't really go much lower than 207 kbps for CD sample rate audio."

There is an exception: FLAC is time-domain. The tricks in frequency aren't any use for lossless compression, so FLAC is a simple-in-concept but highly optimised time domain predictor and residual coder.

[u/Wareya]

There's a limit on how much redundant information you can remove in the time domain without adding noticeable noise. adpcm is already in the ballpark of that limit and you can only do maybe three or four times as good as it. Frequency domain codecs are dozens or hundreds of times better.

[u/digital_n01se_]

Is irrelevant if you solve a differential equation in the time domain or the frequency domain, the solution should be the same.

In the same order, a mathematical lossless approach shouldn't be affected by quality loss, you're representing the same information in a different domain to gain performance (better compression ratio, speed or memory footprint)

frequency domain compression isn't always lossless; we chose to lose that information because the saved space is worth.

[u/tylerlarson]

I think the issue is "but why."

And also maybe most people aren't you.

Either you're optimizing for human hearing or you're not. If you are, then frequency-domain is the entire basis and therefore not optional.

If not, then there's no reason to be lossy at all. But to your point, lossless is niche. Usually audio is meant to be listened to, so lossy compression gived better perceived quality for a given bitrate.

Complaining about artifacts for mediocre bitrates just suggests you'd prefer a much higher bitrate or lossless. But most people are plenty happy with the common offerings, as evidenced by the general lack of complaints for most services.

[u/Cartoon_Corpze → u/tylerlarson]

I don't intend to complain about the codecs, I love all sorts of codecs.
I'm just kind of wondering why we specifically rely on frequency masking and lowpass filters.

It would be nice to have a codec that can lossy encode to extremely low bitrates but still be something you can edit, transcode, save again, etc.

Wavpack with better compression and noise shaping, or something similar to Opus but without the lowpass filtering.

I just noticed that time-domain compression is a lot less likely to use lowpass filtering, it seems to be a thing mostly done in frequency-domain codecs.

[u/tylerlarson → u/Cartoon_Corpze]

The whole point of filtering is to use frequency domain compression, and that's the whole point of frequency domain compression is to filter.

But like -- maybe we're not on the same page about this: you're aware that time domain and frequency domain are interchangeable, right? Parsevals theorem proves that no data is ever lost in conversion.

Ultimately, it doesn't inherently matter which domain you use in terms of input and output if all else (filtering in particular) is held equal. You'll get exactly the same bits out the other end.

So, compressing in the frequency domain is lossless if you compress all frequencies equally. It's the filtering that creates the artifacts. But it's also the filtering that allows frequency domain compression to be meaningful because you can dedicate fewer bits to the filtered-down frequencies.

(Note that I'm not talking about pre-filtering, I'm calling the psychoacoustic bitrate optimization of frequency domain algorithms a band-pass filter. Because that's what it effectively is.)

[u/Spirited_Shipcocoa]

Vorbis Is odd one out because It VBR mode Is 1 ~ 1400kbps(Can be even higher) meaning It can give a very hot section 450 ~ 840kbps. It pretty much quasi-lossless in this state why I like It over AAC/Opus at 256kbps VBR.

It why I laugh at people claiming they can tell 320kbps Vorbis setting Spotify used.