Sample Rates and Codecs

By the end of this session you will be able to:

• Name the frequency ceiling of any sample rate, and say which speech sounds fall above the ceiling of a phone line
• Encode and decode G.711 mu-law in both ffmpeg and Python, and explain why one byte per sample sounds better than 8-bit linear PCM
• Push a clean 16 kHz recording through a telephone-grade path and measure exactly what the round trip destroyed

Nyquist decides which sounds survive

A sample rate is a promise about the highest frequency you can represent. The ceiling is half the rate, and nothing above it survives digitization. So 16 kHz audio carries content up to 8 kHz. 8 kHz audio carries content up to 4 kHz. There is no clever decoder that gets the rest back, because it was never written down.

That matters because human speech does not stop at 4 kHz. Vowels are fine: the first two formants that carry vowel identity sit roughly between 300 Hz and 2500 Hz, comfortably inside the phone band. Fricatives are not fine. The /s/ in "Sam" puts most of its energy above 4 kHz. So does the /f/ in "Frank", the /th/ in "three", and the /sh/ in "shipping". Cut the audio off at 4 kHz and those consonants collapse toward each other, because the part that told them apart lived in the part you deleted.

Real telephony is narrower than the 4 kHz Nyquist ceiling suggests. The classic phone channel is band-limited to roughly 300 Hz to 3400 Hz, so you lose the bottom too. That gap is why a caller reading an account number over a phone line is the single most reliable way to break a speech pipeline that scored well on your laptop mic. Whole words survive because context fills them in. Isolated letters and digits do not have context.

The trap that follows: almost every current speech model wants 16 kHz mono input. NVIDIA Parakeet-TDT-0.6B-v3 wants 16 kHz. Silero VAD v6 accepts 16 kHz or 8 kHz and nothing else. So you will resample telephone audio up to 16 kHz to feed the model, the file will report 16000 in its header, and the top half of the spectrum will be empty. Upsampling is interpolation, not restoration. Never read a header and conclude you have wideband audio.

mu-law is a number line, not a zip file

G.711 mu-law is what Twilio Media Streams and most of the phone network actually put on the wire, and it is not a codec in the sense that Opus or MP3 are codecs. There is no analysis, no frames, no decoder state, no variable bitrate. It is a lookup: take a 16-bit linear sample, map it through a logarithmic curve, store the result in one byte.

The logarithm is the whole idea. Straight 8-bit linear PCM spaces its 256 levels evenly, so a quiet passage gets the same absolute step size as a loud one and quiet speech drowns in quantization noise. Mu-law spaces the levels logarithmically: fine steps near zero, coarse steps near full scale. The error stays roughly proportional to the signal instead of fixed, which buys you something close to 12 bits of usable dynamic range in 8 bits of storage.

You can see it directly:

import audioop
import struct
 
pcm = struct.pack("<4h", 0, 1000, -1000, 32767)
encoded = audioop.lin2ulaw(pcm, 2)
decoded = audioop.ulaw2lin(encoded, 2)
 
print(len(pcm), "bytes in,", len(encoded), "bytes out")
print(struct.unpack("<4h", decoded))

That prints 8 bytes in, 4 bytes out and (0, 988, -988, 32124). Look at the two errors. The sample at 1000 came back 12 low, the sample at 32767 came back 643 low. Both are off by about two percent. That constant relative error is the design goal, and it is why mu-law at 8 bits is usable for voice and 8-bit linear is not.

On Python 3.13 and later, audioop was removed from the standard library. Install the maintained port, which still imports under the original name:

pip install audioop-lts

The other consequence of "no frames, no state" is that the byte layout is trivially predictable. One byte per sample at 8000 samples per second is 8000 bytes per second, which is exactly 64 kbit/s, which means 20 ms of audio is always 160 bytes. Any byte offset maps to a time offset with a division. You will lean on that constant in the next session.

Build the phone line on your desk

Start from a clean mono 16 kHz recording called clean16k.wav. Encode it down and back with ffmpeg:

ffmpeg -i clean16k.wav -ar 8000 -ac 1 -c:a pcm_mulaw -f mulaw phone.ulaw
ffmpeg -f mulaw -ar 8000 -ac 1 -i phone.ulaw -ar 16000 -c:a pcm_s16le phone16k.wav

Note that -f mulaw -ar 8000 -ac 1 appears on the input side of the second command. Raw mu-law has no header, so nothing in phone.ulaw tells a decoder its sample rate or channel count. Prove it to yourself:

ffprobe -v error -f mulaw -show_entries stream=codec_name,sample_rate -of default=noprint_wrappers=1 phone.ulaw

Without -ar 8000 on that command, ffprobe reports sample_rate=44100. It is guessing, because the file contains no answer. Every "the audio plays back as chipmunks" bug in telephony traces to somebody guessing here.

The same round trip in Python, which is the form you will reuse when the bytes arrive over a socket rather than in a file:

import audioop
import wave
 
with wave.open("clean16k.wav", "rb") as w:
    assert w.getnchannels() == 1 and w.getsampwidth() == 2
    rate = w.getframerate()
    pcm = w.readframes(w.getnframes())
 
down, _ = audioop.ratecv(pcm, 2, 1, rate, 8000, None)
encoded = audioop.lin2ulaw(down, 2)
decoded = audioop.ulaw2lin(encoded, 2)
up, _ = audioop.ratecv(decoded, 2, 1, 8000, rate, None)
 
print("source     ", len(pcm), "bytes at", rate, "Hz")
print("on the wire", len(encoded), "bytes at 8000 Hz mu-law")
print("ratio      ", round(len(pcm) / len(encoded), 1), "to 1")
 
with wave.open("phone16k.wav", "wb") as w:
    w.setnchannels(1)
    w.setsampwidth(2)
    w.setframerate(rate)
    w.writeframes(up)

The ratio prints 4.0 to 1. Half of that came from halving the sample rate and half from halving the bytes per sample. Neither half is free.

Try it

Record about fifteen seconds of yourself, mono, 16 kHz, doing two things a real caller does: spell a surname letter by letter using words ("S as in Sam, F as in Frank"), then read a ten-digit account number.

Run the Python script above to produce phone16k.wav. You now have two files with identical duration and identical headers, one of which has been through the phone.

Transcribe both with the same model. ElevenLabs Scribe v2 or Parakeet-TDT-0.6B-v3 both work here. Then diff the two transcripts by hand.

Success condition: you can point at a specific character that changed and say which frequency band killed it. An /s/ heard as /f/, a "five" heard as "nine", a spelled letter dropped entirely. If the two transcripts are identical, your recording was too clean or too slow. Re-record at conversational speed, with a bit of room noise, and try again.

Common mistakes

• Resampling to 16 kHz and calling the problem solved. The header now says 16000 and the model stops complaining, so it feels fixed. The band above 4 kHz is still empty. Your evaluation audio has to be recorded through the same path as production, not upsampled after the fact.
• Storing raw mu-law without recording its rate somewhere. The bytes carry no metadata. Write the rate into the filename, the database row, or a WAV container, and never let a default guess it.
• Reaching for a "better" codec on the telephony leg. You do not get to pick. If the call arrives as 8 kHz mu-law, transcoding to Opus afterwards adds latency and loss and restores nothing.
• Benchmarking your model on 16 kHz laptop audio only. A number that came from a headset mic tells you nothing about a phone line. Run both, keep both, and report both.
• Assuming mono. Stereo interleaves two samples per frame, so a stereo stream fed to a mono decoder plays at half speed and looks exactly like a rate bug.

Where this goes next

You now have audio in the right format, sitting still in a file. The next session, Frames, Buffers, Backpressure, puts it on a wire in 20 ms pieces and shows you the queue that grows quietly behind a slow consumer until the call drops.