Plex iOS stall-and-replay loop: the server ships Matroska/Opus HLS segments AVPlayer cannot play, and the quality downshift is the symptom

The trap

A Plex iOS client buffers constantly and replays the same few seconds; Android on the same LAN is fine. The server log shows the iPhone requesting directPlay=0&directStream=0&videoQuality=40&maxVideoBitrate=720 on location=lan, while every Android request is directPlay=1&videoQuality=100. The obvious read — the phone's Video Quality is capped, tell the user to set it to Maximum — is wrong, and I shipped it before checking. The phone was already on Maximum.

videoQuality=40/60 is the app's Automatically adjust quality stepping down after it stalls. Sort the decisions chronologically before interpreting them: inside one session you see vq=100 → 60 → 40. A static cap does not have a ladder.

The actual cause

Grep what the transcoder was told to emit:

grep -oh "segment_format [a-z0-9]*" "Plex Media Server"*.log | sort | uniq -c
grep -oh "encoder=[a-z0-9_]*" "Plex Media Server"*.log | sort | uniq -c | sort -rn

Result on the affected server: 32 segment_format matroska, 2 mpegts, and 15 encoder=libopus — every Opus job aimed at the iPhone. Apple's AVPlayer cannot demux Matroska and cannot decode Opus outside MP4. The player abandons the stream and restarts it at the same segment, forever.

Plex does this because the client asked for it. The TranscodeUniversalRequest: adapting profile with augmentation data: line carries the client's declared capabilities, and there are two very different shapes:

A profile advertising ffv1, dirac and rawvideo is not AVPlayer. If you see that from an iOS device, the enhanced/experimental player is in the loop, and the fix is a client toggle, not server hardware.

Prove the server is innocent before blaming it

Segment production rate is logged. Transcoder segment range: <lo> - <hi> (<requested>) fires on each client segment request; -segment_time N in the Job running: line gives the segment length.

23:26:52 → 23:27:27   segments 437 → 454 = 136 s of video in 35 s = 3.9x realtime
23:27:41 onward       client requests 12-20 s apart for 8 s segments
23:33:06 → 23:34:00   ten resets to highest_ready=0, each re-seeking to 475/476

The server produced faster than realtime and the player still could not continue. Any resets to highest_ready=0 mean full session teardowns — that is the "replays the same segment" the user describes, since each restart re-seeks to the same offset (-ss, -segment_start_number).

Also rule out the usual suspects cheaply:

grep -oh "Used slots for CPU[ ]*is now [0-9]*" "Plex Media Server"*.log | grep -oE "[0-9]+$" | sort -n | uniq -c
grep -oh "exit code for process [0-9]* is [-0-9]* ([^)]*)" "Plex Media Server"*.log | sed -E "s/process [0-9]+/process/" | sort | uniq -c

Many -9 (signal: Killed) are normal — every seek, quality change, or restart kills the transcoder. Confirm it is not the OOM killer separately. And measure achieved throughput from the Completed: ... <ms>ms <bytes> bytes lines on /library/parts/ before blaming Wi-Fi: our stalling client peaked at 127 Mbps while the failing stream was capped at 720 kbps.

Tally decisions correctly

Join each universal/decision request to its Reached Decision ... codes=(...) line by the shared #reqid, and keep the timestamp so you can order them:

import glob, re
req, rows = {}, []
for f in sorted(glob.glob("Plex Media Server*.log")):
    for line in open(f, errors="ignore"):
        u = re.search(r"universal/decision\?([^ ]*)", line)
        rid = re.search(r"#([0-9a-f]{6}) ", line)
        if u and rid and "Request:" in line:
            q = u.group(1)
            g = lambda k: (re.search(k + r"=([^&\s]+)", q) or [None, "?"])[1]
            req[rid.group(1)] = (line[:24], g("protocol"), g("directPlay"), g("videoQuality"), g("session")[:8])
        d = re.search(r"\[Req#([0-9a-f]+)/Transcode\] Streaming Resource: Reached Decision .* codes=\(([^)]*)\)", line)
        if d and d.group(1) in req:
            rows.append(req[d.group(1)] + ("DIRECT PLAY" if "Direct play OK" in d.group(2) else "TRANSCODE",))
for r in sorted(rows): print(r)

protocol=hls is iOS/tvOS, dash is web, * is Android.

Two benchmark gotchas if you do want the CPU ceiling

Use Plex's own binary, copying args from the Job running: line:

export LD_LIBRARY_PATH=/usr/lib/plexmediaserver/lib
export FFMPEG_EXTERNAL_LIBS="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Codecs/<hash>-linux-x86_64/"
"/usr/lib/plexmediaserver/Plex Transcoder" -nostdin -codec:0 libdav1d -i in.mkv ... -f null -

• Without an explicit -codec:0 <decoder> it reports Decoder (codec hevc) not found; Plex's decoders are external .so shims in the Codecs dir plus built-ins like libdav1d, always named explicitly.
• Without -nostdin (or < /dev/null) ffmpeg eats the rest of your shell script from stdin and every benchmark after the first silently never runs.

For reference, a 2012 4-core Xeon E3-1220 v2 with no iGPU: HEVC 1080p→1080p 2.0x, →480p 5.1x; AV1 10-bit 1080p→1080p 1.17x, →480p 2.7x. Slow, but in this incident it was not the problem.

Generalisable lesson

When a client and a server negotiate capabilities, a malformed negotiation looks identical to a resource shortage: the client stalls, backs off, and requests less. Read what was negotiated and what was actually emitted before you size hardware. The quality ladder is downstream telemetry, not a setting.