Findings on three observations from the demo grid · measured on real training shards, 2026-09-14 · written in plain language, every number measured (sample sizes stated)
The prompting-scheme page (prompt_schema_2026-08-25.html, §3) defines four tag kinds. The ladder packing code (ladder/code/pack_worker.py via procedural-voice-captions/burst_captions.py) renders something else. Measured on 2,000 real German + 1,500 real/synth English training prompts read straight from the packed .tar shards:
| Tag | Scheme page says | Training data actually has | Measured |
|---|---|---|---|
| Sentence duration | [3.9 seconds duration] | nothing — sentences start directly with (direction cue) | 0 / 3500 |
| Pause | [0.8 seconds pause] | [pause 2.6s] | impl format only (67/2000 DE tier0 · 1364/1500 EN synth); spec format 0 / 3500 |
| Burst, timed | (contented sigh, 0.2 seconds) | (Chuckle) — bare label, no number, often mid-line | timed form 0 / 3500; bare labels 548/1500 (EN synth) |
| Direction cue | (clearly amused, warm and open) — no number | (very smooth, very smooth, very formal) — no number | matches (this one is correct) |
Root cause. There is no timed/untimed switch anywhere in the shipped code — the implementation only ever renders the untimed style (burst_captions.py lines ~881–969: cues as ({cue}) {text}, bursts as bare ({label}), pauses as [pause X.Xs]; prompt_meta.template = 'tags'). The scheme page even describes the untimed 30% arm ("nothing carries a number") — but the 70% timed arm was never built into the pack path. So the data is self-consistent, it just follows the code, not the page.
What this means for the models. Neither the old score run nor the new multitask run has ever seen a numeric duration tag. Duration control comes from elsewhere: the Tokens field (frame budget = seconds × 12.5) plus pause spellings in plain text. The score-run's score_prompt() even depends on the untimed shape (it requires one leading (...) per line and replaces it with conditioning blocks).
Not from the script. It is rendered by caption.py (~line 599) from the vocal-burst blend regression score (blend_0_10), z-scored against the bundled baseline: with-bursts if the z-score is clearly positive, burst-free if clearly negative — and the word is omitted entirely when the score sits in the neutral band. Same mechanism as genuine / semi-genuine / performed, which comes from the genuineness score. So GENERAL's last words are score readings, while the script's (Chuckle) tags are detector spans — two independent pipelines that can disagree.
Measured agreement (EN synth training sample, n=1500): with-bursts appears in 1,393 prompts (92.9%), but only 36.4% of those have any visible burst tag in their script. Conversely, of 12 burst-free prompts, 4 (33%) visibly contain a burst tag. On the DE tier0 sample (n=2000): with-bursts 1,714 (85.7%), burst-free 64, neither 222.
Why the model says "with bursts" so often. It is the majority class (85–93% of GENERAL lines), so guessing it is right most of the time without listening at all. That is also why the short caption "works": the template is closed-vocabulary and the last token is nearly constant.
On the 60 demo clips (greedy backbone decoding, no beam search, no repetition penalty) the new model reaches a mean ASR word error rate of 2.176 (German 0.911 / English 3.441) — worse than one error per word, driven by repetition collapse (ja, ja, ja, …). The GENERAL part of the captions, meanwhile, reads plausibly on most clips. This gap is expected, not a surprise:
| ASR (transcript) | GENERAL caption | |
|---|---|---|
| Vocabulary | open — every word of two languages, exact spelling | closed — ~57 voice dims + ~40 emotions + fixed phrases (very smooth, genuine, burst-free…) |
| Correctness | exact word order; one slipped word breaks the alignment | bag of attributes; any plausible subset looks right |
| Decoding used | greedy, no penalty, no beam — the weakest possible setup | same weak setup, but the template carries it |
| Training signal | 1 presentation per clip among 5 tasks; 639 clips had no transcript at all | present in every TTS prompt the model also trains on (constant exposure) |
What to do. Do not judge ASR by these greedy demo numbers: re-decode with beam search + repetition penalty + a maximum length, then score WER properly. If it is still bad, the fixes in order are: (a) more ASR weight (today 1/5 of presentations), (b) an explicit no-repeat loss or coverage term, (c) checking whether the talker-head audio tokens preserve enough phonetic detail at 12.5 Hz. The demo page stays as the honest before-picture.
Per ~/ARBEITSPROTOKOLL.md §11:00, the old 4-worker pack jobs (1777747) were stopped and restarted with 30 workers (job 1785746, 5 h limit) for the 50k/100k stages; packing needs no GPU but runs on booster nodes for lack of a CPU partition. Verified just now: all 60 tasks of 1785746 COMPLETED, ~7,980 packed .tar shards under out/scaling_stages/pack/ (plan talked about 11,034 new + 2,682 reuse shards — remaining counts live in that session's reports/STATE_scaling_stages.md).
DATA (re-pack required): [ ] Decide timed vs untimed script (§1, options A/B/C) and implement it in burst_captions.py [ ] Unify pause spelling to ONE pattern (today "[pause 2.6s]", spec says "[2.6 seconds pause]") [ ] Fix the with-bursts/burst-free token (§2, options A/B/C) so GENERAL and SCRIPT agree [ ] Keep the 639 transcript-less clips out of ASR (already done in multitask build) or fix transcripts CODE (must change together with the data): [ ] m2_20k_score/packing.py score_prompt() verifier regex for the new shapes [ ] m2_multitask/tasks.py bracket_contents_concat() for the new shapes [ ] Rebuild manifests (base records change) — 20k and multitask plans re-pinned (new SHAs) EVAL (before judging): [ ] ASR with beam + repetition penalty; report WER, not greedy [ ] Caption scores: emotion-tag F1 + burst-tag recall against kept spans (not against the blend word)
Evidence: prompt_schema_2026-08-25.html §3–4 · ladder/code/pack_worker.py:49–91 · procedural-voice-captions/burst_captions.py:881–969 · caption.py:592–603 · training tars publish/scaling_ladder/data/{de/real/tier0,en/synth/tier0} (n=2000+1500) · demo results out/m2_multitask/demo/wavs/results_rank*.jsonl (60 clips) · ~/ARBEITSPROTOKOLL.md §11:00 · sacct 1785746 (60/60 COMPLETED).