Nemotron 3.5 Transcribe, speech to text, on an RTX 5060 Ti (16 GB)

59:39 of audio in 120 s on an RTX 5060 Ti, 29.8× real time

One hour of audiobook speech (59:39, 19 chapters) went through the engine in 31 pieces of up to 2 minutes. The whole run took 120.3 s, including the model load. The word error rate is 1.54 % against the reference text.

What the output looks like

Where it stops working on this card

Both limits are from this 16 GB card with this quant. I report them as they came out.

All runs

The time is the wall time of one run, taken around the whole container run, from just before it starts to the moment it has exited. Speed is 3,579.4 s of audio divided by that time. The word error rate counts every wrong, missing and extra word against the reference text. Peak GPU memory is the whole card.

What to keep in mind

• What the time covers. The wall time is taken around the whole container run, from just before it starts to the moment it has exited. Starting the container, loading the model (the engine's “path=batched” line, which comes after the load, is 12.9 s after the start) and all 31 pieces are inside. Cutting the audio into pieces is not. I did that once, before the runs.
• I cut the pieces on known gaps. The test file has 0.25 s of silence between utterances, so every cut fell on a gap. A real recording has no such marks, and I did not test a way to cut one. The cutting dropped 7.5 s of silence from the audio, so the engine got 3,571.9 s. The speed above uses the full 3,579.4 s. On 3,571.9 s it would be 29.7×.
• The settings are the 16 GB settings of the quant. I took them from the README of the quant and changed nothing: max_sequences 2, n_ctx_per_sequence 8192, max_context 8192, mtp_draft 0, n_batch 512, encoder_max_batch 2, concurrency 1. The README gives a peak of 15.6 GiB on a test file of 3.7 minutes and adds “measured on an A100, not on a 16 GB card: it is tight”. Here the card showed 15.2 GiB of 15.9 GiB. The README is here.
• A quant, not NVIDIA's weights. The decoder is the AtomicChat AD-Q2_0_M quant (13.57 GB) and the encoder is F16 (1.67 GB). The word error rate is the rate of this quant. It says nothing about NVIDIA's original weights. I did not run them.
• How the word error rate is counted. Run 2 has 101 substitutions, 24 deletions and 13 insertions against 8,960 reference words after the Whisper English text normaliser. That normaliser also changes number words and spellings, so the count differs from the 8,835 words of the plain text. With a plain normaliser (lower case, no punctuation) the same output scores 1.64 % (wer.json, line 393).
• Easy audio. LibriSpeech test-clean is clean read speech from audiobooks, in English (the engine was told -l en). The result says nothing about noisy recordings, calls or other languages. I did not test them.
• Two runs. Run 2 is the one in the boxes above. Run 1 ran minutes before with the same settings and took 120.7 s, 29.6× real time, a difference of 0.4 %. The two outputs are the same text, byte for byte (all 31 files compared).
• The README on 400 s pieces. The README says that a context of 8192 holds “about 7 minutes” of audio per request. My longest piece of the 400 s run was 6.7 minutes, and one piece still came back wrong. I did not look into why.
• Not run: the original weights, other quants, other cards, noisy audio, other languages, speaker labels and the engine's server mode. I do not rate the model and give no score.
• Licence note: the model card lists OpenMDW-1.1, the engine is Apache-2.0 and the test audio is CC BY 4.0 (cut and joined by me). Check the terms before you use any of it.

Setup

• Model: NVIDIA Nemotron 3.5 Transcribe 30B-A4B
• Task: English speech in, text out (English)
• Weights: AtomicChat/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-GGUF, Hugging Face revision dbd06c5, a quant and not NVIDIA's original weights
• Quantization: AD-Q2_0_M (decoder) and F16 (encoder), AtomicChat GGUF
• Engine: NVIDIA NeMo-Speech.cpp, commit 6a62501, built from source in a Docker image (CUDA 13.3, architecture 120a), one container per run
• Settings: the 16 GB settings from the README of the quant: max_sequences 2, n_ctx_per_sequence 8192, max_context 8192, mtp_draft 0, n_batch 512, encoder_max_batch 2, concurrency 1, language en, output text
• Hardware: NVIDIA GeForce RTX 5060 Ti 16 GB (driver 595.71.05), one card in use. Before each run the card was idle with 2 MiB in use.

Exact command and run record

# Nemotron 3.5 Transcribe 30B-A4B, speech to text, on an RTX 5060 Ti 16 GB, runs of 29 Oct 2026.
# Placeholders:
#   <n>        index of the RTX 5060 Ti on its machine
#   <work>     the work folder: audio/, models/, NeMo-Speech.cpp/ (see environment.txt)
#   <runs>     the folder of the run, mounted writable as /w/runs/<label>
# Inside the container the work folder is /w. In the logs the runs are labelled e1 (run 1), e2 (run 2), r1 and r1b (whole hour),
# c1 and c2 (400 s pieces), d1 (piece 9 alone).

# One container per run, one GPU only, image built from the Dockerfile in scripts/ (see environment.txt):
docker run --rm --gpus '"device=<n>"' --user $(id -u):$(id -g) -e HOME=/tmp -v <work>:/w:ro -v <runs>:/w/runs/<label>:rw nemo-dev-2910:1 bash -c "<command>"

# The pieces of up to 2 minutes were cut once, before the runs, between utterances: scripts/make_chunks_short.py.txt <work>/audio 120 chunks120

# Run 1 and run 2 (<label> e1 and e2), the numbers on the page. The engine's 16 GB settings from the README of the quant, nothing lowered:
/w/NeMo-Speech.cpp/build/cuda-server/bin/nemo-speech transcribe /w/audio/chunks120 --model /w/models/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-AD-Q2_0_M.gguf --device cuda:0 -l en --format text --output-dir /w/runs/e2/out --concurrency 1 --asr.salm.max_sequences 2 --asr.salm.n_ctx_per_sequence 8192 --asr.salm.max_context 8192 --asr.salm.mtp_draft 0 --asr.salm.n_batch 512 --asr.salm.encoder_max_batch 2

# Whole hour in one request (<label> r1): the settings for one long request. It failed with out of memory.
/w/NeMo-Speech.cpp/build/cuda-server/bin/nemo-speech transcribe /w/audio/librispeech-test-clean-1h.wav --model /w/models/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-AD-Q2_0_M.gguf --device cuda:0 -l en --format text --output /w/runs/r1/out.txt --asr.salm.max_sequences 1 --asr.salm.n_ctx_per_sequence 73728 --asr.salm.max_context 73728 --asr.salm.mtp_draft 0 --asr.salm.n_batch 512 --asr.salm.encoder_max_batch 2 --asr.salm.encoder_ahead_seconds 1200

# Whole hour, second try (<label> r1b): the same, with smaller memory settings (n_batch 256, encoder_max_batch 1, encoder_ahead_seconds 120). It failed the same way.
/w/NeMo-Speech.cpp/build/cuda-server/bin/nemo-speech transcribe /w/audio/librispeech-test-clean-1h.wav --model /w/models/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-AD-Q2_0_M.gguf --device cuda:0 -l en --format text --output /w/runs/r1b/out.txt --asr.salm.max_sequences 1 --asr.salm.n_ctx_per_sequence 73728 --asr.salm.max_context 73728 --asr.salm.mtp_draft 0 --asr.salm.n_batch 256 --asr.salm.encoder_max_batch 1 --asr.salm.encoder_ahead_seconds 120

# Pieces of up to 400 s (<label> c1, repeated as c2): the same 16 GB settings as run 2, ten pieces.
/w/NeMo-Speech.cpp/build/cuda-server/bin/nemo-speech transcribe /w/audio/chunks --model /w/models/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-AD-Q2_0_M.gguf --device cuda:0 -l en --format text --output-dir /w/runs/c1/out --concurrency 1 --asr.salm.max_sequences 2 --asr.salm.n_ctx_per_sequence 8192 --asr.salm.max_context 8192 --asr.salm.mtp_draft 0 --asr.salm.n_batch 512 --asr.salm.encoder_max_batch 2

# Word error rate of run 1 and run 2 (<run> is run1 or run2). The 31 pieces of audio/chunks120.json are given with --chunk-list:
python wer.py --ref audio/reference-librispeech.txt --manifest audio/manifest.json --chunk-list audio/chunks120.json --hyp-chunks <run>/out --out wer.json --errors wer-errors.txt

# Word error rate of the 400 s pieces (limits/pieces-400s/out and limits/pieces-400s-repeat/out): the same line without --chunk-list, the ten pieces of audio/manifest.json are the default:
python wer.py --ref audio/reference-librispeech.txt --manifest audio/manifest.json --hyp-chunks <pieces folder>/out --out wer.json --errors wer-errors.txt

command.txt · environment.txt · sha256.txt · wer.py · make_chunks_short.py · Dockerfile · All raw files

19 chapters of LibriSpeech test-clean

The test audio is 19 whole chapters of LibriSpeech test-clean (8 speakers, 539 utterances, Panayotov, Chen, Povey and Khudanpur, ICASSP 2015, CC BY 4.0). I joined them in corpus order with 0.25 s of silence between utterances and 1 s between chapters. The file is 3,579.355 s long. I do not publish it. Its sha256 and the link to the corpus are in environment.txt. The reference text is reference-librispeech.txt, the list of the pieces is chunks120.json.

Also made this possible

• GGUF quant (AD-Q2_0_M)AAtomicChatHugging Face
• EngineNNVIDIA NeMo-Speech.cppGitHub
• Test audio (Panayotov, Chen, Povey, Khudanpur, ICASSP 2015, CC BY 4.0)LLibriSpeechOpenSLR

What is the same in the runs

Run 1, run 2 and the 400 s pieces used the same card, the same engine build and the 16 GB settings of the quant. Between them only the size of the pieces changed. The two tries with the whole hour in one request had their own settings, listed under limit 1. The only quality number is the word error rate against the reference text of the corpus. It is not a rating of the model. The text on this page is the real output of those runs, unchanged. The transcripts are machine output and AI-generated.