MINT-Bench: Testing Multilingual TTS Instructions
Explore MINT-Bench's multilingual instruction evaluation and learn to separate spoken content, requested style, and audio quality in voice tests.

MINT-Bench evaluates whether a speech system follows instructions across languages while preserving the spoken text and producing usable audio. Its important distinction is between saying the right words and saying them in the requested way. A fluent voice can succeed at the first task while failing at the second.
Research reviewed September 17, 2026.
What the benchmark measures
Huakang Chen and colleagues introduced MINT-Bench on April 20, 2026; the reviewed revision is dated August 20. It organizes instruction-following TTS evaluation around a structured taxonomy and combines content, instruction, and perceptual assessment across ten languages. The authors report that compositional and paralinguistic requests remain difficult, and that overall model rankings can differ from results in particular languages. Read MINT-Bench.
The paper provides a benchmark for evaluating controllability. It does not imply that every voice tool supports free-form instructions, or that the same prompt syntax can be transferred between products unchanged.
Separate the script from the performance request
An original test could use the script “The next session begins after the break” with the instruction “Read this as a calm announcement.” The spoken content and the desired delivery are different inputs.
Keep them separate in the evaluation record. Otherwise, an instruction accidentally spoken aloud might be mistaken for a content error in the script itself.
Write a plain-language acceptance note: all words should be present, the announcement should sound calm, and the audio should remain clear. Review those requirements independently.
This separation also helps resolve ambiguity. If reviewers disagree about what “calm” means, rewrite the request or provide an authorized reference example before treating the model's output as a failure.
Test one control before combining several
Begin with simple requests that target one property. Then add combinations such as a particular pace and a particular emotional tone.
The point is to locate the boundary of competence. If a system handles each request separately but fails when they are combined, the combined instruction is the interesting test case.
Use the same script across conditions where possible. A sentence filled with emotional vocabulary can make it easier for listeners to infer a mood from the words rather than the voice.
Include paraphrases of the instruction. “Read this calmly” and “Use a measured, composed delivery” should be evaluated as different phrasings of an intended request, not assumed to be identical commands inside a model.
Make multilingual comparisons linguistically fair
Translate both the script and the intended performance brief with qualified language review. A literal translation of a style instruction may not describe the same social situation.
Keep a note explaining the intended meaning and audience. For example, a formal public announcement and an informal reminder may require different wording even when their information is equivalent.
Do not force identical word counts across languages. Instead, document length and structure so that reviewers understand the material being compared.
Our multilingual voice-over workflow covers production coordination. For a research evaluation, add explicit controls for script equivalence, instruction interpretation, and the language competence of the reviewers. These controls help prevent translation quality from being mistaken for synthesis quality.
Score content, control, and listening quality separately
Use a review sheet with one section for required words, one for instruction fulfillment, and one for the listening experience.
For content, mark omissions, substitutions, and unexpected additions. For control, ask whether the requested property is audible. For quality, inspect continuity, pronunciation, and distracting artifacts.
A useful illustrative outcome is “correct words, weak style, clear audio.” That diagnosis points toward instruction following. Another is “strong style, missing phrase, clear audio,” which points toward content preservation.
Retain these combinations in the report. An aggregate score can be useful for summary, but it should not erase the reason an output fails. When automatic evaluators are involved, manually inspect a sample of both accepted and rejected outputs.
Examine the evaluator's language coverage
A multilingual generator and a multilingual judge are separate systems. Evidence that one covers a language does not validate the other.
Ask whether the evaluator understands the instruction, can recognize the spoken words, and can assess the requested style in that language. If those abilities are uncertain, use qualified human review before treating its score as decisive.
Keep reviewer disagreement visible. Some expressive requests admit several valid performances, so a narrow preferred example may be an unsuitable reference for all outputs.
Also separate accent preference from task success. If the request does not specify an accent, reviewers should not automatically penalize a valid variety merely because it differs from their own. The rubric should state what is required and what is allowed to vary.
Build a diagnostic report instead of a single league table
Create rows by language and instruction type. Include simple controls, combined requests, paraphrased requests, and unsupported conditions.
For each row, show the number of evaluated examples and the main failure categories. Keep a small collection of representative audio pairs so that a reader can connect the score to an audible difference.
Our guide to translating voice-over scripts helps prepare the underlying text, but research on control requires an additional question: did the system realize the performance request?
MINT-Bench is valuable because it makes that question explicit across languages. Our interpretation is that a useful follow-up study should prioritize diagnostic coverage over a dramatic overall ranking. A model may be suitable for a narrow multilingual task even when it is weaker on open-ended expression, and a strong overall result may still conceal the exact instruction type your project needs.
Try Our Voice Clone Demo
Hear your words come to life
Choose a voice and try a short preview.
Listen to sample voices
Hear examples before choosing a voice. Generated results can vary with the script and reference sample.
Looking for another voice?
Explore the library and listen to a sample before you create.
Morgan Freeman
Stephen Hawking
Christiano Ronaldo
Donald Trump
Kokoro
Disney XD Announcer
Cute Japanese Girl
Vin
Adam Stone
Transform Your Content with AI Voice Technology Today
Try a short voice preview, then create speech and save your audio in a workspace built for your next project.
Generate Your Voice Now