You say 你好, tap stop, and half a second later an app tells you your first syllable scored 94 and your third tone "sounded like a second tone". Where does that verdict come from? Is it trustworthy? And when it disagrees with your teacher, who is right?
This is the honest version, written by the team that builds TonePerfect and its pronunciation assessment engine. It is the same explanation we give developers who use our pronunciation assessment API, minus the code. By the end you will know what a Mandarin pronunciation score actually measures, why it splits every syllable into three parts, why tone sandhi trips up naive scorers, and which situations still fool the AI.
Want to see it on your own voice first? Take the free two-minute test — no account, in the browser — then come back and read what the numbers mean.
Step 1: the app decides what you were supposed to say
Every scripted pronunciation checker starts from the text. 你好世界 becomes four syllables, and each syllable becomes three targets: an initial (the consonant, like n or sh), a final (the vowel part, like i or ao) and a tone (1–4, or neutral). That is not a technical convenience; it is how Mandarin is taught, and it is why a good scorer reports those three things separately instead of one number per word.
This step already has a trap in it. Dictionaries store canonical tones: 你 is tone 3, always. But nobody says two full third tones in a row. 你好 is pronounced ní hǎo — the first syllable becomes a second tone. This is third-tone sandhi, it is obligatory, and a scorer that compares you against the dictionary will mark your correct pronunciation as wrong. More on that below, because it is the single most common way pronunciation apps mislead learners.
Step 2: the AI lines up your audio with those targets
A neural acoustic model — trained on thousands of hours of Mandarin — listens to your recording frame by frame (a frame is about 20 milliseconds) and estimates, for each frame, which sound is being made. A search algorithm then finds the best way to fit the expected sequence n-i-3, h-ao-3, sh-i-4, j-ie-4 onto those frames. This is called forced alignment, and it is what lets the app say "the syllable 世 is between 1.26 and 1.64 seconds" and play it back to you.
If a syllable cannot be aligned at all — you skipped it, or the recording cut off — that is what "completeness" scores measure.
Step 3: each part gets compared with native speakers
For the initial and the final, the model asks two questions about the aligned slice of audio: how much does this look like the expected sound? and if it looks like something else, what? The second question is what produces the "heard" value — "expected sh, heard s" — and it is the part of the feedback you can actually act on. A number alone tells you the syllable was off; the heard sound tells you how.
For the tone, pitch is extracted from the syllable and a separate model, trained on a large set of learner syllables labelled for tone and checked against expert ratings, decides which tone contour was produced and how confidently. Pitch tracking is noisy — creaky voice, background noise and very low or high voices all distort it — which is why serious engines train a model on labelled learner data rather than hand-writing rules like "tone 4 should fall".
Crucially, the score is placed on a scale calibrated against native speakers, per sound. Natives vary a lot on some finals and very little on others; a learner who is a bit off on a sound where natives are all over the place should not be punished as hard as one who is off on a sound natives produce identically. This is also why scores are not percentages: 72 does not mean "72% correct", it means "this far from typical native speech on this particular sound".
Step 4: deciding whether to tell you
Here is the part most apps skip. A score says how far you were from native; it does not say whether that gap is worth interrupting you for. A slightly off-centre vowel that scores 78 would not make a teacher stop the lesson. An engine that flags every 78 buries the one thing you actually need to fix under a wall of amber.
Worse is the opposite mistake: telling a learner they were wrong when they were right. We measure this deliberately. Before any model reaches the app, we run it over thousands of recordings by native speakers from public research corpora and count how often it accuses them of an error. That number — false accusations on natives — is the one we care about most, because a learner who is corrected for something they said correctly stops trusting the app, and rightly so.
So the engine produces two things per component: a score for display and progress, and a decision — correct it, uncertain, leave it — that the app uses to decide what to say. When you see TonePerfect show a modest score but no correction, that is not a bug; it is the engine saying "not native-like, not worth stopping for".
The sandhi problem, properly
Back to 你好. When we measured our own engine scoring third-tone-plus-third-tone sequences against the dictionary tone, its ability to tell a right tone from a wrong one in that context was no better than a coin flip — on the exact context where learners make the most tone mistakes. The fix was not to retrain the model (learners produce correct sandhi only about one time in ten, so relabelling the training data teaches the model that most 3-3 sequences are second tones, which breaks everything else). The fix was to accept either realisation at scoring time — tone 3 or tone 2 are both right for the first syllable of 你好 — while still reporting which one was heard.
The same is done for 不 before a fourth tone (不对 bú duì). It is deliberately not done for 一, because 一 changes tone depending on the word it is in — it stays tone 1 in 第一 and when counting — and applying the rule blindly measurably made things worse. If you are a developer, the technical write-up is in Tone sandhi in pronunciation scoring.
What this means for you: if an app marks your ní hǎo wrong, the app is wrong. If it shows "expected 3, heard 2" with a good score, it is showing you sandhi in action.
Where the AI still gets it wrong
We would rather you knew these than found out the hard way.
Very short or clipped recordings. If the microphone starts late or you stop early, alignment has nothing to hold on to and the tone model guesses. Leave a beat of silence before and after.
Speaking much faster or slower than the model expects. Tone models are sensitive to tempo. Stretching a syllable to "make the tone clear" can move it out of the range the model was trained on and cost you points you did not earn — the opposite of what learners expect. Speak at a natural pace.
Creaky voice and whispering. Pitch tracking needs voicing. If you drop into vocal fry on a third tone (many English speakers do), the contour becomes unreadable and the tone score becomes unreliable.
Regional accents. The reference is standard Mandarin. A speaker with a southern accent who merges sh and s will be told so — that is the engine doing its job, but it is worth knowing that "native" here means Putonghua, not any native speaker anywhere.
Sounds outside the model's coverage. Every engine has a list of sounds it does not assess well; honest ones tell you. Ours publishes the list per language in its API documentation and marks those sounds as "not assessed" rather than "wrong".
Meaning, stress and emotion. The engine grades sounds. It does not know whether you said the right word, whether your emphasis matched the context, or whether you sounded friendly. A tutor does. Use the AI for the thing it is good at — tireless, consistent, per-syllable feedback on hundreds of repetitions — and a human for the rest.
How to use scores without being misled
- Read the heard sound, not just the number. "Expected sh, heard s" is a drill; 68 is a feeling.
- Fix one thing per attempt. The engine already picks the most confident error; ignore the rest for now.
- Compare yourself with yesterday, not with 100. Scales are calibrated per sound; a 100 on every syllable is not what native speech looks like.
- When a correct sandhi is flagged, distrust the app, not yourself.
- Record naturally. Normal pace, normal volume, a beat of silence on each side.
Try it on your own voice
The free TonePerfect test scores tone, initial and final for every syllable you say, in the browser, in about two minutes. If you build language software and want the same engine in your own product — for Mandarin and six other languages — it is available as a pronunciation assessment API.