AI music detector accuracy is useful only when the test material and errors are defined. Use files with documented origins, distinguish missed AI files from wrongly flagged human recordings, and keep failed analyses separate.

Start with documented origins

Choose material you may submit and whose creation history you know. Keep generator records for generated music and recording or project records for human performances. List mixed-origin work separately rather than forcing it into a label you cannot justify.

Select music resembling what you plan to check. Results on one genre, one generator or near-identical exports cannot establish performance across a different catalogue. Count related versions of a song as related observations, not independent evidence of coverage.

Count both directions of error

A false positive is a human recording flagged as AI; a false negative is an AI recording missed under your chosen decision rule. Record those counts separately, together with the number of inputs that could not be analyzed.

For illustration, a set of ninety human tracks and ten AI tracks gives a classifier that always answers “human” 90% accuracy, even though it misses every AI track. This is arithmetic, not a result for any service. The test set’s composition changes what the headline number tells you.

Fix the decision rule before testing

Define how scores become verdicts before inspecting outcomes. Do not move a threshold for each song or retain only the most favourable export. When comparing services, record each service’s categories rather than averaging scores with potentially different meanings.

SH Labs’ terms describe probabilistic predictions and variable accuracy. A score is not a measured percentage of generated instruments, and a label is not conclusive proof of the production history.

Use results to improve a review process

The Demorphus checker lets you inspect a file and retain its report. When an outcome is surprising, verify the actual input and its history first. Our false-positive guide explains how to review a flag without jumping to an accusation.

For an evaluation, report input counts, categories, dates, detector identity and each error type. State what the sample excludes. A small test can help choose a workflow; it cannot certify every future result. If you process the file afterward, assess its sound and detector response as a separate comparison.

Prepared with AI assistance. Sources below.

Sources and documentation