Fault-code test plan
How each AI tool will be scored once shop testing starts. Nothing has been scored yet.
1. Real cases only
Each test case is a real car from the shop with a confirmed repair: the fault codes, freeze frame, symptoms, mileage and year/make/model/engine are recorded before the fix, and the confirmed root cause is recorded after. Identifying customer details and full VINs are removed.
2. Same inputs for every tool
Every tool gets the same codes, symptoms and vehicle info, entered the way the tool expects (pasted code, scan-tool session, or repair order). Screenshots of each answer are saved.
3. Case mix
- Common powertrain codes (e.g. misfire, lean, catalyst efficiency)
- Network / U-codes and module communication faults
- Intermittent and no-code drivability complaints
- Platform-specific known failures
- A few cases where the obvious code-definition answer is wrong
4. Scoring per case
| Measure | What it means |
|---|---|
| Top-1 hit | Confirmed root cause is the tool's #1 suggestion |
| Top-3 hit | Confirmed root cause is in the top 3 |
| Test-first | Tool recommends a test that would confirm the cause before replacing parts |
| Parts-cannon risk | Tool recommends replacing a part that wasn't the fault, with no test first |
| Safety / harm | Any advice that is unsafe or could damage the vehicle |
| Time to answer | Seconds from submitting to a usable answer |
5. Cost
Real cost to use for a year at the listed price, including required subscriptions and renewals.
6. Disclosure
Paid "verified" listings never change scores. Any tool provided free by a maker, or any affiliate link, is disclosed on that tool's page.
Get new AI diagnostic tool tests by email
Placeholder: signup is not connected yet.Listings use only public info from makers and cited sources. Prices are as listed on the date checked and can change. No tool here has been tested by us yet.