The 6-layer detection stack that catches what legacy panels miss
PNAS 2025: AI bots evade legacy fraud detection 99.8% of the time. Carevoices' Q3 2025 Panel Fraud Transparency Report measured 0.4% verified leak rate across 1,108 audited transcripts — versus 18-31% on benchmarked legacy panels.
Tell me about the moment you decided to switch providers.
Trust and transparency are the #1 decision drivers across all segments.
Trusted by teams at
AI-mediated research participation has crossed the threshold where it materially contaminates healthcare research. PNAS 2025 measured AI bots evading legacy fraud detection 99.8% of the time. Carevoices' Q3 2025 Panel Fraud Transparency Report audited 1,108 transcripts and measured 18-31% leak rates on benchmarked legacy panels vs. 0.4% on the Carevoices panel. The difference is a purpose-built 6-layer detection stack: KYC, license + NPI verification, voice baseline, AI-on-AI dynamic challenge, behavioral fingerprint, and payment infrastructure that catches duplicate-device patterns. For research informing FDA submissions, the verification gap is material risk.
Why Legacy Fraud Detection Stopped Working in 2025
AI-mediated research participation moved from possible to easy in 2024-2025. Legacy fraud detection — bot-trap questions, IP fingerprinting, completion-time outliers — was built for an era when fraud meant farm workers, not LLMs. PNAS 2025 confirmed: legacy detection misses AI bots 99.8% of the time.
Legacy Fraud Detection Misses LLMs
Bot-trap questions, IP fingerprinting, completion-time outliers, attention checks — all designed for human-fraud patterns. LLMs answer attention checks correctly, type at human cadence, and respond to bot-traps with plausibly expert text. PNAS 2025: 99.8% evasion.
Self-Attestation Panels Are Most Exposed
When a panel relies on self-attestation, an LLM-mediated participant claims to be a cardiologist, fabricates the workflow detail, and the panel has no rebuttal. Our Q3 2025 audit measured 18-31% leak rates on benchmarked legacy survey panels.
Survey Tools Have No Voice Signal
Text-only survey tools cannot establish a voice baseline, cannot run AI-on-AI dynamic challenges, and cannot detect prosody anomalies. The voice modality is structurally necessary for AI-respondent detection — and structurally missing from generic research tools.
Healthcare Research Has Highest Stakes
Consumer brand research can survive 20% sample contamination. FDA submission documentation, hospital workforce decisions, and pharma launch strategy cannot. The verification gap shifts findings materially when even 20% of a panel is AI-mediated.
The 6 Layers That Drive 0.4% Leak Rate
What matters most to teams after switching to AI-moderated research.
Identity verification at panel onboarding. Government-ID or institutional-email validation, depending on practitioner type. Catches identity-fabrication before the panel touches a study.
Credentials checked against state board records and the NPPES registry. Re-checked on a rolling basis. Documentation available for pharma audit and IRB review.
Cadence, prosody, semantic patterns, and engagement signals captured at first interview. Subsequent interviews compare against the baseline to flag anomalies.
AI moderator runs dynamic in-conversation challenges (clinical workflow probes, role-specific edge cases, latency-sensitive prompts) that catch LLM mediation in real time.
Continuous fingerprint across every interview the practitioner participates in. Drift detection flags accounts that may have changed hands or mediation.
Duplicate-device, farm-pattern, and payment-fraud signals caught at honoraria payment. Prevents single operators from running multiple panel identities.
What Is AI-Respondent Fraud Detection?
AI-respondent fraud detection is the methodology for identifying research participants who are using LLMs to mediate or fabricate their responses. Carevoices' 6-layer detection stack combines KYC at intake, license + NPI verification, voice baseline, AI-on-AI dynamic challenge, behavioral fingerprint, and payment infrastructure to measure 0.4% verified leak rate — versus 18-31% on benchmarked legacy panels.
Key Questions About AI-Respondent Fraud Detection
AI-respondent fraud detection is the methodology for catching research participants using LLMs to mediate or fabricate responses. Carevoices' 6-layer detection stack combines KYC at intake, license + NPI verification, voice baseline, AI-on-AI dynamic challenge questions, behavioral fingerprinting across interviews, and payment infrastructure that catches duplicate-device and farm patterns. The Q3 2025 Panel Fraud Transparency Report audited 1,108 transcripts and measured 0.4% verified leak rate on the Carevoices panel — vs. 18-31% on benchmarked legacy survey panels and a PNAS 2025 baseline of 99.8% bot evasion against legacy detection.
How big is the AI-respondent problem in research?
PNAS 2025 measured AI bots evading legacy fraud detection 99.8% of the time. Carevoices' Q3 2025 audit measured 18-31% AI-respondent leak rates on benchmarked legacy survey panels.
What's in the 6-layer detection stack?
(1) KYC at intake, (2) license + NPI verification against state and NPPES records, (3) voice baseline at first interview, (4) AI-on-AI dynamic challenge during interviews, (5) behavioral fingerprint across interviews, (6) payment infrastructure that catches duplicate-device and farm patterns.
What is the Carevoices verified leak rate?
0.4% verified leak rate across 1,108 audited transcripts in the Q3 2025 Panel Fraud Transparency Report. The full methodology and findings are published openly.
Why does this matter for healthcare research?
Research informing FDA submissions, hospital workforce decisions, or pharma launch strategy can't tolerate 18-31% sample contamination. Findings shift materially when even 20% of a panel is AI-mediated.
What Each Layer Catches
The six layers operate independently and combine — no single signal is the verdict.
KYC at intake
Identity verification at panel onboarding through government-ID or institutional-email validation, depending on practitioner type.
License + NPI verification
Credentials checked against state board records and the NPPES registry, re-checked on a rolling basis.
Voice baseline
Cadence, prosody, semantic patterns, and engagement signals captured at first interview as a reference for subsequent interviews.
AI-on-AI dynamic challenge
AI moderator runs dynamic in-conversation challenges — clinical workflow probes, role-specific edge cases, latency-sensitive prompts — that catch LLM mediation in real time.
Behavioral fingerprint
Continuous fingerprint across every interview the practitioner participates in. Drift detection flags accounts that may have changed hands or mediation.
Payment infrastructure
Duplicate-device, farm-pattern, and payment-fraud signals caught at honoraria payment. Prevents single operators from running multiple panel identities.
How the 6 Layers Combine on a Real Study
Same simple process, whether you're running 10 interviews or 1,000.
Layers 1-2 run at panel intake
Every clinician runs through KYC and license + NPI verification before joining the panel. Failed verification means no panel access; the panel is curated for verification depth, not self-reported volume.
Layer 3 establishes voice baseline
Cadence, prosody, semantic patterns, and engagement signals captured during the first AI-moderated interview. The baseline is the reference for every subsequent interview.
Layers 4-5 run continuously
AI-on-AI dynamic challenge questions inject during the conversation. Behavioral fingerprint compares each interview to the baseline; drift triggers re-verification.
Layer 6 catches farm patterns
Payment infrastructure detects duplicate device, farm-pattern, and payment-fraud signals. Suspect accounts are flagged, paid out, and removed from future studies.
Carevoices 6-Layer Stack vs. Legacy Survey Panel
vs. UX Research Panel
| Dimension | Carevoices | Legacy survey panel | UX research panel |
|---|---|---|---|
| Verified AI-respondent leak rate | 0.4% (Q3 2025 audit) | 18-31% (Q3 2025 audit) | Not measured |
| KYC at intake | Yes | No | No |
| License + NPI verification | Standard | Self-attestation | Self-attestation |
| Voice baseline | Native | Not applicable (text-only) | Limited |
| AI-on-AI dynamic challenge | In every interview | Static bot-trap questions | Static bot-trap questions |
| Behavioral fingerprint | Continuous | IP / device only | IP / device only |
| Payment fraud detection | Standard | Limited | Limited |
Where Fraud Detection Makes the Most Difference
Methodology Published Openly
The Q3 2025 Panel Fraud Transparency Report audited 1,108 transcripts. Methodology and findings are public so healthcare buyers can audit Carevoices' claims rather than take vendor numbers on faith.
What's measured
- AI-respondent leak rate (verified)
- Time-to-detection per layer
- False-positive rate (legitimate clinicians flagged)
- False-negative rate (LLM-mediated missed)
- Per-specialty breakdown of detection efficacy
What's published
- Q3 2025 Panel Fraud Transparency Report (1,108 audited transcripts)
- Methodology for each detection layer
- Comparison panel selection process
- Audit panel benchmarking results
- Roadmap for Q4 2026 and 2027 audit cycles
Why we publish
- Healthcare buyers can audit Carevoices' claims directly
- Research informing FDA submissions deserves transparent verification
- Industry-wide AI-respondent transparency raises the floor
- PNAS 2025 evasion findings demand vendor-level response
Read the full Q3 2025 Panel Fraud Transparency Report at /research/panel-fraud-transparency-report-q3-2025/.
"We ran 1,203 patient interviews in 48 hours for one sponsor, with HIPAA-grade de-identified transcripts delivered straight into our analysis stack, and the AI moderator went deeper than our human moderators on the first round."
Stephane Nyombaire, CEO, Nivella Health
Frequently Asked Questions
Related resources
Built for these healthcare teams
Deep-dive guides covering this topic from strategy to execution.
Solutions where verification matters most
Practical frameworks and platform-specific guides for teams ready to act.
AI-respondent fraud reading
Reference deep-dives on methodology, best practices, and applied research.
How Carevoices compares on verification
Side-by-side comparisons with competing platforms and approaches.
Read the Q3 2025 Panel Fraud Transparency Report
1,108 audited transcripts. Methodology and findings published openly. Healthcare buyers can audit our claims directly rather than take vendor numbers on faith.
Audit refreshed quarterly. Q4 2026 report due in January 2027.