AI-powered voice deepfake detection and cognitive health monitoring for elderly users — delivered entirely through real phone calls.
Live App: hughknew.lovable.app
Demo Video: youtu.be/IyitBw9--3c
Built for the Clawckathon 2026 (Telnyx Hackathon, March 20–22)
Americans over 60 lost $3.4 billion to fraud in 2023, with phone calls as the #1 attack vector. The FBI received over 101,000 elder fraud complaints that year — and the real number is estimated to be 5–10x higher due to underreporting.
AI voice cloning has made this dramatically worse. It now takes as little as 3 seconds of audio to clone someone's voice. Voice phishing attacks grew 550% between 2021 and 2023, and deepfake voice fraud increased 10x between 2022 and 2023. The classic "grandparent scam" — where a caller impersonates a loved one in distress — now carries a median loss of $9,000 per victim.
Elderly users are the most vulnerable because they trust the phone, but they lack the tools to verify who's really on the other end.
HughKnew is a web dashboard paired with real PSTN phone calls that helps elderly users:
- Register trusted voices — Build a "Voice Circle" of contacts whose voices are cloned and enrolled for identity verification
- Practice spotting scams — Receive AI-powered training calls that simulate realistic scam scenarios using their contacts' cloned voices, at adjustable difficulty levels
- Detect deepfakes — Upload any suspicious audio clip for instant AI-powered deepfake detection
- Stay cognitively sharp — Receive daily companion calls from "Hugh," an AI companion that conducts friendly cognitive health check-ins and tracks verbal fluency, memory recall, and conversational flow over time
- SMS notifications — Receive text message summaries after every companion call with your wellness score, and get SMS reminders before scheduled daily check-ins
Every interaction happens over a real phone call or text message — no apps to install, no new technology to learn. Just the phone they already know how to use.
Live App: hughknew.lovable.app
Demo Video (click thumbnail to watch on YouTube):
Users add trusted contacts and register their voices through three methods:
- Shareable recording link — Send a link to the contact; they record their voice in-browser
- Verification call — HughKnew calls the contact directly, plays a prompt, and records ~10 seconds of natural speech
- Manual upload — Upload an existing recording
Once captured, the audio goes through a pipeline:
- Uploaded to Supabase Storage
- Sent to Telnyx for voice cloning (used in training calls)
- Sent to Resemble AI for identity enrollment (used for verification)
Users choose a registered contact and a difficulty level (Easy, Medium, Hard). Their phone rings. On the other end is a Telnyx AI Assistant speaking in their contact's cloned voice, running a scam scenario:
| Difficulty | Approach | Exchanges |
|---|---|---|
| Easy | Obvious — immediately asks for money, PINs | 2–3 |
| Medium | Starts with small talk, escalates to a plausible story | 4–5 |
| Hard | Emotional manipulation — fake hospital emergency, secrecy, time pressure | 5–6 |
The AI uses a WebhookTool to report results to our server in real-time, and a HangupTool to end the call after delivering a verbal reveal with tips. After hangup, the full transcript is analyzed by Telnyx's LLM API to score whether the user caught the scam or fell for it. Results push to the dashboard instantly via Supabase Realtime.
Each contact in the Voice Circle has an "Upload Audio to Check" button. Users can upload any suspicious audio clip (a voicemail, a recording of a call) and it's analyzed by Resemble AI's deepfake detection API, which returns whether the audio is likely real or AI-generated with a confidence score.
Users click "Have Hugh Call Me" and within ~90 seconds, Hugh — an AI companion powered by ClawdTalk with OpenClaw — calls their phone for a friendly cognitive health check-in.
Hugh follows a structured conversational assessment:
- How they've been doing lately
- A favorite memory, with follow-up questions for specific details
- Verbal fluency test (name as many fruits/animals as you can)
- Short-term memory (what did you do yesterday?)
- Temporal orientation (what day of the week is it?)
After the call, ClawdTalk provides the transcript and conversation insights. Our server scores verbal fluency, memory recall, and conversation flow, then pushes results to the dashboard with a detailed call report including score trends and recording playback.
Automated daily scheduling: Users set a preferred daily call time in the Companion tab. A cron-triggered endpoint checks which users are due, sends an SMS heads-up ("Hugh will call you in 2 minutes"), and automatically schedules the companion call — no manual button press needed. Post-call, users receive an SMS summary with their wellness score.
Frontend (React/Vite/TypeScript) ─── Lovable (hosting)
├── Supabase (Auth + Postgres + Storage + Realtime)
└── Webhook Server (Express/Node.js) ─── Fly.io (hosting)
├── Telnyx Call Control
│ ├── Outbound calls (dial, speak, record, hangup)
│ ├── AI Assistants (scam roleplay with cloned voices)
│ │ ├── HangupTool (AI-controlled call termination)
│ │ └── WebhookTool (real-time result reporting)
│ ├── Voice Cloning (contact voice → training calls)
│ └── LLM API (transcript analysis, scam scoring)
├── ClawdTalk + OpenClaw
│ ├── Companion AI Assistant (cognitive health instructions)
│ └── Scheduled calls via Events API
└── Resemble AI
├── Identity Enrollment (voice profile creation)
├── Identity Search (speaker verification)
└── Deepfake Detection (real vs AI-generated audio)
- PSTN-native: All calls go through real phone networks via Telnyx Call Control — no WebRTC, no apps. This is critical for elderly users who are comfortable with their phone but not with technology.
- AI Assistant tools: Telnyx AI Assistants use HangupTool and WebhookTool to autonomously manage call flow — the AI decides when to reveal the training exercise, reports results, and hangs up cleanly.
- Real-time updates: Supabase Realtime channels push call status, training results, and companion scores to the dashboard as they happen — users see results before they put the phone down.
- Elderly-first design: 18px minimum font, 48px touch targets, WCAG AAA contrast, skeleton loaders instead of spinners, generous spacing, large accessible buttons.
ClawdTalk is the primary platform powering the Companion Call feature — the daily cognitive health check-in that is central to HughKnew's value proposition.
How we use ClawdTalk:
- Custom AI Assistant (
POST /v1/assistants) — We created a dedicated "HughKnew Companion" assistant with custom instructions for cognitive health assessment, including a structured conversation flow covering memory recall, verbal fluency, temporal orientation, and conversational coherence. - Scheduled Calls (
POST /v1/assistants/:id/events) — Companion calls are scheduled through ClawdTalk's Events API, which handles Telnyx call origination under the hood. Users can trigger an immediate call or set a preferred daily time. - Transcript & Insights (
GET /v1/calls/:id/assistant-data) — After each call, we fetch the full conversation transcript and ClawdTalk's conversation insights to score cognitive health metrics. - Call Status Polling (
GET /v1/calls/:id) — The frontend polls call status to detect when conversations end, then triggers our scoring pipeline. - OpenClaw Integration — The companion AI has access to OpenClaw's deep tool system for context-aware responses during conversations.
ClawdTalk enables us to deliver a warm, intelligent, voice-first companion experience without building our own conversational AI infrastructure — we define the instructions, and ClawdTalk handles the voice synthesis, conversation management, and call orchestration.
ClawHub Skill: clawhub.ai/noahvandal/persona-plugin
We built and published the Persona Plugin to ClawHub as a reusable ClawdTalk skill that gives any voice agent persistent, evolving caller memory. We developed it specifically for HughKnew's companion calls, then published it to ClawHub so other ClawdTalk developers can add caller memory to their own assistants.
The Persona Plugin maintains three living documents per caller:
| Document | Purpose |
|---|---|
| Soul | Personality traits, communication style, and conversational preferences — what makes this caller unique |
| Identity | Name, relationship context, personal details, and biographical facts the AI should know |
| Memory | Accumulated context from past calls — topics discussed, recurring interests, health observations |
How it works in HughKnew:
- When a companion call starts, the Persona API looks up the caller by phone number and compiles their Soul + Identity + Memory documents into a prompt context string
- This context is injected into the ClawdTalk companion assistant's instructions, so Hugh greets the user by name, references previous conversations, and adapts its tone to their personality
- After each call, the transcript and insights are fed back into the Persona system, which updates the Memory document — so the next call builds on the last one
- The Persona API is mounted inside our webhook server at
/persona/*and exposes REST endpoints for caller profiles, document management, and call history
This creates a longitudinal cognitive health record — Hugh doesn't just check in, Hugh remembers. If a user mentioned their grandchild's birthday last week, Hugh can ask how the party went. If verbal fluency scores have been declining, Hugh can gently adjust conversation complexity. We built it as a standalone plugin so it's not locked into HughKnew — any ClawdTalk assistant can install it from ClawHub and get the same persistent memory system.
Telnyx powers the telephony backbone and AI-driven training calls:
- Call Control API — Outbound dialing, TTS, recording, playback, hangup with client_state tracking across webhook events
- AI Assistants — Three difficulty-tuned scam roleplay assistants with custom instructions, voice cloning, and tool calling (HangupTool, WebhookTool)
- Voice Cloning — Contact voices cloned for use in training calls, making scam simulations sound like real family members
- LLM API — Post-call transcript analysis to determine if the user caught the scam or fell for it
- Verification Calls — Automated voice enrollment pipeline (call → prompt → record → process → clone)
Resemble AI provides voice identity and deepfake detection:
- Identity Enrollment — Voice identity profiles created from contact recordings for speaker verification
- Identity Search — Match incoming audio against enrolled identities to verify callers
- Deepfake Detection — Upload any audio clip to determine if it's real or AI-generated, with confidence scoring
Lovable hosts the frontend React/Vite application with automatic deployments from the repository. The UI was deliberately designed for elderly users with specific accessibility-first decisions:
- High-contrast color palette — Teal accent on dark backgrounds with WCAG AAA contrast ratios, ensuring readability for users with age-related vision changes
- Large, clear typography — 18px minimum body text, 24px+ headings, tabular numerals for scores so digits don't shift
- Oversized touch targets — All buttons are at minimum 48px tall with generous padding, reducing mis-taps for users with reduced motor precision
- No spinners, no jargon — Skeleton loaders instead of spinners (less anxiety-inducing), plain-language labels, and non-technical error messages
- Generous whitespace — Cards, sections, and controls are spaced apart so nothing feels cramped or overwhelming
- Subtle animations only — Gentle fade-in-up transitions that feel polished without being disorienting; no flashy motion that could cause discomfort
- Dark mode support — Class-based theming via
next-themeswith clear visual hierarchy in both modes - Mobile-first layout — Designed at 375px and scaled up, since many elderly users browse on phones or tablets
- Node.js 18+
- pnpm
pnpm install
pnpm devcd server
pnpm install
cp .env.example .env # Fill in API keys
pnpm dev| Variable | Where | Purpose |
|---|---|---|
VITE_SUPABASE_URL |
Frontend | Supabase project URL |
VITE_SUPABASE_ANON_KEY |
Frontend | Supabase anonymous key |
VITE_WEBHOOK_SERVER_URL |
Frontend | Webhook server URL |
TELNYX_API_KEY |
Server | Telnyx API authentication |
TELNYX_CONNECTION_ID |
Server | Telnyx Call Control app |
TELNYX_PHONE_NUMBER |
Server | Outbound caller ID |
CLAWDTALK_API_KEY |
Server | ClawdTalk API authentication |
RESEMBLE_API_KEY |
Server | Resemble AI authentication |
SUPABASE_URL |
Server | Supabase project URL |
SUPABASE_SERVICE_ROLE_KEY |
Server | Supabase service role (bypasses RLS) |
Built by the HughKnew team for Clawckathon 2026.
- Caregiver SMS escalation — Expand the existing SMS system to automatically text designated family members or caregivers when cognitive scores drop below a threshold or show a declining trend, turning HughKnew into an early-warning system for dementia onset
- Two-way SMS conversations — Let users text Hugh directly using ClawdTalk's inbound SMS handling for quick check-ins, medication reminders, or to request a callback — extending the companion relationship beyond scheduled calls
- Inbound companion calls — Let users call Hugh directly at any time using ClawdTalk's inbound call handling, so they don't have to wait for a scheduled call or use the app — just dial the number and Hugh picks up
- Caregiver dashboard — Family members can monitor cognitive health trends and scam readiness remotely, with SMS-delivered weekly digest reports
- Real-time inbound call screening — Intercept incoming calls via Telnyx Call Control, run voice samples through Resemble AI deepfake detection in real-time, and block or flag suspicious calls — then SMS the user and their caregiver with the result
- Multi-language support — Extend companion calls, scam training, and SMS notifications to non-English-speaking elderly populations
- Integration with telecom providers — Partner with carriers to offer HughKnew as a built-in call screening service
- Clinical validation — Partner with healthcare providers to validate cognitive scoring against established assessments (MoCA, MMSE)