Lumina

Lumina is a voice-first AI language tutor, built solo end to end. Learners hold real-time spoken conversations with an AI guide — Lumina — across Spanish, Portuguese, Croatian, and English, then reinforce each session with AI-generated quizzes, live captions, progress tracking, and a points-based leaderboard.

Key Features Built

Live voice tutoring

• Real-time spoken conversation with Lumina, powered by Google's Gemini Live native-audio model — a single model handling speech-in, reasoning, and speech-out over one bidirectional stream
• Live captions for both speakers, driven by input and output audio transcription
• A pause/resume conversation loop with bounded auto-reconnect: after a dropped connection it re-seeds the recent transcript so the tutor picks up mid-conversation instead of starting over
• Low-latency microphone capture and playback through the Web Audio API (16 kHz PCM in, 24 kHz out)
• Selectable target and native language per learner across all four supported languages

AI study & quizzes

• On-demand Vocabulary and Grammar quizzes, generated per language and difficulty by a configurable text model
• A resume-in-progress flow — leave a quiz and return to the exact question, difficulty, and tab you left
• Session-aware generation so content tracks the learner's current level

Progress, gamification & social

• Per-user history of every conversation and quiz, each reviewable in full
• Knowledge Lumens, momentum, and levels that reward consistent practice
• A cross-user Lumens leaderboard with per-user opt-out

Accounts & identity

• Email/password and Google sign-in via Better Auth, merged directly into the CMS's user model
• Account linking, so a Google login attaches to an existing email account by address
• Database-backed sessions plus bearer-token support, leaving a clean path to native mobile clients
• Transactional email — verification and password reset — via Resend

Content & admin

• A Payload CMS admin over the same Postgres database for users and content
• Per-feature model configuration through an app-settings global, so the text model can be swapped without a deploy
• A fully localized interface — every UI string maintained across all four languages, which double as both UI locales and learning targets

Engineering Highlights

Real-time bidirectional voice over a single native-audio stream Lumina's core is a live spoken conversation built on Google's Gemini Live API. Rather than stitching together separate speech-to-text, LLM, and text-to-speech services, one native-audio model handles all three over a single bidirectional WebSocket, with input and output transcription feeding the on-screen captions. The browser captures microphone audio through the Web Audio API — 16 kHz PCM upstream, 24 kHz down — and never sees the API key: each session opens with a short-lived ephemeral token minted server-side, so the credential stays on the backend. A pause/resume state machine handles the messy realities of a live connection, applying bounded auto-reconnect on a drop and re-seeding the recent transcript on reconnect, so the tutor resumes mid-conversation rather than restarting.

Live session state that survives client-side navigation A voice session that dies every time the user changes pages is unusable, so the Speak session is lifted into a persistent provider mounted above the app layout, with route-aware suspend and resume — the mic releases when the user leaves the Speak screen and reconnects, transcript re-seeded, when they return. The study flow applies the same discipline: quiz state persists and resumes at the exact question, with the reset deliberately keyed on the language/difficulty/tab tuple so it survives React StrictMode's double-effect invocation in development rather than silently resetting.

Authentication merged into the CMS, with field-level anti-cheat guards Rather than run a separate auth service alongside the CMS, Better Auth is merged directly into Payload's users collection on one Postgres schema — disabling Payload's native local strategy and adding sessions, accounts, and verification models in the same database, so there's one identity system and one source of truth. On top of that, the gamification fields that drive the leaderboard — Lumens, momentum, completed exercises — are locked to server-level access, so a client can't PATCH its own score to climb the rankings. Points are awarded only through trusted server actions.