How appleorange/fluently works
1. Top-level shape
- This is a Next.js 14 App Router application with a browser UI, a small set of server API routes, and a Redis-backed persistence/recommendation layer.
- The system’s core is an oral reading fluency assessment pipeline: record audio, transcribe with timestamps, align against the expected passage, compute metrics, generate a diagnostic report, and recommend the next passage.
2. Browser-facing app
- The main user flow lives in src/app/practice/page.tsx, with supporting pages for the landing page (/) and progress view (/progress).
- UI is assembled from components like AudioRecorder, PassageDisplay, PassageMap, DiagnosticReport, MetricsDashboard, and Nav.
- The practice page is the primary orchestration point for the assessment experience.
3. Client-side recording and playback boundary
- AudioRecorder is the key boundary between the browser and external speech services.
- The presence of src/app/api/deepgram-token/route.ts strongly suggests the browser first requests a short-lived Deepgram token from the server, then streams audio or transcription data using that token.
- This keeps API credentials off the client while allowing low-latency speech capture.
4. Server API layer
- The app exposes four focused API routes:
- deepgram-token for authentication bootstrap
- diagnose for generating the plain-language assessment report
- generate-passage for selecting or creating the next practice passage
- session for persisting and retrieving session history
- These routes form the main backend surface; the browser talks to them, and they in turn call external services or Redis.
5. Deterministic scoring pipeline
- The src/lib/alignment.ts and src/lib/metrics.ts modules implement the core grading logic.
- The flow described in the README is:
1. Deepgram returns word-level transcript data with timestamps
2. The expected passage is aligned to the transcript using a deterministic Levenshtein-style process
3. Metrics are derived from the alignment, including WCPM, error counts, pause placement, and self-corrections
- This is the main “truth layer” of the system: it produces structured results before any AI-generated narrative is added.
6. Passage generation and recommendation
- src/lib/generatePassage.ts, src/lib/passageVectors.ts, and src/lib/sessionVector.ts implement the passage-selection subsystem.
- The app maintains a library of passages and uses vector search in Redis Stack to find the next passage that matches a reader’s inferred skill profile.
- If no sufficiently close passage exists, a new one can be generated via the AI passage-generation route.
7. Redis as the longitudinal memory
- src/lib/redis.ts backs persistence and search.
- Redis Stack is doing two distinct jobs:
- storing session history over time
- supporting vector similarity search for passage recommendation
- This is the long-term memory boundary of the system, distinct from the transient per-session assessment pipeline.
8. Skill profile over time
- The README describes a per-reader skill vector with dimensions like complexity handling, register handling, WCPM, pause placement, and self-correction.
- progress/page.tsx and MetricsDashboard likely visualize this longitudinal state.
- This means the app is not just scoring a single reading; it is modeling improvement across sessions and feeding that history back into future recommendations and diagnostics.
9. AI-generated narrative layer
- src/app/api/diagnose/route.ts and src/app/api/generate-passage/route.ts are the primary Anthropic integration points.
- Claude is used only after deterministic analysis has produced structured data.
- The README emphasizes a strong boundary: Claude sees metrics, tiers, and prior-session summaries, but not raw audio or raw transcript.
10. External services and infrastructure
- Deepgram provides speech transcription with word timestamps.
- Anthropic generates the diagnostic report and likely passage text when needed.
- Redis Stack is required, not plain Redis, because RediSearch/vector search is part of the architecture.
- These are all runtime dependencies, not optional add-ons.
11. Data flow summary
- A typical practice session flows like this:
- user selects or is assigned a passage
- browser records reading
- transcription data is obtained through Deepgram
- alignment and metrics are computed in the app backend
- session data is stored in Redis
- passage recommendation is computed from the session vector
- Claude produces the human-readable report
- The browser then presents both the scorecard and the recommended next step.
12. Important architectural boundary
- The repository cleanly separates:
- interactive UI in React components
- deterministic scoring and vector logic in src/lib
- server orchestration in App Router route handlers
- external AI/speech services behind the server boundary
- That separation is what makes the system diagram straightforward: client UI, Next.js server, Redis, Deepgram, Anthropic.
13. Implementation style
- The stack is TypeScript-first with Tailwind styling and Next.js App Router conventions.
- globals.css, layout.tsx, and component CSS like DotGrid.css support the UI layer, while the “brains” live in library modules.
- Overall, this is a productized assessment pipeline rather than a generic CRUD app or a pure front-end demo.
Main entry points: page.tsx Practice flow · diagnose/route.ts Diagnose API · generate-passage/route.ts Passage API · redis.ts Redis stack
Components 22
Browser UI
React pages and components for the practice and progress experience.
- App shell · Root layout and landing page composition for the application.
src/app/layout.tsx - Landing page · Public entry page for the product.
src/app/page.tsx - Practice flow · Main orchestration page for recording, scoring, and feedback.
src/app/practice/page.tsx - Progress view · Longitudinal reader progress and history view.
src/app/progress/page.tsx - Nav · Primary navigation shared across pages.
src/components/Nav.tsx - Audio recorder · Browser recording boundary that initiates speech capture.
src/components/AudioRecorder.tsx - Passage display · Shows the target passage during practice.
src/components/PassageDisplay.tsx - Passage map · Visualizes alignment or reading flow against the passage.
src/components/PassageMap.tsx - Metrics dashboard · Displays scoring metrics and reader skill indicators.
src/components/MetricsDashboard.tsx - Diagnostic report · Presents the narrative assessment output.
src/components/DiagnosticReport.tsx
Server API
Next.js route handlers that orchestrate authentication, scoring, storage, and generation.
- Token API · Issues short-lived Deepgram credentials to the browser.
src/app/api/deepgram-token/route.ts - Diagnose API · Creates the plain-language assessment report from structured results.
src/app/api/diagnose/route.ts - Passage API · Selects or creates the next practice passage.
src/app/api/generate-passage/route.ts - Session API · Persists and retrieves session history and progress.
src/app/api/session/route.ts
Core Logic
Deterministic alignment, metrics, recommendation, and shared types.
- Alignment · Deterministic passage-to-transcript alignment using timestamps and edit distance.
src/lib/alignment.ts - Metrics · Derives WCPM, errors, pauses, and self-corrections from alignment.
src/lib/metrics.ts - Passage vectors · Represents passage library features for similarity search.
src/lib/passageVectors.ts - Session vector · Builds the reader skill profile used for future recommendations.
src/lib/sessionVector.ts - Passage generator · Fallback passage creation and recommendation support.
src/lib/generatePassage.ts - Types · Shared domain types for transcripts, metrics, and sessions.
src/lib/types.ts
Infrastructure
Redis-backed persistence and external service adapters.
- Redis stack · Persistence and vector search boundary for sessions and passage matching.
src/lib/redis.ts - Deepgram · External transcription service with word-level timestamps.
src/lib/deepgram.ts
Connections 29
- Landing page → Nav: uses · Landing page includes shared navigation.
- Practice flow → Nav: uses · Practice experience uses shared navigation.
- Practice flow → Audio recorder: orchestrates · Main practice flow starts recording from the browser.
- Practice flow → Passage display: shows · Displays the current target passage.
- Practice flow → Passage map: renders · Shows reading alignment feedback.
- Practice flow → Metrics dashboard: renders · Surfaces computed performance metrics.
- Practice flow → Diagnostic report: renders · Shows the assessment narrative.
- Progress view → Metrics dashboard: reuses · Progress view reuses longitudinal metrics presentation.
- Audio recorder → Token API: auth bootstrap · Browser fetches a short-lived speech token before capture.
- Audio recorder → Deepgram: streams · Uses the speech service for transcription and timestamps.
- Token API → Deepgram: authenticates · Server-side token flow protects Deepgram credentials.
- Diagnose API → Alignment: consumes · Narrative generation is based on deterministic alignment outputs.
- Diagnose API → Metrics: consumes · Uses scoring metrics as structured input.
- Diagnose API → Redis stack: reads history · Pulls session context for report generation.
- Diagnose API → Diagnostic report: feeds · Produces the report data shown in the UI.
- Passage API → Session vector: profiles reader · Uses the reader skill profile to choose the next passage.
- Passage API → Passage vectors: searches · Matches against the passage library vector space.
- Passage API → Passage generator: falls back to · Can create a new passage when no close match exists.
- Passage API → Redis stack: queries · Uses Redis Stack for vector similarity and stored passages.
- Session API → Redis stack: stores · Persists session history and longitudinal progress.
- Alignment → Types: uses · Alignment works over shared domain data structures.
- Metrics → Types: uses · Metrics are computed on shared transcript and alignment types.
- Session vector → Metrics: derives from · Reader skill profile is built from prior performance metrics.
- Session vector → Redis stack: reads · Loads historical session data for the reader model.
- Passage vectors → Redis stack: indexes · Passage vectors are stored and queried in Redis Stack.
- Passage generator → Types: uses · Passage generation works with shared domain types.
- Diagnose API → Metrics: narrates · Structured metrics are converted into plain-language feedback.
- Passage API → Session API: informs · Recommendation results feed the persisted session record.
- Practice flow → Session API: syncs · Practice flow saves and restores session state.