How arvindang/illustrative works
1. What this repo is
- A Python 3.12, batch-first pipeline that turns public-domain text into a graphic novel.
- The main runtime dependency is Google Gemini / Vertex AI for long-context analysis and image generation.
- The system is designed as a multi-stage content factory, not a generic web app.
2. Primary entrypoints
- app.py runs the interactive Streamlit UI for test/ops workflows.
- production_run.py runs the full end-to-end book processing path.
- The UI and CLI both drive the same underlying pipeline, but with different operational modes.
3. User-facing surface
- ui/ contains the Streamlit pages: auth, home, dashboard, generate, and novel detail.
- ui/session.py and ui/db_operations.py suggest the UI is stateful and backed by persisted job/user records.
- auth_utils.py and ui/pages/auth.py form the access boundary before a user can launch or inspect jobs.
4. Core pipeline orchestration
- ui/pipeline.py appears to coordinate the main workflow from uploaded source text to finished output.
- The pipeline is split into three major functional stages: scripting, illustration, and composition/export.
- Intermediate artifacts are file-based and tracked in the repo’s asset/output directories.
5. Scripting subsystem
- agents/scripting/ is a multi-pass adaptation engine that ingests an entire novel and produces structured story assets.
- Passes include beat analysis, adaptation filtering, pacing, character deep dives, dialogue polish, asset manifesting, and final scriptwriting.
- Output artifacts include JSON files like full script, assets, beats, and character arcs; this stage is the source of narrative structure for later visual generation.
6. Reference and panel generation
- The illustration side is split into reference creation and panel generation.
- agents/reference_agent.py and agents/panel_agent.py, plus the provider-specific equivalents under agents/openai/, encapsulate model calls for character/object/location references and page/panel images.
- The agents/openai/ layer contains API client, batch manager, and file manager code, indicating async/batch-oriented model interaction and artifact transfer.
7. Layout, composition, and export
- agents/layout_agent.py, agents/compositor_agent.py, and agents/export_agent.py handle page assembly after images are generated.
- The compositor stage places panels on pages, overlays captions and speech bubbles, and prepares final page images.
- Export produces reader-facing deliverables such as PDF and EPUB.
8. Quality gates and validation
- validators/ is a distinct boundary around correctness and consistency.
- Pre-validators check inputs and structure before generation; post-validators check composition and narrative consistency afterward.
- This is important because the pipeline depends on preserving character identity, visual continuity, and page-level formatting across many generated assets.
9. Persistence and domain model
- models/ and migrations/ indicate a relational database is used for users, novels, progress, errors, and cost tracking.
- models/novel.py and models/user.py define the core persisted entities.
- Alembic migrations show the schema evolved to support progress reporting and cost accounting, which are operationally important for long-running generation jobs.
10. Artifact storage
- assets/input/ is the source drop zone for uploaded text.
- assets/output/ is the working and final artifact area for generated characters, pages, and composed outputs.
- storage/bucket.py suggests an abstraction for cloud/object storage, likely used when moving beyond local filesystem-only runs.
11. Operational and deployment boundaries
- The presence of Procfile, railway.toml, and nixpacks.toml shows the app is meant to deploy in hosted PaaS environments.
- .github/workflows/pages.yml and the site/ directory indicate a separate static documentation/demo site, including sample PDF viewing assets.
- production_run.py is the boundary between demo UI and full batch production use.
12. Testing and reliability
- Tests cover smoke, integration, isolation, scripting-only, references, cost calculation, and full pipeline runs.
- This mirrors the architecture: each stage can be validated independently, while tests/test_full_pipeline.py exercises the integrated workflow.
- scripts/ contains environment and syntax checks, reinforcing that the system is operationally sensitive to API credentials, model availability, and pipeline configuration.
13. Mental model for an architecture diagram
- Draw it as a left-to-right content factory: input text and user intent enter on the left, scripted narrative assets are created in the middle, image/reference generation happens next, and page composition/export happens on the right.
- Put the database and storage beneath the pipeline as shared infrastructure.
- Put Gemini/Vertex AI outside the system boundary as the external model service the agents depend on.
Main entry points: agent.py Script Agent · export_agent.py Exporter · pipeline.py UI Pipeline · panel_agent.py Panel Agent
Components 21
UI & Ops
Interactive Streamlit surface and job launching boundaries.
- Streamlit App · Interactive UI entrypoint for test and operations workflows.
app.py - UI Pipeline · Coordinates the main workflow from uploaded text to generated outputs.
ui/pipeline.py - Auth Page · Access boundary before users can inspect or launch jobs.
ui/pages/auth.py - Dashboard · Operational overview for job status and progress.
ui/pages/dashboard.py - Generate Page · User-facing launch point for generation requests.
ui/pages/generate.py
Pipeline
Core end-to-end generation flow from source text to exported book assets.
- Prod Run · Batch-first end-to-end production processing entrypoint.
production_run.py
Scripting
Novel-to-script adaptation passes that create the narrative backbone.
- Script Agent · Multi-pass adaptation engine that converts a novel into structured story assets.
agents/scripting/agent.py - Beat Analysis · Extracts narrative beats and story structure.
agents/scripting/passes/beat_analysis.py - Adaptation Filter · Filters and reshapes source material for graphic-novel adaptation.
agents/scripting/passes/adaptation_filter.py - Character Dive · Builds character behavior, identity, and continuity notes.
agents/scripting/passes/character_deep_dive.py - Scriptwriter · Produces the final structured comic script and scene assets.
agents/scripting/passes/scriptwriter.py
Illustration
Reference and panel image generation stages.
- Reference Agent · Creates character, object, and location references for visual continuity.
agents/reference_agent.py - Panel Agent · Generates page and panel imagery from scripted scenes.
agents/panel_agent.py
Assembly & Export
Page composition, validation, persistence, and delivery artifacts.
- Layout Agent · Decides page and panel placement before composition.
agents/layout_agent.py - Compositor · Assembles pages, captions, and speech bubbles into final page images.
agents/compositor_agent.py - Exporter · Produces reader-facing deliverables such as PDF and EPUB.
agents/export_agent.py - Pre-Validators · Checks inputs and structure before generation begins.
validators/pre_validators.py - Post-Validators · Checks composition and narrative consistency after generation.
validators/post_validators.py
Infra & External
Shared persistence, storage, and external model dependencies.
- Domain Models · Persisted users, novels, progress, errors, and cost tracking records.
models/novel.py - Bucket Storage · Abstraction for moving artifacts between local and cloud/object storage.
storage/bucket.py - Gemini / Vertex AI · External long-context and image generation service used by the agents.
Connections 24
- Streamlit App → UI Pipeline: launches · The Streamlit app enters the shared pipeline through the UI orchestration layer.
- Prod Run → Script Agent: starts batch flow · The production entrypoint drives the end-to-end batch processing path.
- Auth Page → UI Pipeline: gates access · Authentication must succeed before jobs can be launched or inspected.
- Dashboard → Domain Models: reads status · Dashboard surfaces persisted jobs, progress, and errors.
- Generate Page → UI Pipeline: submits job · Generation requests hand off to the pipeline coordinator.
- UI Pipeline → Pre-Validators: prechecks · Pipeline validates inputs and structure before heavy generation work.
- Pre-Validators → Script Agent: allows · Validated inputs proceed into the scripting stage.
- Script Agent → Beat Analysis: runs · The scripting orchestrator executes the narrative analysis passes.
- Script Agent → Adaptation Filter: runs · Source material is filtered for adaptation suitability.
- Script Agent → Character Dive: runs · Characters are expanded for continuity and visual consistency.
- Script Agent → Scriptwriter: produces · Final structured script and story assets are emitted here.
- Script Agent → Reference Agent: hands off assets · Script-derived structure informs reference generation.
- Reference Agent → Gemini / Vertex AI: model calls · Reference creation depends on external Gemini / Vertex AI services.
- Panel Agent → Gemini / Vertex AI: image calls · Panel generation uses the external model service for visual outputs.
- Reference Agent → Bucket Storage: stores refs · Generated reference artifacts are persisted for reuse.
- Panel Agent → Bucket Storage: stores panels · Page and panel images are written to artifact storage.
- Script Agent → Domain Models: tracks progress · Long-running jobs persist progress and operational state.
- Panel Agent → Layout Agent: feeds pages · Generated panels move into page layout planning.
- Layout Agent → Compositor: defines layout · Layout decisions drive final page assembly.
- Compositor → Post-Validators: checks output · Composed pages are validated for consistency and formatting.
- Post-Validators → Exporter: approves export · Validated output proceeds to deliverable packaging.
- Compositor → Exporter: exports pages · Final page images are packaged for reader-facing formats.
- Exporter → Bucket Storage: publishes artifacts · Exports are stored as final deliverables and working assets.
- Exporter → Domain Models: records cost · Operational metadata such as cost and completion state is persisted.