How mishok06333/papermerge-core works
1. Overall shape
- This repo is a full-stack document management system with two major runtimes: a Python backend in papermerge/ and a TypeScript/Vite frontend monorepo in frontend/.
- The backend exposes an OpenAPI REST API and also includes CLI commands and background task support.
- The frontend is split into multiple apps and shared packages, with the main product UI in frontend/apps/ui.
2. Backend application boundary
- papermerge/app.py is the main server entry point, with papermerge/celery_app.py for async/background work.
- The backend is organized around papermerge/core/, especially papermerge/core/features/*, where each business area owns its router, schema, DB access, tests, and sometimes CLI hooks.
- This makes the backend modular: routes are feature-owned rather than centralized in one large service layer.
3. Core domain and persistence
- The main persistent domain is built around users, groups, roles, nodes, documents, document versions, pages, tags, custom fields, document types, shared nodes, preferences, audit logs, API tokens, and search metadata.
- SQLAlchemy-style DB layers appear under papermerge/core/features/*/db, with Alembic migrations in papermerge/core/alembic/.
- papermerge/core/db/ and papermerge/core/dbapi.py look like shared persistence infrastructure used across feature modules.
4. Document and node model
- The central business object is the “node”: folders and documents are both modeled in the node tree, with special folders like home/inbox and support for ownership and sharing.
- Document-specific behavior is layered on top: versions, pages, thumbnails, OCR processing state, page operations, and document-level actions.
- The node/document layer is the main source of UI interactions such as folder navigation, preview display, page management, and drag-and-drop moves.
5. Search and indexing path
- Full-text search is a separate feature area under papermerge/core/features/search, with its own DB SQL, router, schema, and tests.
- The README states OCR text is extracted and indexed for search; backend migrations also include search index functions and triggers.
- Search therefore depends on document ingestion and OCR output, and is not just a client-side filter.
6. OCR and background processing
- OCR and page processing are not purely synchronous request/response operations; the presence of Celery and task-oriented modules indicates asynchronous jobs.
- Document upload and page manipulation likely trigger deferred work such as preview generation, OCR, and search index updates.
- This creates a pipeline boundary: HTTP request creates or mutates documents, then background workers finish processing.
7. Storage layer
- papermerge/storage/ abstracts file storage backends, including local storage, CloudFront-related signing, and R2 support.
- The backend also has CLI support for storage URL signing.
- File/blob storage is therefore external to the relational DB, and the app distinguishes metadata persistence from binary document storage.
8. Authentication, authorization, and tenancy
- Auth is a dedicated backend feature with support for remote/SSO-style schemes and scopes.
- The repo supports users, groups, roles, ownership, and shared-node access rules, which means authorization is not a simple global admin/non-admin toggle.
- The UI and backend both reflect this model, with routes and components for managing access, sharing, and role assignments.
9. Frontend architecture
- frontend/apps/ui is the main user-facing application and is built as a large modular SPA.
- It uses a central store, router, hooks, i18n assets, theme support, and many feature folders aligned with backend domains.
- Other frontend apps exist for narrower surfaces: commander.dev, viewer.dev, hooks.dev, and kommon.dev.
10. Frontend feature alignment with backend
- The frontend mirrors backend domains almost one-for-one: documents, nodes, shared nodes, users, groups, roles, tags, custom fields, document types, search, audit, preferences, API tokens, notifications, and file upload.
- Each feature folder typically contains components, pages, storage/API code, and hooks, which suggests a consistent pattern: UI presentation sits on top of feature-specific data-fetching/state logic.
- Shared UI primitives live in frontend/packages/kommon, while a reusable viewer component set lives in frontend/packages/viewer.
11. Document-centric UI flows
- The ui app contains the main document browser, folder views, inbox/home views, document viewer, thumbnails, page list, OCR dialog, download actions, and drag/drop page manipulation.
- This frontend is optimized around “desktop-like” workflows: dual panels, split panes, context menus, and bulk/page-level operations.
- Those interactions map directly to backend document/node/page routes and to asynchronous processing states.
12. Search and metadata editing in the UI
- Search is a rich feature with many filter types: full-text, categories, tags, owners, dates, numeric/custom fields, users, and status-related filters.
- Metadata editing is first-class for document types and custom fields, and document detail views can edit custom fields with autosave.
- This means the UI is not just browsing files; it is a metadata-heavy administration and classification tool.
13. Deployment and runtime variants
- Docker is a major deployment boundary: there are standard and OIDC-specific container setups under docker/standard and docker/oidc.
- The OIDC variant integrates with Keycloak, nginx, supervisord, DB init, and realm export configuration.
- The standard setup is a simpler self-contained deployment, and the README explicitly shows a minimal one-container run for quick starts.
14. Data flow summary
- User action in the browser goes to the frontend SPA.
- The SPA calls the backend REST API.
- The backend reads/writes PostgreSQL-backed metadata and uses blob storage for file content.
- Document ingestion and page changes can enqueue async work for OCR, thumbnail generation, previews, and search indexing.
- Search queries then hit the backend search feature, which depends on indexed metadata/text rather than live file inspection.
15. Operational/tooling boundaries
- Database schema evolution is managed by Alembic; quality gates are covered by Python tests and frontend tests in GitHub Actions.
- The frontend uses Vite, Yarn workspaces, TypeScript, and feature-specific test setup.
- The backend exposes CLI commands for operational tasks like user, permission, token, and search management, which indicates some admin workflows can be run outside the HTTP API.
Main entry points: router_loader.py Feature router · features UI features · document Documents · app.py API entry
Components 25
Backend
Python API, domain features, storage, and async processing.
- API entry · Main backend server entry point and API bootstrap.
papermerge/app.py - Worker app · Background task runtime for deferred document processing.
papermerge/celery_app.py - Feature router · Loads and composes feature-owned routers into the API.
papermerge/core/router_loader.py - Auth · Authentication, remote schemes, scopes, and access dependencies.
papermerge/core/features/auth - Nodes · Folder/document node model, thumbnails, and node-level routes.
papermerge/core/features/nodes - Documents · Document versions, pages, upload flow, and document actions.
papermerge/core/features/document - Search · Full-text and metadata search over indexed document data.
papermerge/core/features/search - Sharing · Shared-node permissions and access updates for users and groups.
papermerge/core/features/shared_nodes - Users · User management, profiles, and admin routes.
papermerge/core/features/users - Groups · Group management and membership-related access control.
papermerge/core/features/groups - Roles · Role definitions and assignment workflows.
papermerge/core/features/roles - Tags · Tag model and tag management routes.
papermerge/core/features/tags - Custom fields · Typed custom fields and document metadata definitions.
papermerge/core/features/custom_fields - Doc types · Document type definitions and metadata templates.
papermerge/core/features/document_types - Preferences · User and system preference storage and formatting settings.
papermerge/core/features/preferences - Audit · Audit log capture, queries, and contextual metadata.
papermerge/core/features/audit - API tokens · Personal token issuance and lifecycle management.
papermerge/core/features/api_tokens - DB infra · Shared database engine, ORM base, and DB APIs used across features.
papermerge/core/db - Migrations · Alembic migrations and SQL trigger scripts.
papermerge/core/alembic - Blob storage · Storage abstraction with local, CloudFront, and R2 backends.
papermerge/storage
Frontend
TypeScript monorepo with the main SPA and shared packages.
- UI app · Main user-facing frontend application with routing, store, and feature screens.
frontend/apps/ui - UI features · Feature-aligned UI slices for documents, nodes, search, sharing, and admin tools.
frontend/apps/ui/src/features - Shared UI · Reusable frontend packages for viewer widgets, common controls, and hooks.
frontend/packages - Dev apps · Smaller auxiliary frontend apps for viewer, commander, hooks, and shared component development.
frontend/apps
Ops
Docker deployment variants and runtime setup.
- Docker runtime · Standard and OIDC container setups, entrypoints, and compose files.
docker
Connections 48
- API entry → Feature router: assembles · Bootstraps the API from modular feature routers.
- API entry → Auth: uses · Applies authentication and authorization to requests.
- API entry → DB infra: connects · Uses shared DB infrastructure for persistence.
- API entry → Blob storage: stores blobs · Delegates binary file storage to the storage layer.
- API entry → Worker app: enqueues · Hands off deferred work to background workers.
- Feature router → Nodes: mounts · Includes node routes in the API surface.
- Feature router → Documents: mounts · Includes document routes and document-version routes.
- Feature router → Search: mounts · Includes search endpoints.
- Feature router → Sharing: mounts · Includes sharing and access routes.
- Feature router → Users: mounts · Includes user management routes.
- Feature router → Groups: mounts · Includes group management routes.
- Feature router → Roles: mounts · Includes role management routes.
- Feature router → Tags: mounts · Includes tag routes.
- Feature router → Custom fields: mounts · Includes custom-field routes.
- Feature router → Doc types: mounts · Includes document-type routes.
- Feature router → Preferences: mounts · Includes preference routes.
- Feature router → Audit: mounts · Includes audit routes.
- Feature router → API tokens: mounts · Includes API token routes.
- Search → Documents: indexes · Search depends on document text and document metadata.
- Documents → Nodes: extends · Documents are represented within the node tree.
- Documents → Blob storage: reads/writes · Uses blob storage for uploads and document content.
- Documents → Worker app: dispatches · Sends OCR, preview, and page-processing work to async jobs.
- Search → DB infra: queries · Search uses shared persistence and SQL-backed indexing.
- Nodes → DB infra: persists · Stores node tree and thumbnail metadata.
- Sharing → Users: checks access · Shares nodes with users and enforces permissions.
- Sharing → Groups: checks access · Shares nodes with groups and enforces permissions.
- Sharing → Roles: uses roles · Permission rules include role-based access.
- Users → Groups: membership · Users are organized into groups.
- Roles → Auth: authorizes · Roles feed authorization scopes and access decisions.
- Custom fields → Doc types: defines · Document types reference custom-field definitions.
- Audit → DB infra: records · Audit events are persisted centrally.
- Preferences → DB infra: persists · Preferences are stored in the shared DB layer.
- API tokens → Auth: protects · Token issuance is part of authenticated API access.
- UI app → UI features: composes · Main SPA is built from feature-aligned screens and stores.
- UI app → Shared UI: reuses · Uses shared component packages and common viewer widgets.
- UI app → Docker runtime: deploys via · Frontend is packaged into the containerized runtime.
- UI features → Documents: calls API · Document browser and viewer talk to backend document routes.
- UI features → Nodes: calls API · Navigation and folder views depend on node routes.
- UI features → Search: calls API · Search UI queries backend search endpoints.
- UI features → Sharing: calls API · Sharing dialogs manage access through backend sharing routes.
- UI features → Users: calls API · Admin screens manage users.
- UI features → Groups: calls API · Admin screens manage groups.
- UI features → Roles: calls API · Admin screens manage roles.
- UI features → Tags: calls API · Tag management and tag filters use backend tag routes.
- UI features → Custom fields: calls API · Metadata editors depend on custom-field routes.
- UI features → Doc types: calls API · Document classification screens depend on doc-type routes.
- UI features → Preferences: calls API · UI settings and user preferences are backed by preference routes.
- UI features → Audit: calls API · Audit log screens consume audit routes.