How luxkatana/goback works
Project type and purpose
• goback is a full-stack web application (“a pocket-sized version of wayback”) that lets users capture, store, and browse archived snapshots of web pages.
• The backend is written in Python (webserver.py using a minimal web framework), and the frontend is a React + TypeScript single-page application built with Vite.
• It relies on external services: an Appwrite storage bucket for binary/html storage, and a MySQL database for metadata.
File structure
Top-level (backend)
• webserver.py – HTTP API endpoints for user auth, job management, snapshot retrieval
• scraper.py – fetches and processes target web pages, saves artifacts
• models.py – ORM definitions mapping jobs, snapshots, users to MySQL
• appwrite_session.py – wrapper to authenticate & talk to Appwrite storage
• config_manager.py – loads .env, goback.toml settings
• Dockerfile & compose.yaml – container definitions for backend, MySQL; orchestrates services
• test_html_documents/ – sample documents for scraper unit tests
• requirements.txt – Python dependencies
goback-frontend (frontend)
• src/routes – page-level React components (Login, Dashboard, CreateBackupPage, JobViewer, etc.)
• src/components & src/components/ui – UI building blocks (Navbar, ProtectedComponent, theming, toasters)
• src/utils/AuthContext.tsx – React context for auth state
• tsconfig*, package.json, vite.config.ts – TS/vite build config
README for insights
• States prerequisites: Python3, Appwrite API key + bucket, MySQL server, Docker Compose
• Installation uses Docker Compose to wire up the backend and MySQL; Appwrite is assumed as a managed/external service
Main components & interactions
a. Components
– Client (Browser) running React SPA
– goback-frontend container (serving the built SPA or dev server)
– Backend API container (Python webserver + scraper)
– MySQL database container (metadata store)
– Appwrite service (external storage bucket)
b. Data flows & relationships
1. User → Browser → goback-frontend UI
2. Frontend sends HTTPS requests to Backend API (login, signup, list jobs, create job)
3. Backend API reads/writes metadata in MySQL (users, jobs, snapshot records)
4. When a user triggers “Create Backup,” Backend invokes scraper.py
• scraper fetches the target URL, downloads assets (HTML, JS, CSS, media)
• scraper pushes each artifact to Appwrite via appwrite_session API
• scraper updates the job’s status & records in MySQL
5. For “View Job,” Frontend asks Backend for snapshot metadata & presigned URLs → Backend queries MySQL & generates Appwrite URLs → Frontend displays archived content
c. Architectural patterns & design principles
– Separation of concerns: frontend SPA vs. backend API vs. data stores
– Single-page app with protected routes (ProtectedComponent wrapper)
– Background task pattern: synchronous or queued scraping sub-process within the backend
– Configuration driven by environment (.env) + goback.toml + config_manager
– Containerized micro-services (via Docker Compose) on an isolated network
d. Key technologies & frameworks
– React + TypeScript + Vite for the UI
– Python 3 + minimal web framework (Flask/FastAPI) for the API
– SQLAlchemy or equivalent ORM (models.py) for MySQL
– Appwrite REST SDK (appwrite_session.py) for object storage
– Docker & Docker Compose for local/production orchestration