How abhaytr/muj-camp works
Project type and purpose
• MUJ CAMP is a full‑stack alumni management portal with two main subsystems:
• A LinkedIn data‐syncing engine (“ATLIS”) that scrapes and stores alumni profile data into MongoDB.
• A React‑based web portal plus a Node.js/TypeScript WebSocket/REST backend that provides authentication, real‑time updates, and AI‑driven dashboards.
File structure
• Top‑level directories:
• ATLIS/ATLIS_Desktop – Python desktop UI and orchestration scripts (app.py, linkedin_instance.py, data.js)
• ATLIS/ATLIS_Engine – Core scraping engine, queue management, DB adapter (atlis_db.py), logging, LinkedIn API wrapper
• muj_camp – React front‑end (src/, public/, package.json) using Redux‑style app_state
• muj_camp_server – Node.js/TypeScript backend, with WebSocket server (CampWSServer.ts), auth modules, data managers, AI managers, utils, TypeScript config
README for insights
• Purpose: sync 15,000+ alumni LinkedIn data in days via WebSockets‐backed scraping engine.
• Features: AI dashboard for ad‑hoc MongoDB queries powered by Llama.
a. Main components
• Desktop Scraper UI (ATLIS_Desktop)
• Scraper Engine (ATLIS_Engine)
• Data Store (MongoDB)
• Backend API & WebSocket Server (muj_camp_server)
• Front‑end Web Client (muj_camp)
• External Services: LinkedIn, Llama/AI model endpoints
b. Relationships & interactions
1. Operator triggers scraping in Desktop UI → invokes Engine modules → fetch data from LinkedIn (via linkedin_instance)
2. Engine writes/updates records in MongoDB (atlis_db adapter)
3. Backend server (CampWSServer + REST endpoints) listens for WebSocket events & incoming web requests
4. Server reads from MongoDB, processes data (aiManager, dataManager), and pushes real‑time updates via WS to clients
5. Front‑end connects via WS + HTTP, dispatches Redux actions, renders dashboard and AI‑query results
6. For AI insights, backend queries MongoDB and/or invokes Llama, returns results to front‑end
c. Architectural patterns & design principles
• Separation of concerns: scraping engine separated from web server
• Event‑driven & queue‐based scraping (atlisq_manager)
• Client‑server with real‑time channel (WebSockets) plus REST
• Modular code organization (auth, data, AI managers in server; Redux slices in front‑end)
• Reusable core engine library (ATLIS_Engine) vs. thin orchestration layer (ATLIS_Desktop)
d. Key technologies & frameworks
• Python 3, requests/selenium (assumed) for ATLIS_Engine
• MongoDB for alumni data store
• Node.js + TypeScript + ws or socket.io for WebSocket server
• Express‑style REST API (in muj_camp_server/src/utils)
• React.js + Redux (app_state) + WebSocket client in front‑end
• Llama (or OpenAI) for AI‑driven dashboard
Final recommendations
• Every module boundary (Desktop UI vs. Engine vs. Server vs. Client).
• Data flows (scrape→store, store→AI query, AI→UI).
• Auth flow separately (login → JWT manager → protected WS/REST).
• Incorporate at least one example of a sequence (e.g., admin triggers scrape → desktop updates UI via WebSockets).