How pranavtechie/modelith-cli works
Project Type and Purpose
• Modelith is an open-source, CLI-based tool designed to help professors, TAs, and teachers compare coding assignments – primarily by analyzing ASTs of submission files (currently supporting Jupyter notebooks) to identify similarities and potential plagiarism.
• In addition to its CLI functionality, the project incorporates a web client (frontend) for filtering submissions, visualizing the similarity matrix, and possibly extended features like trend analysis and class management.
• It also includes some automation components (e.g., Kaggle automation) indicating that the tool can interface with external platforms for additional data extraction or login functionalities.
File Structure and Key Components
• Top-level directories and files:
• The “src” directory encapsulates the main source code within a “modelith” package.
• “src/modelith/cli” – Contains the command-line interface and its commands (extract, init, kaggle_dump). This is the entry point when using the tool via the command line.
• “src/modelith/core” – Houses the core logic such as AST comparison, database operations (using sqlite), folder hashing, sanity checks, and notebook analysis. This is essentially the “engine” that performs the assignment comparisons.
• “src/modelith/frontend” – Contains the web client part of the project, built with modern web technologies (a React-based single page application using Vite, TypeScript, and a package manager lock file such as bun.lock). This component likely provides additional interactive features like threshold-based filtering and visual reports.
• “src/modelith/kaggle_automation” – Contains scripts aimed at automating aspects of Kaggle interaction, e.g., logging in.
• Other supportive files include configuration files (Makefile, pyproject.toml), test scripts in the “tests” directory, CI/CD configurations under “.github/workflows”, and development tooling under “devtools”.
Defining the Main Architectural Components
a. Components:
• CLI Interface: Represented by the “cli” directory with its commands (extract, init, kaggle_dump). This component is the user’s gateway to invoking functionality.
• Core Engine: Located in the “core” directory, it includes AST comparison, database operations, and various helper utilities (e.g., folder hashing and sanity checks).
• Database Module: A part of the core logic that uses SQLite for storing submissions and metadata.
• Frontend Web Client: A modern web application residing in “frontend” that interfaces with the core’s data and provides visualization (similarity matrix, filtering mechanisms, and likely dashboards).
• Kaggle Automation Module: A specialized component that handles external service interactions (Kaggle-specific operations).
• Testing and CI/CD: Represented by “tests” for unit tests and the “.github/workflows” for continuous integration and deployment pipelines.
b. Relationships and Interactions:
• The CLI tool (via its commands) calls into the core engine functions, triggering processes like AST generation, analysis, and database updates.
• The core engine communicates with the SQLite database to store, retrieve, and compare submission data.
• The frontend module reads from (or interfaces with) the database or core APIs (if there is an API layer in place) to display analysis results and similarity matrices.
• The Kaggle automation module might interact with external APIs to pull or dump data that will then flow into the core’s processing pipeline.
• Continuous integration (CI) workflows ensure that changes to any of these components are tested and deployed correctly.
Architectural Patterns or Design Principles
• Separation of concerns: The project clearly delineates functionality into CLI, core processing, and web presentation layers.
• Modular design: The use of separate directories for distinct responsibilities (e.g., separate “cli”, “core”, and “frontend” modules) supports maintainability and scalability.
• Extensibility: With dedicated directories for future features (e.g., class management, multi-language support), the architecture shows plans for modular expansion.
• Integration with external package management and deployment tooling (such as uv for Python environments and GitHub Actions) helps keep the tool lightweight and easily installable.
Technologies and Frameworks
• Python for the CLI and core logic.
• SQLite as the lightweight database solution.
• uv (the new package manager for Python) is used to manage the Python environment and dependencies.
• React/TypeScript and Vite for the web frontend development.
• Build/configuration tools including Makefile, pyproject.toml, and bun (for frontend package locking).