How opennccu/nccucourse-crawler-by-andyjjrt works
Project type and purpose
• This is a Python-based CLI tool (“NCCUCourse”) rather than a full-stack web app.
• Its main purpose is:
• Authenticate to the NCCU iNCCU student portal
• Fetch course results, descriptions, rating data
• Translate rating text via Google Cloud
• Persist the cleaned and translated data locally (CSV/old JSON) or into a database
File structure
• Top-level scripts and modules:
• main.py – orchestrates the workflow
• fetchResult.py, fetchDescription.py, fetchRate.py – individual data-fetch steps
• translateRate.py – passes raw rating text to Google Translation API
• migrate.py – ingests old_data JSON into current storage format
• DB.py – encapsulates database / persistence logic
• User.py, util.py, constant.py – domain model and shared utilities
• Data directories:
• data/ – holds output CSV files for each semester
• old_data/ – legacy JSON dumps
• Configuration:
• .env – user credentials, YEAR/SEM, GOOGLE_APPLICATION_CREDENTIALS, OPENSSL_CONF
• openssl.conf – SSL settings for HTTPS
README for additional insights
• Specifies Python ≥3.9, uses pip requirements.
• Entry point python main.py (with possible OPENSSL_CONF=… shim).
• No built-in UI—pure CLI pipelines with HTTP, translation, and storage steps.