How it-gssb/sycamore works
Project type and purpose
• This is a command-line Python ETL tool whose primary purpose is to extract student roster data from the Sycamore API, cache it locally, transform it into various target formats (SDS 2.1, Clever import, etc.), and output CSVs. It also ships a set of PowerShell scripts for account-management tasks against an Active Directory (or similar) after the CSVs are generated.
File structure
• src/lib: low-level modules (SycamoreRest, caching, entity models, generators) that encapsulate the HTTP client, local disk cache, and in-memory representations of Sycamore data.
• src/extract: pipeline stages that read from the cache or API, validate and filter, project CSV file schemas, and merge multiple CSV fragments.
• src/sds: “consumer” scripts that orchestrate the end-to-end flow for SDS 2.1 and SDS-to-Clever data generation. These scripts invoke extract modules and write “outv21” or “outClever” directories.
• src/clever: transformer module for Clever-specific data shaping.
• src/powershell: standalone PS1 scripts for creating/deleting groups, setting passwords, toggling account enabled status, using the CSVs to drive AD.
Key insights from the README
• The workflow is: CLI invocation → check/cache Sycamore data → transformation pipelines → write CSV output.
• External dependencies: Sycamore API (requires access token), Python libs (pandas, phonenumbers).
• Two major “consumers”: SDS 2.1 importer and Clever importer.
Capturing essential details
• That both SDS and Clever exporters depend on the same upstream extract modules.