How chrisdonahue/sheetsage works
Main Components
• Command-line Interface & Script Launcher: The main shell scripts (sheetsage.sh, prepare.sh, sheetsage.sh) act as the entry points for setup and transcription – they handle user input (e.g., a YouTube URL or local file path) and kick-off processing.
• Core Transcription Engine (“sheetsage” Package): This component comprises the bulk of the processing code. Within the sheetsage directory, there are submodules handling multiple aspects:
• Alignment (align.py, align_test.py)
• Beat and tempo detection (beat_track.py, beat_track_test.py)
• Data management (data.py and data_test.py)
• Evaluation (eval.py, eval_test.py)
• Inference (infer.py, infer_test.py)
• Various modules for representations (both handcrafted and using Jukebox) and theoretical music processing (under theory/)
• Utility functions in utils.py
• Assets/Models and Data: Located under sheetsage/assets, this includes configuration files (like sheetsage.json), musical JSON data for HookTheory, and other auxiliary data required for both training and inference.
• Serving Backend: Under sheetsage/serve/backend, there is a backend main application intended to support server-side transcription or a hosted API.
• Docker Infrastructure: The docker and cloudrun directories provide Dockerfiles and scripts (e.g., run.sh, build.sh, shell.sh) enabling containerized development, testing, and deployment.
• Notebooks: The project includes notebooks (e.g., Inference.ipynb and Dataset.ipynb) that demonstrate usage for both dataset exploration and transcription with interactive environments.
• External Dependencies & Licensing: The README refers to other components like madmom, OpenAI Jukebox, and Melisma, which are integral to transcription quality improvements and beat/downbeat detection. Licensing considerations also mark this as a tool with multiple integration points.
Relationships & Data Flow Between Components
• The user starts by running a shell script (e.g., sheetsage.sh) which determines the source of input (YouTube URL or local file).
• This launcher delegates to the core transcription engine where the audio processing happens:
• First, preprocessing (beat detection and possibly downbeat correction) is conducted.
• Then, the transcription modules (inference module, alignment, and theory-based modules) convert the audio features into a lead sheet (chords & melody) and produce intermediary representations (like MIDI files and LilyPond files for rendering).
• The output is then saved to an “output” directory.
• For development and testing, Docker scripts allow this entire pipeline to run inside containers; similarly, Jupyter notebooks help in interactive debugging and exploration.
• For service or API use, a dedicated backend (in sheetsage/serve/backend) could be used – for example, to accept requests and return transcription results.
Architectural Patterns & Design Principles
• Layered Architecture: The design naturally separates user interfacing (shell scripts, command-line), core audio and music-driven processing (transcription engine), and external serving components.
• Modular Design: Each logical part (alignment, beat tracking, representation, data handling) is broken up into its own module and tested individually (as seen with “_test.py” files).
• Containerization: Docker and Cloud Run directories show design for consistent build environments and service deployment.
• Extensibility: Support for integration with both “handcrafted” and “jukebox” representations implies an architecture designed to plug in additional models or processing techniques based on the configuration.
Final Recommendations for the Principal Software Engineer
• Use annotations to note which directories or files map onto which component (e.g., “sheetsage/theory” for music theory processing, “docker/” for container scripts).