How sugarlabs/musicblocks works
1. Overall shape
- Music Blocks is a browser-based visual programming environment for music and graphics.
- The core app is a single-page frontend centered on index.html / index.js, with most behavior implemented in vanilla JavaScript under js/.
- It also ships as a PWA/mobile package and has an Electron entry point (electron-main.js) plus Android packaging metadata (config.xml, manifest.*).
2. Primary runtime boundary
- The main execution boundary is the browser: UI rendering, event handling, audio synthesis, and persistence all happen client-side.
- A local web server is recommended for asset loading and full functionality; the repo includes dev tooling for serving, testing, linting, and packaging.
3. Core interaction model
- Users compose music/graphics programs by dragging blocks in the workspace.
- Blocks are organized into palettes and translated into executable behavior by the runtime.
- Execution drives “turtles” that produce sound, visuals, and stateful actions.
4. Block system
- js/blocks/ contains the domain-specific block families, grouped by concept: pitch, rhythm, drums, tone, graphics, pen, intervals, meter, sensors, flow, numbers, booleans, dictionaries, widgets, etc.
- Shared block infrastructure lives in js/block.js, js/blocks.js, js/blockfactory.js, and js/protoblocks.js.
- These modules define block creation, palette registration, dependencies, and serialization-friendly representations.
5. Execution and music engine
- js/logo.js, js/LogoDependencies.js, and related files implement the interpreter/execution layer that walks block structures and schedules actions.
- The turtle runtime (js/turtle.js, js/turtles.js, js/turtleactions/, js/turtle-singer.js, js/turtle-painter.js) is the bridge from blocks to output.
- This layer coordinates musical notes, timing, articulation, and drawing on a per-turtle basis.
6. Audio, notation, and export
- Audio and musical output are handled through utility modules such as js/midi.js, js/mxml.js, js/lilypond.js, js/notation.js, and sound adapters in js/p5-sound-adapter.js.
- sounds/ contains sampled instruments, and lib/midi.js and lib/voices/ support playback/speech features.
- There is a separate JS-to-block export path in js/js-export/, which converts JavaScript representations back into block structures via AST and conversion utilities.
7. UI shell and controls
- The top-level application UI is assembled from modules like js/toolbar.js, js/palette.js, js/pastebox.js, js/trash.js, js/themebox.js, js/languagebox.js, and js/piemenus.js.
- The workspace is highly interactive and uses mouse/keyboard events plus custom dialogs and overlays from js/utils/ and js/widgets/.
- Styling is split across css/, dist/css/, fonts/, images/, and lib/ assets.
8. Widgets subsystem
- js/widgets/ holds secondary interactive tools such as the keyboard, tempo, tuner, oscillator scope, rhythm ruler, sampler, mode/temperament tools, AI debug widget, and reflection/help components.
- Widgets are not the main programming surface; they are auxiliary panels that inspect, edit, or generate musical material.
- Several widgets have their own sub-APIs and tests, indicating they are modular UI subsystems rather than simple dialogs.
9. Planet sharing and project storage
- The planet/ directory is a distinct subsystem for project browsing, publishing, viewing, caching, and server/local synchronization.
- It includes local storage and remote request abstractions (ProjectStorage, RequestManager, ServerInterface, CacheManager, Publisher, SaveInterface) and a separate UI shell (planet/index.html, planet/js/main.js).
- This subsystem is the architecture’s boundary to project sharing and remix workflows; it is not required for core offline use.
10. Localization and content pipeline
- Translations are managed through locales/ JSON files and po/ source translations.
- convert_po_to_json.py and po-to-json-validation automation indicate a build-time translation conversion pipeline.
- User-facing documentation is also heavily localized, with language-specific docs under Docs/ and translated guides/manuals.
11. Build, packaging, and deployment
- Tooling includes Node-based workflows (package.json, gulpfile.*, jest.config.js, eslint.config.mjs, cypress.config.js, lighthouserc.js) plus Docker support.
- The repo produces browser assets, distribution CSS, and packaged app targets; CI covers linting, unit tests, Cypress E2E, Lighthouse, and security scanning.
- Service worker and manifest files (sw.js, lib/sw.js, manifest.json, manifest.webapp) support offline/PWA behavior.
12. Test and quality boundaries
- Tests are extensive and mirror the main architecture: block logic, turtle execution, utilities, widgets, export pipeline, and Planet components all have unit coverage under js/**/__tests__ and planet/js/__tests__.
- End-to-end browser behavior is validated in cypress/e2e/.
- This test layout reflects the repo’s main boundaries: block model, execution engine, UI widgets, and sharing/storage subsystem.
13. External dependencies that matter architecturally
- The app relies on browser-native APIs, plus a few bundled libraries in lib/ such as jQuery, require.js, Chart.js, and code editor/highlight assets.
- Audio and speech capabilities depend on bundled samples/voice data and browser audio behavior.
- The important architectural point is that most dependencies are vendored and front-end focused; there is no central backend service required for basic operation.
14. How to diagram it
- A useful diagram should show: browser UI and asset layer at the top; block palette/workspace in the middle; the logo interpreter and turtle/audio engine beneath; widgets as sidecar tools connected to the same runtime; and the Planet/project-sharing subsystem as an optional external-facing branch.
- Separate the offline core from optional services like sharing, localization conversion, and packaging/testing infrastructure.
Main entry points: activity.js Activity core · package.json Build tools · widgetWindows.js Widgets · logo.js Logo runtime
Components 20
Core app
Browser-based visual programming environment and its execution engine.
- App shell · Main browser entrypoint and single-page app bootstrap.
index.html - Startup code · Application initialization and runtime wiring.
index.js - Activity core · Activity-level state, lifecycle, and browser event coordination.
js/activity.js - Workspace · Block editing, palette organization, and shared block infrastructure.
js/blocks.js - Block model · Core block creation, prototypes, and factory logic.
js/block.js - Block families · Concept-specific block sets such as pitch, rhythm, graphics, flow, and widgets.
js/blocks/PitchBlocks.js
UI shell
Top-level controls, workspace chrome, and auxiliary panels.
Runtime and output
Interpretation, turtles, audio, notation, and export paths.
- Logo runtime · Executes block structures, resolves dependencies, and schedules actions.
js/logo.js - Turtle engine · Per-turtle orchestration for notes, visuals, and stateful actions.
js/turtles.js - Turtle actions · Music- and graphics-specific action handlers invoked by the runtime.
js/turtleactions/PitchActions.js - Audio stack · MIDI, notation, synthesis adapters, and export-friendly music utilities.
js/midi.js - JS export · Converts JavaScript back into block structures through AST and block generation utilities.
js/js-export/export.js
Planet sharing
Optional project browsing, storage, and publishing subsystem.
- Planet · Project browsing, storage, caching, viewing, and publishing.
planet/js/Planet.js - Project storage · Local/remote project persistence and request coordination.
planet/js/ProjectStorage.js
Build and support
Localization, packaging, docs, tests, and vendored libraries.
- Locales · Translation files and conversion pipeline for user-facing strings.
locales/en.json - Build tools · Packaging, linting, testing, CI, and app distribution tooling.
package.json - Docs · Guides, architecture notes, troubleshooting, and manuals.
Docs/guide/index.html - Vendored libs · Bundled browser libraries, voice data, and service-worker support assets.
lib/require.js
Connections 23
- App shell → Startup code: boots · The HTML shell loads the startup script.
- Startup code → Activity core: initializes · Startup wires the activity lifecycle and app state.
- Activity core → Toolbar: assembles · Activity code mounts top-level controls.
- Activity core → Palettes: mounts · The shell connects editing and palette controls.
- Activity core → Widgets: opens · Activity state drives auxiliary widget windows.
- Activity core → Planet: links · Activity can hand off to the sharing subsystem.
- Workspace → Block model: uses · Workspace behavior depends on the shared block model.
- Block model → Block families: organizes · Core block infrastructure instantiates the domain-specific families.
- Block families → Logo runtime: compile to · Blocks are executed through the Logo runtime.
- Logo runtime → Turtle engine: drives · Execution schedules turtle-level actions and state.
- Turtle engine → Turtle actions: dispatches · Turtle runtime invokes domain action handlers.
- Turtle actions → Audio stack: produces · Actions emit musical output and notation-facing state.
- Audio stack → Vendored libs: adapts with · Audio and playback depend on bundled browser libraries.
- Logo runtime → JS export: supports · Execution model is mirrored by the JS-to-block export path.
- Widgets → Turtle engine: inspects · Widgets share the same execution and music runtime.
- Widgets → Audio stack: controls · Several widgets manipulate synthesis and musical parameters.
- Planet → Project storage: persists · Planet coordinates storage, cache, and request management.
- Planet → Activity core: integrates · Planet is an optional UI branch connected to the main app.
- Build tools → App shell: packages · Tooling builds the browser entry and distributable assets.
- Build tools → Docs: validates · Build and CI workflows include documentation and quality checks.
- Build tools → Locales: generates · Translation conversion is part of the build pipeline.
- Docs → Workspace: describes · Documentation explains the block-based programming model.
- Vendored libs → App shell: supports · Vendored assets support the browser application shell.