How cybercentrecanada/assemblyline-base works
Main Components
a) Data Storage Layer:
• CacheStore (/assemblyline/cachestore)
• DataStore (/assemblyline/datastore)
• FileStore (/assemblyline/filestore)
b) Object Document Mapper (ODM) Layer:
• Models (/assemblyline/odm/models)
• Messages (/assemblyline/odm/messages)
c) Remote Data Types Layer:
• Queues
• Caches
• Counters
• Sets
• Locks
d) Common Utilities Layer:
• Classification
• Security
• Identification
• Archiving
• Metrics
Component Relationships
┌─────────────────────────────────────────────────────────┐
• Application Layer │
• ┬─────────────────┬──────────────────┘
┌───────────────────▼─────┐ ┌────────▼────────┐
• Storage Layer │ │ ODM Layer │
• ┌─────┐ ┌─────┐ ┌────┐│ │ ┌────────────┐ │
• Cache│ │Data │ │File││ │ │ Models │ │
• Store│ │Store│ │Store││ │ └────────────┘ │
• ┘ └─────┘ └────┘│ │ ┌────────────┐ │
• ┘ │ │ Messages │ │
• ┘ │
• ┘
┌──────────────────────────────────────────┐
• Remote Data Types │
• ┌─────┐ ┌─────┐ ┌─────┐ ┌────┐ ┌────┐ │
• Queue│ │Cache│ │Count│ │Sets│ │Lock│ │
• ┘ └─────┘ └─────┘ └────┘ └────┘ │
• ┘
┌──────────────────────────────────────────┐
• Common Utilities │
• Security, Classification, Identification │
• ┘
Important Details
a) Storage Layer:
• Supported backends (Azure, S3, FTP, HTTP, Local)
• Classification boundaries
• Caching mechanisms
b) ODM Layer:
• Models (File, Result, Submission)
• Message types for system communication
• Validation flows
c) Remote Data Types:
• Queue priority systems
• Distributed lock mechanisms
• Counter and cache relationships
Implementation Notes
• Version information (4.x)
• Notes about Python 3.11 requirement
• Containerization boundaries
Remember to maintain a balance between detail and clarity, focusing on the core components while providing enough information to understand the system's architecture.