How petrobras/3w works
Main Components
The system should be divided into three primary components:
a) Dataset Component:
• Numbered subdirectories (0-9) containing WELL and SIMULATED data files
• Parquet file storage format
• Dataset versioning system
• Dataset structure documentation
b) Toolkit Component:
• Base module
• Dataset handling
• Processing (feature & label mappers)
• Training (multiclass experiments)
• Visualization
• Utils
c) Problems Component:
• Individual problem definitions
• Baseline implementations
• Evaluation metrics
Relationships and Data Flow
• Dataset → Toolkit (data ingestion)
• Toolkit → Problems (problem-specific processing)
• Problems → Toolkit (results and metrics)
Specific elements
• Data format specifications (Parquet)
• Version control integration
• Environment management (conda)
• Documentation links
• License information
Architectural Patterns
Highlight:
• Modular design of toolkit
• Pipeline processing flow
• Experiment management structure
• Version control system
┌─────────────────────────────────────────────────────┐
• Documentation Layer │
• (README, Structure Docs, License, Contributing) │
• ┤
• Dataset Layer │
• ┌─────────┐ ┌─────────┐ ┌─────────┐ │
• Well │ │Simulated│ │ Drawn │ │
• Data │ │ Data │ │ Data │ │
• ┘ └─────────┘ └─────────┘ │
• ┤
• Toolkit Layer │
• ┌─────────┐ ┌─────────┐ ┌─────────┐ │
• Dataset │ │Processing│ │Training │ │
• Handling │───▶│ Module │───▶│ Module │ │
• ┘ └─────────┘ └─────────┘ │
• ▲ │ │ │
• ▼ ▼ │
• ┌─────────┐ ┌─────────┐ ┌─────────┐ │
• Utils │ │Visualiza│ │Problems │ │
• Module │◀───│ tion │◀───│ Module │ │
• ┘ └─────────┘ └─────────┘ │
• ┘
Key Considerations:
Integration with external tools (conda, jupyter)
• Double lines: API boundaries