How theendofline/webhook-firestore-sheets-pipeline works
Main Components
a) External Systems:
• Service X (Data Source)
• Google Sheets (Final Destination)
b) Google Cloud Components:
• Cloud Functions (2 separate functions)
• Cloud Firestore
• Cloud Scheduler
• IAM/Service Accounts
Component Relationships
• Service X → webhook-to-firestore-pipeline (HTTP POST/Webhook)
• webhook-to-firestore-pipeline → Firestore (Write)
• Cloud Scheduler → firestore-to-google-sheets-pipeline (HTTP Trigger)
• firestore-to-google-sheets-pipeline → Firestore (Read)
• firestore-to-google-sheets-pipeline → Google Sheets (Write)
Additional Details
Annotations for:
• Data types being transferred (date, name, hyperlink)
• Authentication points (service account)
• Validation steps
• Transformation processes (timestamp localization)
• Scheduling frequency
Process Highlights
• Data validation
• Deduplication
• Timestamp conversion
• Data reconciliation