Introduction and Architecture
Introduction and Architecture
Integration three point zero is an in-house ETL solution that provides a visual, low-code environment for building business logic and integrating external systems with the o nine ecosystem.
It replaces traditional SSIS-based SQL Server execution with massive parallel processing via Spark on Kubernetes clusters.
Primary storage utilizes Cloud Data Lakes (AWS S three, Google Cloud Storage, Azure Data Lake Storage) rather than SQL servers.
Parquet is the standard storage format, optimized for reading and storage via its columnar structure.
Airflow is integrated into the backend to generate and orchestrate Data Directed Acyclic Graphs.
Data Lake Architecture
Data Lake Architecture
Integration three point zero organizes the o nine Data Lake into distinct zones to track data progression.