Skip to content

Latest commit

 

History

History
32 lines (29 loc) · 3.71 KB

README.md

File metadata and controls

32 lines (29 loc) · 3.71 KB

Data Architecture

Architecture

Technical Stack

Tools Distributions Descriptions Status
kubernetes Container orchestration Done
kubeadm Container orchestration Done
k0s Container orchestration Done
rke2 Container orchestration Done
k3s Container orchestration Done
Minio Secure object storage solution for reliable data storage in distributed environments. Done
Hive Metastore Schema management tool ensuring seamless evolution and organization of data. Done
Trino High-performance query engine for distributed data processing. Done
Spark Distributed processing engine Done
Doris Real-time data warehouse Done
StarRocks Real-time data warehouse Not started
Kafka Stream-processing platform Done
Flink Robust stream processing framework for real-time data analytics. In Process
RisingWave streaming database Not started
Camel K Open Source integration framework Not started
Airflow Workflow management platform Done
Superset Data exploration and data visualization Not started
Jupyter Notebook Web-based interactive development environment for notebooks, code, and data. Not started
MLflow A platform to streamline machine learning development Not started
Streamlit Python framework for data scientists and AI/ML engineers to deliver interactive data apps. Not started
Spring Boot Web framework for building RESTful APIs in Java. Not started
FastAPI Modern web framework for building RESTful APIs in Python. Not started
Jenkins Open-source CI/CD server Not started
Argo CD Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes. Not started