Docs · Report Bug · Roadmap · Get Help · Watch Demo
GlassFlow is an open-source ETL tool that enables real-time data processing from Kafka to ClickHouse with features like deduplication and temporal joins.
This guide walks you through a local installation using Docker Compose — perfect for development, testing, or trying out GlassFlow on your machine.
- Clone the repository:
git clone https://github.com/glassflow/clickhouse-etl.git
cd clickhouse-etl- Start the services:
docker compose up-
Access the web interface at
http://localhost:8080to configure your pipeline. -
View the logs:
# Follow logs in real-time for all containers
docker compose logs -f
# logs for the backend api
docker compose logs api -f
# logs for the UI
docker compose logs ui -fGlassFlow is open source and can be self-hosted on Kubernetes. GlassFlow works with any managed Kubernetes services like AWS EKS, GKE, AKS, and more. For local testing or a small POC, you can also use Docker and Docker Compose to run GlassFlow on your local machine.
| Method | Use Case | Docs Link |
|---|---|---|
| ☸️ Kubernetes with Helm | Kubernetes deployment | Kubernetes Helm Guide |
| 🐳 Local with Docker Compose | Quick evaluation and local testing | Local Docker Guide |
| ☁️ AWS EC2 with Docker Compose | Lightweight cloud deployment for testing | AWS EC2 Guide |
See a working demo of GlassFlow in action at demo.glassflow.dev.
GlassFlow Pipeline showing real-time streaming from Kafka through GlassFlow to ClickHouse
For detailed documentation, visit docs.glassflow.dev. The documentation includes:
Check out our public roadmap to see what's coming next in GlassFlow. We're actively working on new features and improvements based on community feedback.
Want to suggest a feature? We'd love to hear from you! Please use our GitHub Discussions to share your ideas and help shape the future of GlassFlow.
- Real-time data processing from Kafka to ClickHouse
- Deduplication with configurable time windows
- Temporal joins between multiple Kafka topics
- Scalable and robust architecture built for Kubernetes
- Web-based UI for pipeline management
- Docker version for local testing and evaluation
This project is licensed under the Apache License 2.0.