I'm a Data Engineer & Analytics professional with 2+ years of experience building robust data pipelines, automations, and turning complex data into actionable insights. I hold a B.Tech in Mechanical Engineering (2021) and a PGP in Data Science and Engineering from Great Learning.
What I do:
- 🔧 Design and build end-to-end data pipelines on Databricks
- ⚡ Optimize PySpark & Spark SQL for large-scale data processing
- 🖥️ Build Databricks Apps and Streamlit applications for data tools
- 📊 Create interactive dashboards with Power BI
- 🤖 Automate workflows using Python & Selenium
What drives me: The challenge of building data systems that teams can depend on - from raw ingestion to polished dashboards. There's something satisfying about transforming chaotic data into clean pipelines and clear insights.
Deepening my expertise in building production-grade data systems:
| Area | Focus |
|---|---|
| Data Pipelines | Delta Live Tables, Structured Streaming, Batch & Real-time Ingestion |
| Databricks Platform | Performance Tuning, Unity Catalog, Workflows Orchestration, Databricks Apps |
| Delta Lake | Z-ordering, Compaction, CDC, Liquid Clustering, VACUUM |
| Spark Optimization | Partitioning, Caching, Broadcast Joins, Adaptive Query Execution |
├── Data Engineering
│ ├── PySpark & Spark SQL
│ ├── ETL/ELT Pipeline Design
│ ├── Delta Lake & Data Lakehouse
│ ├── Data Ingestion (Event Hub, Blob Storage)
│ └── Databricks Workflows & Jobs
│
├── Data Processing
│ ├── Large-scale Transformations
│ ├── Data Quality & Validation
│ ├── Performance Optimization
│ └── Structured Streaming
│
├── Apps & Automation
│ ├── Databricks Apps
│ ├── Streamlit Applications
│ ├── Selenium Web Automation
│ └── Python Scripting
│
└── Analytics & Visualization
├── Power BI Dashboards
├── SQL Modeling & Optimization
├── Python Data Analysis (Pandas, NumPy)
└── Data Storytelling & Reporting
✅ Data Pipeline Design - Building reliable ETL/ELT workflows on Databricks
✅ Databricks Apps & Streamlit - Building internal data tools and applications
✅ SQL Modeling & Optimization - Complex queries, performance tuning, data modeling
✅ Data Visualization & BI - Power BI dashboards, Python visualization libraries
✅ Automation - Selenium web automation, Python scripting to reduce manual work
✅ Cloud - Azure (Event Hub, Blob Storage), Databricks, Snowflake
- PGP in Data Science and Engineering - Great Learning
- B.Tech in Mechanical Engineering - 2021
- 📚 Preparing for: Databricks Data Engineer Associate Certification
- 🔧 Data Pipeline Projects - Building scalable ETL/ELT solutions
- ⚡ Spark & Delta Lake - Performance tuning and lakehouse patterns
- 🖥️ Databricks Apps - Building data tools and applications
- 📊 Analytics & Dashboards - BI solutions and data visualization
- 🚀 Open Source - Contributing to data engineering projects
When I'm not building pipelines, you can find me:
- 🌿 Gardening - Growing vegetables and experimenting with plant care
- 🧁 Baking - Creating desserts and exploring new recipes
- 💎 Diamond Painting - Relaxing with some sparkle therapy
"From raw data to reliable pipelines to clear insights." 🔧📊