Skip to content
View Aashish-Pandey07's full-sized avatar

Block or report Aashish-Pandey07

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
Aashish-Pandey07/README.md

Hi πŸ‘‹, I'm Aashish Pandey

Data Engineer | BI Developer, Agile Solutions Nepal

aashish-pandey07

  • πŸ’Ό BI Developer at Agile Solutions Nepal β€” SQL Server, SSAS, Power BI, and ERP data migrations from Dynamics NAV β†’ Business Central
  • πŸ”§ Outside of work I build data engineering systems from scratch β€” warehouses, ETL frameworks, streaming pipelines, an orchestrator, a metadata catalog, an observability tool β€” to understand how tools like Airflow, dbt, and Monte Carlo actually work under the hood
  • 🧠 Also have academic project experience in ML/Deep Learning β€” image classification, computer vision, predictive modeling
  • 🎯 Looking for Data Engineer / Analytics Engineer roles, Nepal and remote
  • πŸ“« ap2081148@gmail.com Β· LinkedIn
  • ⚑ Fun fact: every project below runs on my own laptop β€” no cloud bill, no Docker. DuckDB and SQLite as the warehouse, and it still holds up at millions of rows.

πŸš€ Data Engineering Projects

Project What it does Stack
retail-dwh Kimball star schema warehouse on 531K+ retail transactions β€” SCD Type 2, incremental + idempotent loads, 29 data quality checks gating every load, Streamlit dashboard on top. 162 tests. Python, SQL, DuckDB, Streamlit
etl-framework Config-driven ETL framework β€” point it at a YAML file, it extracts/validates/transforms/loads/quality-checks, no code changes per dataset. CSV, Excel, JSON, REST API sources; plugin architecture; incremental watermarking. 50 tests. Python, DuckDB, YAML
mini-workflow-scheduler Built my own mini-Airflow to understand orchestration from first principles β€” DAG dependency resolution (Kahn's algorithm), parallel task execution, retries, failure recovery with resumable runs, SQLite state store. Zero external dependencies. 31 tests. Python, SQLite (stdlib only)
kafka-order-pipeline Event-driven streaming pipeline on Kafka 4.3 (KRaft) β€” idempotent producer, dead-letter queue for bad payloads, incremental loading from consumer offsets, 5 data quality checks. 29 tests. Python, Apache Kafka, DuckDB
nyc-taxi-pipeline Batch ETL over ~3M NYC taxi trips/month without ever loading the full dataset into memory β€” chunked PyArrow streaming, hive-partitioned parquet lake. ~9.2s runtime, ~455MB peak memory. 24 tests. Python, PyArrow, pandas, DuckDB
github-analytics ETL pipeline that builds the history GitHub's API doesn't give you β€” daily snapshots via 7 REST endpoints, with pagination, rate-limit handling, and retries. 17 data quality checks. 283 tests. Python, DuckDB, GitHub REST API
data-observability A Monte Carlo/Soda-style observability tool, built on the standard library only β€” 7 health checks (schema drift, freshness, nulls, duplicates, distribution, volume) with HTML/JSON/Markdown reports and trend sparklines. 34 tests. Python (stdlib only)
data-lineage-catalog A metadata catalog and lineage tracker in the spirit of Amundsen/DataHub β€” schema discovery, column profiling, declared + inferred lineage, rendered as an interactive HTML graph. 31 tests. Python (stdlib only)

🧠 Machine Learning / Deep Learning Projects

Project What it does Stack
Plant Leaf Disease Classification Comparative benchmark of multiple deep learning architectures for image classification Python, TensorFlow, Jupyter
Fitness at Home β€” Human Pose Estimation Real-time exercise form tracking using pose estimation Python, OpenCV, Jupyter
Diabetes Detection System Predictive model for early diabetes risk detection C++, ML
AI Healthcare System with Blockchain Healthcare data records app combining AI with blockchain for verifiability TypeScript

🧰 Core Stack

Data Engineering & Cloud

sqlserver python azure git postgresql mysql mongodb

BI & Analytics

powerbi excel

Machine Learning & AI

pytorch tensorflow pandas scikit_learn opencv

Data warehousing & tooling used in the projects above

duckdb kafka sqlite streamlit

Also comfortable with: C, C++, JavaScript, React, Node.js, PHP (prior full-stack coursework β€” kept for breadth, not my current focus)


🀝 Connect with me

linkedinΒ  gmail

Pinned Loading

  1. retail-dwh retail-dwh Public

    End-to-end retail data warehouse on DuckDB: incremental ETL, Kimball star schema, data quality gates, and a Streamlit dashboard.

    Python

  2. etl-framework etl-framework Public

    Configuration-driven ETL framework in Python β€” YAML-defined pipelines, plugin architecture, incremental loading, data quality checks, DuckDB warehouse.

    Python

  3. mini-workflow-scheduler mini-workflow-scheduler Public

    Minimal workflow orchestrator in pure Python: DAG scheduling, parallel task execution, retries, resumable runs and SQLite-backed state. Zero external dependencies.

    Python

  4. kafka-order-pipeline kafka-order-pipeline Public

    Event-driven streaming pipeline: Python producer/consumer, Apache Kafka (KRaft), DuckDB warehouse, dead-letter handling, SQL analytics.

    Python

  5. nyc-taxi-pipeline nyc-taxi-pipeline Public

    Memory-bounded ETL pipeline for NYC Taxi data β€” PyArrow streaming, DuckDB warehouse, SQL analytics.

    Python

  6. Comparative-Analysis-of-Deep-Learning-Models-for-Plant-Leaf-Disease-Classification Comparative-Analysis-of-Deep-Learning-Models-for-Plant-Leaf-Disease-Classification Public

    Jupyter Notebook