Skip to content
View ibrahim-yeryaran's full-sized avatar

Block or report ibrahim-yeryaran

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
ibrahim-yeryaran/README.md

Hi, I'm İbrahim 👋

Junior Data Engineer — I build end-to-end data pipelines: ingesting data, moving it reliably, modeling it, and serving it. Comfortable across batch, streaming, and cloud, with a habit of writing tested, documented, CI-checked code.

  • 🔭 Building a portfolio of hands-on data-engineering projects (see below)
  • 🌱 Deepening my skills in orchestration, cloud warehouses, and distributed processing
  • 💬 Ask me about ETL/ELT, dbt modeling, Kafka streaming, Spark, or BigQuery
  • 📫 Reach me: ibrahimyeryaran@gmail.com · LinkedIn

🛠️ Tech Stack

Languages · Python · SQL

Processing & Modeling · Apache Spark (PySpark) · dbt · pandas

Orchestration · Apache Airflow

Streaming · Apache Kafka

Storage & Warehouses · PostgreSQL · DuckDB · Google BigQuery

DevOps · Docker · Docker Compose · Git · GitHub Actions (CI/CD)

📊 Featured Projects

Every project has a README, tests, and a green CI pipeline.

Project What it demonstrates Stack
🌬️ EU Air Quality Platform ⭐ Capstone: one Airflow DAG orchestrating ingestion → dbt → a data-quality gate → EU/WHO exceedance reporting Airflow, dbt, PostgreSQL, Docker
🚕 NYC Taxi — PySpark Analytics Distributed processing of ~9M rows: broadcast joins, window functions, mixed-schema handling PySpark, Java
📡 Wikimedia Real-Time Pipeline Real-time streaming of Wikipedia's live edit feed into running aggregates Kafka, PostgreSQL, Docker
🚕 NYC Taxi — dbt Pipeline ELT + a tested, documented star schema (fact/dim, referential-integrity tests) dbt, DuckDB
☁️ CO₂ Cloud Analytics — BigQuery Cloud warehouse with cost-aware partitioning & clustering BigQuery, SQL
🌦️ European Weather Pipeline Scheduled batch ingestion orchestrated with Airflow Airflow, PostgreSQL, Docker
🌍 CO₂ Emissions Explorer Interactive dashboard with a live public demo 🔗 Streamlit, Plotly

Together they cover the full data-engineering lifecycle: ingestion → streaming → transformation → modeling → cloud warehousing → distributed processing → serving — integrated end-to-end in the capstone platform.

Pinned Loading

  1. bigquery-co2-pipeline bigquery-co2-pipeline Public

    Cloud data pipeline loading the global OWID CO₂ dataset into Google BigQuery and transforming it into a partitioned, clustered analytics layer with SQL — on the free BigQuery sandbox.

    Python

  2. co2-emissions-dashboard co2-emissions-dashboard Public

    Interactive Streamlit dashboard exploring global CO₂ emissions (Our World in Data) — country comparison, metric selector, and interactive charts. Live demo included.

    Python

  3. european-weather-pipeline european-weather-pipeline Public

    End-to-end ETL pipeline: fetches live European weather from Open-Meteo, stores it in PostgreSQL, orchestrated hourly with Apache Airflow — fully containerized with Docker.

    Python

  4. nyc-taxi-dbt-pipeline nyc-taxi-dbt-pipeline Public

    ELT pipeline loading 3M+ real NYC taxi trips into DuckDB and transforming them into a tested star schema with dbt — reproducible in two commands.

    Python

  5. nyc-taxi-spark nyc-taxi-spark Public

    PySpark batch pipeline processing ~9M NYC taxi trips: broadcast joins, window functions, partitioned parquet output — with a real mixed-schema fix.

    Python

  6. wikimedia-realtime-pipeline wikimedia-realtime-pipeline Public

    Real-time streaming pipeline ingesting Wikipedia's live edit feed through Apache Kafka (KRaft) into PostgreSQL aggregates — fully containerized with Docker.

    Python