Tutorials and examples of how to deploy Presto and connect it to different data sources

This project offers a robust data pipeline solution designed to efficiently extract, transform, and load (ETL) Reddit data into a Redshift data warehouse. Leveraging a blend of industry-standard tools and services, the pipeline ensures seamless data processing and integration.

This project demonstrates how you can build downstream data pipeline using dbt in athena

AgTech BI dashboard project using Python, SQLite, SQL, CSV and Power BI for telemetry analytics.

Data Engineering Specialization offered by Joe Reis in partnership with DeepLearning.AI through Coursera...

AWS big data pipeline project: S3 upload triggers Lambda to start a Glue ETL job that transforms CSV data and loads ~5.6M records into DynamoDB, with scripts to provision and tear down all infrastructure

AWS Glue is a fully managed, serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning, and application development. It automates the ETL (Extract, Transform, Load) pipeline process, allowing users to focus on building workflows rather than managing infrastructure.

This project focuses on real-time data streaming with Kinesis, using Flink for advanced processing and OpenSearch for analytics. This architecture has succinctly handled the complete lifecycle of data from ingestion to actionable insights, making it a comprehensive solution.

End-to-end AWS data pipeline with ML churn prediction | S3 · Glue · Athena · Lambda · Python · Power BI

This projects uses ETL (Extract, Transform and Load) pipeline to extract data from Spotify using its API and loads the data to a data source(AWS Athena). The entire pipeline will be built using Amazon Web Services (AWS).

In this project I have used the Trending YouTube Video Statistics data from Kaggle to analyze and prepare it for usage.

I am dedicated to delivering innovative solutions that align with business objectives while ensuring optimal performance, reliability, and security. My strong analytical skills, attention to detail, and problem-solving abilities drive me to create effective and efficient solutions.

Transformed YouTube’s raw JSON data to parquet & loaded it in an S3 bucket, used Glue Data Catalog for storing metadata & Athena to query the cleaned data. Developed an ETL process using a Lambda job that would be triggered when raw data is loaded into an S3 bucket, processed, and stored for analytical purposes in an S3 bucket.

Serverless AWS ETL pipeline (Glue, S3, Redshift) with data quality gates and Step Functions orchestration. Transforms transaction data with PySpark, validates it, and loads it into Redshift for reporting.

Incremental Data Load from S3 Bucket to Amazon Redshift Using AWS Glue

Big data and Cloud Deployment

Built a cloud-native ETL pipeline using Python, AWS, and Snowflake to analyze public job listings. Automated data ingestion with AWS Lambda, transformation with AWS Glue (PySpark)Stored cleaned data in Amazon S3 and loaded it into Snowflake for advanced analytics. Visualized job trends, skill demand, and salaries using interactive Tableau dashboard

Power BI dashboard for Memphis public safety analytics, built on AWS Athena, Glue, and S3.