Skip to content

Repository files navigation

Data Management Project

This project is the simulation of a real-world e-commerce data environment. Documentation of developing end-to-end data management.

Business problem:

Imagine an e-commerce platform with 1000 products. How to simulate a real-world e-commerce data environment for this business?

Aim of the project

  • To develop a thorough understanding of relational database design and management.
  • To gain proficiency in automating data-related tasks with GitHub Actions.
  • To enhance skills in data analysis and report generation using R and Quarto.

Executive summary of the project:

  • Database Design and Implementation
    • Entity-Relationship (E-R) Diagram
    • SQL Database Schema Creation
  • Data Generation and Management
    • Synthetic Data Generation
    • Data Import and Quality Assurance
  • Data Pipeline Generation
    • GitHub Repository and Workflow Setup
    • GitHub Actions for Continuous Integration
  • Data Analysis and Reporting with Quarto in R
    • Advanced Data Analysis in R
    • Comprehensive Reporting with Quarto
  1. E-R Diagram Design. The key entities of this e-commerce platform has been identified and they are Suppliers, Customers, Products, Product Categories, Orders, Order Details, Transactions, and Promotions. The E-R was designed using an online flowchart maker website called draw.io, which is shown below.

ER Diagram

  1. Advanced Data Analysis in R

Using R programming language, key characteristics such as total orders, total revenue, and best-selling products are extracted for the e-commerce platform. The graph below shows that the best-selling product categories.

Data-Analysis Best-Selling Product Categories

About

Data Management - Simulation of a real-world e-commerce data environment. Documentation of developing end-to-end data management. Involves database design using E-R diagram, database management using SQLite, automation of workflow processes using GitHub Actions, data analysis with R, and reporting on Quarto.

Resources

Stars

1 star

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages