Skip to content

Repository files navigation

cpp-pipeline-engine

CI

🚧 Project under construction

Description: a modular C++ library used to build data processing pipelines. It acts as an interface for other applications to read data from different sources, easily transform it and export it to different destinations.

Stack

Layer Technology
Language C++20
Build CMake
Tests GoogleTest + GMock
Quality Clang-format + Clang-tidy + sanitizers
CI GitHub Actions

Documentation

  • Requirements: what the product must do
  • Architectural drivers: what shapes this design
  • Conceptual model: what the tool does
  • Decisions map: architecturally significant decisions + dependency sequence with reversal cost
  • Error scenarios: everything that can fail as data flows through the pipeline
  • Architecture Decision Records (docs/adr/): ADR-000 (ADR process), ADR-001 through ADR-004 (data model), ADR-005 through ADR-007 (execution model), ADR-008 through ADR-013 (error model), ADR-014 through ADR-016 (extensibility)

Status

Implemented so far: the data model, the acquisition stage (in-memory and file components), record delimitation, record parsing, and the pipeline engine that orchestrates them under a pull-based model, returning records one at a time. Serialization and delivery are designed but not yet built.

About

Modular C++ library used to build data processing pipelines

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages