Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 16 additions & 15 deletions .github/workflows/python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,32 +13,33 @@ jobs:
strategy:
fail-fast: true
matrix:
python-version: ["3.9", "3.10"]
python-version: ["3.12"]
steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install python dependencies
check-latest: true
- name: Install uv and sync dependencies
run: |
pip3 install -e .[dev]
python -m pip install --upgrade pip
python -m pip install uv
uv sync --all-groups
- name: Lint with flake8
run: |
bash py-lint.sh
- name: Check static types with mypy
uv run bash py-lint.sh
- name: Check static types with ty
run: |
mypy traceflow --ignore-missing-imports
mypy tests --ignore-missing-imports
uv run ty check traceflow tests
- name: Test with pytest
run: |
pytest -sv tests
uv run pytest -sv tests
- name: Generate coverage report
run: |
sudo apt-get install -y lcov
coverage run -m pytest -sv tests
coverage html --omit tests/*.py
coverage lcov
uv run coverage run -m pytest -sv tests
uv run coverage html --omit tests/*.py
uv run coverage lcov
ls
# Coverage is now in htmlcov/

3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -163,3 +163,6 @@ cython_debug/
.DS_Store

test.pdf
playwright/node_modules/
*.webm
playwright/test-results/
2 changes: 0 additions & 2 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
include LICENCE
include setup*.py
include README.md
include requirements.txt
include MANIFEST.in

recursive-include traceflow *
Expand Down
26 changes: 25 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ By leveraging TraceFlow, you can:
- Generate a traceability matrix linking all requirements to all tests.
- Create fillable PDF forms for manual tests.
- Run automated tests and capture their output as Markdown, then include them in the PDF report.
- The result will look something [like this](example.pdf)

## Getting Started

Expand Down Expand Up @@ -38,10 +39,33 @@ Create your Markdown files based on the provided examples:

With your Markdown files in place, you can run TraceFlow using the following command:

traceflow path/to/project-docs project.pdf
traceflow path/to/project-docs 1.0.0 project.pdf

The second argument is the version string that will be shown on the report (change it to whatever release identifier you need).

Two optional arguments let you replace the logos that appear in the header (top-left) and footer (top-right):

- `--top-left-logo` (alias `--traceflow-logo`) points to the image shown in the top-left of every page (header).
- `--top-right-logo` (alias `--voxelflow-logo`) points to the image shown in the bottom-left of every page (footer).

If you do not pass these options, TraceFlow falls back to the packaged images in `traceflow/res/`.

traceflow path/to/project-docs 1.0.0 project.pdf --top-left-logo assets/traceflow.png --top-right-logo assets/voxelflow.png

This command will generate a PDF for all of your documentation and test cases, as well as a validation pack and traceability matrix.

### Running the Playwright example

TraceFlow includes the `examples` directory, which demonstrates `autoplaywright` tests that invoke the Playwright scripts in this repo. To reproduce that example, run:

```
uv run traceflow examples 1.0.0 test.pdf --playwright-dir playwright/
```

On macOS you may need to prefix the command with `DYLD_FALLBACK_LIBRARY_PATH=/opt/homebrew/lib` so the bundled `cairosvg` finds `libcairo`. This example writes `test.pdf` into the current directory, stores temporary LaTeX artifacts in `report/`, and captures the Playwright video and logs via `playwright/run-test-video.sh`.

If your documentation contains `autoplaywright` test blocks that capture Playwright runs, append `--playwright-dir PATH` so TraceFlow can invoke the runner and gather the videos + logs that back these tests.

### Requirements
```
# Requirements
Expand Down
Binary file added example.pdf
Binary file not shown.
27 changes: 24 additions & 3 deletions examples/tests/manual-acceptance-test.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,20 +25,41 @@ The user is logged in and redirected to the main dashboard.
```manualtest
```

## TEST-002: Automatic test
## TEST-002: Automatic Playwright test

**Requirement ID:** REQ-003

### Test Steps:

1. This automated test creates a user, logs in, and then logs out.
1. The `autoplaywright` block runs the Playwright sample located in the `playwright/` directory and captures the video, stdout/stderr, and a 3×3 key frame grid from the test execution.

### Expected Result:

The test should pass successfully.

### Test Outcome:

```autoplaywright
0001-test-counter
```

## TEST-003: Automatic test

**Requirement ID:** REQ-003

### Test Steps:

1. This `autotest` block demonstrates running a simple pytest command as part of the report.

### Expected Result:

The test should pass successfully.

### Test Outcome:

```autoplaywright
0001-test-counter
```
```autotest
pytest --disable-warnings -sv tests/test_hello_world.py
```
```
15 changes: 15 additions & 0 deletions playwright/counter.spec.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
// counter.spec.js
const { test, expect } = require('@playwright/test');
const path = require('path');

test('0001-test-counter', async ({ page }) => {
await page.goto('file://' + path.join(__dirname, 'index.html'));

await expect(page.locator('#count')).toHaveText('0');
await page.waitForTimeout(300);

await page.click('#increment');

await expect(page.locator('#count')).toHaveText('1');
await page.waitForTimeout(300); // To improve video output
});
34 changes: 34 additions & 0 deletions playwright/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>Simple Counter App</title>
<style>
body {
font-family: sans-serif;
padding: 2rem;
}
#count {
font-size: 2rem;
margin-right: 1rem;
}
</style>
</head>
<body>
<h1>Simple Counter App</h1>
<div>
<span id="count">0</span>
<button id="increment">Increment</button>
</div>

<script>
const countSpan = document.getElementById('count');
const button = document.getElementById('increment');

button.addEventListener('click', () => {
const current = parseInt(countSpan.textContent, 10);
countSpan.textContent = String(current + 1);
});
</script>
</body>
</html>
79 changes: 79 additions & 0 deletions playwright/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 12 additions & 0 deletions playwright/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"name": "playwright-simple-example",
"version": "1.0.0",
"private": true,
"scripts": {
"test": "playwright test"
},
"devDependencies": {
"@playwright/test": "^1.56.1",
"playwright": "^1.48.0"
}
}
5 changes: 5 additions & 0 deletions playwright/playwright.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
module.exports = {
use: {
video: 'on',
}
};
56 changes: 56 additions & 0 deletions playwright/run-test-video.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
#!/usr/bin/env bash
set -euo pipefail

if [ $# -ne 3 ]; then
echo "Usage: ./run-test-video.sh \"test name regex\" output.webm output.txt"
exit 1
fi

TEST_NAME="$1"
OUTPUT_VIDEO="$2"
OUTPUT_TEXT="$3"

# Ensure clean test-results
if [ -d test-results ]; then
echo "▶ Removing old test-results folder..."
rm -rf test-results
fi

echo "▶ Running test: $TEST_NAME"
echo "▶ Video will be saved as: $OUTPUT_VIDEO"
echo "▶ Text output will be saved as: $OUTPUT_TEXT"

# Temporary file to capture test output
TMP_OUTPUT=$(mktemp)

# Run Playwright, capturing output while also displaying it live
# tee duplicates output: both to terminal AND TMP_OUTPUT
set +e
set +e
NODE_NO_WARNINGS=1 npx playwright test -g "$TEST_NAME" 2>&1 | tee "$TMP_OUTPUT"
TEST_EXIT_CODE=${PIPESTATUS[0]}
set -e

echo "▶ Playwright exit code: $TEST_EXIT_CODE"

# Write captured output to desired file
mv "$TMP_OUTPUT" "$OUTPUT_TEXT"
echo "📝 Saved test output to: $OUTPUT_TEXT"

# Find the generated video file
VIDEO_PATH=$(find test-results -type f -name "video.webm" | head -n 1 || true)

if [ -z "$VIDEO_PATH" ]; then
echo "❌ ERROR: Could not find video.webm in test-results/"
exit 2
fi

echo "▶ Found video: $VIDEO_PATH"
echo "▶ Moving to: $OUTPUT_VIDEO"

mv "$VIDEO_PATH" "$OUTPUT_VIDEO"

echo "🎥 Saved video as: $OUTPUT_VIDEO"

# Exit using the test’s exit code
exit $TEST_EXIT_CODE
8 changes: 8 additions & 0 deletions playwright/run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# 1. Install dependencies
npm install

# 2. Install Playwright browsers (once)
npx playwright install

# 3. Run the test
npm test -- --video=run1.webm
Loading