ProperPrompt

ProperPrompt is an experiment to help improve user prompts through rating it by the LLM itself. It currently uses a very rudimentary framework - the LLM subjectively scores metrics like 'clarity', 'brevity', 'specificity' out of 10.0 on its own. It also considers our rating of its responses. ProperPrompt is, at its core, implemented as an MCP Server for persistent local memory to store prompting sessions along with their scores. The project further includes an accompanying CLI and a Visualiser to manage and graph sessions and statistics.

Usage Flow and Features

• User initiates a session
• ProperPrompt logs subsequent prompts and their scores automatically, till session is explicitly terminated.
• When you send a new prompt, the previous one is automatically classified as either requiring a followup or not, along with a rationale (retry, clarification, refinement, new topic).
• Optionally, rate how good LLM's responses were, to log and compare whether a better prompt resulted in a more satisfactory response.
• Prompt scores can be aggregated and displayed over a variety of time windows - 7 days, 30 days, 120 days, and all time.
• CLI: Query stats, browse prompt history, list sessions, export to markdown/csv.
• Visualiser: Prompt score trends over time, followup rate by week, and response rating distribution as matplotlib charts.

Installation and Setup

1. Clone the repo

git clone https://www.github.com/hrmtsh2/ProperPrompt.git
cd ProperPrompt

2. Create and activate a virtual environment

python -m venv venv

# Windows
venv\Scripts\activate

# macOS/Linux [NOT TESTED, but seems fine]
source venv/bin/activate

3. Install dependencies

pip install -r requirements.txt

4. Configure your MCP Client

For Claude Desktop, for example, add the following to your claude_desktop_config.json:

• Windows: %APPDATA%\Claude\claude_desktop_config.json
• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "quizzard": {
      "command": "C:/path/to/ProperPrompt/venv/Scripts/python.exe",
      "args": ["C:/path/to/ProperPrompt/server.py"]
    }
  }
}

Replace paths with your actual project path. Make sure to use forward slashed on Windows!

5. For Claude Desktop - Restart Claude Desktop

ProperPrompt will initialise automatically on first run, creating a local database at ~/.proper-prompt/prompts.db.

---

Usage

Start a session by asking Claude naturally:

"start a tracking session, I'm working on X"

That's it. From that point on, every message you send is logged and scored automatically for the rest of the conversation. A small counter appears at the bottom of each response (e.g. 3/) showing how many prompts have been logged in the current session.

To rate a response:

"rate that response 4 out of 5"

To check your stats:

"show my prompt quality stats"
"show stats for the last 30 days"

To end a session:

"end the session"

If you quit Claude without ending a session, you'll be prompted to close it the next time you start a new one.

---

Tools Implemented

Tool	Description
start_session	Start a new tracking session. Enables automatic logging for the conversation
end_session	Close the active session
log_prompt	Log a single prompt (called automatically — you shouldn't need this directly)
score_prompt	Score a prompt on clarity, brevity, specificity, and overall (called automatically)
rate_response	Rate the quality of Claude's response to a prompt (1–5)
get_stats	Aggregate quality stats across one or more time windows
get_prompt_history	Browse logged prompts with scores, followup inference, and ratings

CLI

# Stats across all time windows
python cli.py stats
 
# Stats for a single window
python cli.py stats --window 30d
 
# Browse recent prompts
python cli.py history
 
# Only scored prompts
python cli.py history --scored-only
 
# Filter to a session
python cli.py history --session s_20260524_143201_a3f9c1
 
# List all sessions
python cli.py sessions
 
# Close all open sessions
python cli.py close-all
 
# Export a session to markdown (default)
python cli.py export s_20260524_143201_a3f9c1
 
# Export to CSV
python cli.py export s_20260524_143201_a3f9c1 --format csv --out ~/Desktop/session.csv

---

Visualiser

# Interactive charts
python visualise.py
 
# Save to PNG
python visualise.py --out report.png
 
# Filter to a session or time window
python visualise.py --session s_20260524_143201_a3f9c1
python visualise.py --window 30

Architecture

prompt-quality-tracker/
├── server.py        # FastMCP server and all tool definitions
├── db.py            # SQLite initialisation and all query functions
├── heuristics.py    # Followup inference (keyword and time-gap based)
├── cli.py           # Terminal interface
├── visualise.py     # Matplotlib charts
├── requirements.txt
└── prompts.db       # Auto-created on first run

Scoring is done by the model in-context instead of, say, using LLM API calls. The model receives the rubric via the score_prompt tool's docstring and is instructed to call it immediately after each log_prompt. Followup inference is entirely local, using regex patterns (all generated through Claude itself), and a time-gap threshold.

---

Stats windows

Window	Coverage
7d	Last 7 days
30d	Last 30 days
120d	Last 4 months
all	All time

---

Followup inference

When a new prompt arrives, the previous prompt in the session is classified as:

Reason	Meaning
retry	Previous answer was wrong or off-target
clarification	Previous answer was confusing
elaboration	User wanted more depth on the same answer
refinement	User adjusted or narrowed the same request
new_topic	Unrelated question, or gap exceeded 5 minutes
session_ended	Last prompt in a closed session

---

Open to name change suggestions ;) (I was thinking of 'PromptPerfect' but 'perfection' will depend on the scoring framework used)