Skip to content

fix: remove artificial floor of 50 on risk score in scoring.py - #108

Open
danielbennett888 wants to merge 1 commit into
OpenNSWM-Lab:mainfrom
danielbennett888:fix/scoring-risk-floor
Open

fix: remove artificial floor of 50 on risk score in scoring.py#108
danielbennett888 wants to merge 1 commit into
OpenNSWM-Lab:mainfrom
danielbennett888:fix/scoring-risk-floor

Conversation

@danielbennett888

Copy link
Copy Markdown

Hi, first PR here. I was looking at the scoring module and noticed the risk score calculation has a floor at 50 that makes it impossible to distinguish mildly risky code from extremely dangerous code.

In backend/app/code/eval/scoring.py, the _score_static method caps the risk penalty at 50 and floors the score at 50:

risk_penalty = min(len(result.risks) * 10, 50)
scores["risks"] = max(100.0 - risk_penalty, 50.0)

This means code with 1 risk (score 50, penalty capped at 50) and code with 50 risks (score 50, same) get the same risk score. The penalty cap and the floor work together to compress all risk levels into a single value.

The fix removes the floor and raises the penalty cap so the full 0-100 range is used:

risk_penalty = min(len(result.risks) * 10, 100)
scores["risks"] = max(100.0 - risk_penalty, 0.0)

Now the scores are meaningful:

  • 0 risks: 100
  • 1 risk: 90
  • 3 risks: 70
  • 5 risks: 50
  • 10+ risks: 0

I added tests in backend/tests/test_pr_13_scoring_risk_floor.py that verify all these cases, including an explicit assertion that 1 risk and 50 risks produce different scores. All 6 tests pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant