Skip to content

azure-setup.md

Latest commit

 

History

History
315 lines (232 loc) · 9.51 KB

azure-setup.md

File metadata and controls

315 lines (232 loc) · 9.51 KB
Raw

Azure Setup Guide

This guide gets you from zero to a running OpenShield scan in under 20 minutes using a free Azure account.

Step 1 — Create a Free Azure Account

  1. Go to azure.microsoft.com/free and click Start free.
  2. Sign in with a Microsoft account (or create one).
  3. Complete the sign-up — you will receive $200 in free credits and access to free-tier services.
  4. After sign-up, navigate to the Azure Portal.

Step 2 — Get Your Subscription ID

  1. In the Azure Portal, search for Subscriptions in the top search bar.
  2. Click on your subscription name.
  3. Copy the Subscription ID (a UUID like xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx).

You will need this value for AZURE_SUBSCRIPTION_ID in your .env file.

Step 3 — Create a Service Principal with Reader Role

OpenShield only needs read access to scan your subscription. Use the Azure CLI:

# Install Azure CLI if you haven't already
# https://learn.microsoft.com/en-us/cli/azure/install-azure-cli

# Login
az login

# Create the service principal with Reader role
az ad sp create-for-rbac \
  --name "openshield-scanner" \
  --role Reader \
  --scopes /subscriptions/<YOUR_SUBSCRIPTION_ID> \
  --output json

This command outputs JSON like:

{
  "appId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "displayName": "openshield-scanner",
  "password": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "tenant": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}

Map these values:

  • appId → AZURE_CLIENT_ID
  • password → AZURE_CLIENT_SECRET
  • tenant → AZURE_TENANT_ID

Important: The password is only shown once. Copy it immediately.

Step 4 — Grant Additional Read Permissions (Optional)

For the Conditional Access MFA rule (AZ-IDN-002), the service principal needs the Policy.Read.All Microsoft Graph API permission:

# Get the service principal object ID
SP_OBJECT_ID=$(az ad sp show --id <YOUR_APP_ID> --query id --output tsv)

# Get the Microsoft Graph service principal object ID
GRAPH_SP_ID=$(az ad sp list \
  --filter "appId eq '00000003-0000-0000-c000-000000000000'" \
  --query "[0].id" \
  --output tsv)

# Grant Policy.Read.All application permission
# This requires a Global Administrator to consent
az rest \
  --method POST \
  --uri "https://graph.microsoft.com/v1.0/servicePrincipals/$SP_OBJECT_ID/appRoleAssignments" \
  --body '{
    "principalId": "'$SP_OBJECT_ID'",
    "resourceId": "'$GRAPH_SP_ID'",
    "appRoleId": "246dd0d5-5bd0-4def-940b-0421030a5b68"
  }'

If you skip this step, AZ-IDN-002 will produce a finding by default (it cannot verify MFA status without Graph access).

Step 5 — Configure Your .env File

Create a .env file and fill in your values:

touch .env

Edit .env:

AZURE_SUBSCRIPTION_ID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
AZURE_CLIENT_ID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
AZURE_CLIENT_SECRET=your-client-secret-from-step-3
AZURE_TENANT_ID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
DATABASE_URL=postgresql://openshield:openshield@localhost:5432/openshield
# Generate with: python -c "import secrets; print(secrets.token_urlsafe(32))"
# Must be at least 32 characters. The app refuses to start in production without it.
JWT_SECRET=your-random-secret-at-least-32-chars
OPENSHIELD_ENV=production
SENTINEL_WORKSPACE_ID=
SENTINEL_SHARED_KEY=
SENTINEL_LOG_TYPE=OpenShieldFindings

Step 6 — Start a Local PostgreSQL Database

# Option A: Docker (easiest)
docker run --name openshield-db \
  -e POSTGRES_USER=openshield \
  -e POSTGRES_PASSWORD=openshield \
  -e POSTGRES_DB=openshield \
  -p 5432:5432 \
  -d postgres:15

# Option B: Homebrew (macOS)
brew install postgresql@15
brew services start postgresql@15
createdb openshield

Create the database schema with Alembic before running a scan or starting the API:

set -a; source .env; set +a
alembic upgrade head

See Database Migrations before onboarding an existing production database.

Step 7 — Run Your First Scan

# From the openshield/ directory
cd openshield

# Install dependencies
pip install -r requirements.txt

# Run the scanner directly
python -c "
from dotenv import load_dotenv; load_dotenv()
import json, os
from scanner.engine import ScanEngine
engine = ScanEngine(os.environ['AZURE_SUBSCRIPTION_ID'])
result = engine.run_scan()
print(json.dumps(result, indent=2))
"

Or trigger via the API:

# Start the API server
FLASK_APP=api/app.py flask run

# Trigger a scan
curl -X POST http://localhost:5000/api/scans/trigger \
  -H "Authorization: Bearer <YOUR_JWT>" \
  -H "Content-Type: application/json" \
  -d '{"subscription_id": "your-subscription-id"}'

Compliance posture is available through /api/compliance/cis, /api/compliance/nist, /api/compliance/iso27001, and /api/compliance/soc2.

Azure App Service Deployment

Note: The Flask API is deployed on Render (render.com) rather than Azure App Service F1. Azure App Service F1 sleeps after 20 minutes of inactivity and has a 60 CPU minute per day limit which is not suitable for demo use. See the Render deployment section below for setup instructions.

Render Deployment (Recommended for API)

Render is recommended for hosting the OpenShield API. Use the Starter instance or higher for always-on availability.

Steps

  1. Create a free account at render.com

  2. Click New → Web Service

  3. Connect your GitHub account and select openshield-org/openshield

  4. Configure:

    • Name: openshield-api
    • Branch: main
    • Build Command: pip install -r requirements.txt
    • Start Command: ./startup.sh
    • Instance Type: Free

  5. Add environment variables under Environment:

AZURE_SUBSCRIPTION_ID=your-subscription-id
AZURE_CLIENT_ID=your-client-id
AZURE_CLIENT_SECRET=your-client-secret
AZURE_TENANT_ID=your-tenant-id
DATABASE_URL=your-postgresql-connection-string
JWT_SECRET=your-secret-key

  1. Create a PostgreSQL database:

    • Click New → PostgreSQL
    • Name: openshield-db
    • Copy the Internal Database URL into DATABASE_URL above

  2. Deploy — Render will build and deploy automatically

  3. Your API will be live at: https://openshield-api.onrender.com

Instance Notes

  • Starter instance or higher provides always-on availability
  • Free-tier instances spin down after inactivity — not recommended for production use

Step 8 — Activate the Microsoft Sentinel 90-Day Trial (Optional)

Microsoft Sentinel includes a 90-day free trial for new Log Analytics workspaces.

  1. In the Azure Portal, search for Microsoft Sentinel.
  2. Click Create Microsoft Sentinel.
  3. Click Create a new workspace and fill in:
    • Workspace name: openshield-logs
    • Region: choose the same region as your resources
  4. Click Add Microsoft Sentinel — the 90-day trial activates automatically.
  5. Copy the Workspace ID from the workspace Overview page.
  6. Copy a shared key from Agents or with the Azure CLI:
az monitor log-analytics workspace get-shared-keys \
  --resource-group <resource-group> \
  --workspace-name <workspace-name> \
  --query primarySharedKey \
  --output tsv
  1. Add these values to your .env:
SENTINEL_WORKSPACE_ID=<workspace-id>
SENTINEL_SHARED_KEY=<primary-shared-key>
SENTINEL_LOG_TYPE=OpenShieldFindings

sentinel/ingest.py reads a findings JSON file, normalises each finding, signs the request with SENTINEL_SHARED_KEY, and sends records to the OpenShieldFindings_CL custom log table.

Cost after trial: ~$2.76/GB ingested. For a small subscription with few findings, this is negligible.

Step 9 — Create a Log Analytics Workspace (for SQL Auditing)

The AZ-DB-002 remediation playbook writes SQL audit logs to a storage account. To route them to Log Analytics instead:

  1. Go to your SQL server in the portal.
  2. Under Security, click Auditing.
  3. Set Auditing to ON.
  4. Check Log Analytics and select your openshield-logs workspace.
  5. Click Save.

Step 10 — Configure Azure DevOps Pipeline Scanning (Optional)

AZ-SC-007 and AZ-SC-008 check Azure DevOps pipeline service connections for subscription-wide sharing and password-based authentication. Azure DevOps is a separate system from Azure Resource Manager, so it needs two additional environment variables. Both must be set or neither rule will produce findings — this is treated as "not applicable," not an error.

AZURE_DEVOPS_ORG_URL=https://dev.azure.com/your-org
AZURE_DEVOPS_PROJECT=your-project-name

The scanner reuses the same service principal configured in Step 3, requesting a token scoped to Azure DevOps' well-known resource ID (499b84ac-1321-427f-aa17-267ca6975798). Grant that service principal at least Reader access to the target Azure DevOps project's service connections (Project Settings > Service connections > Security).

Troubleshooting

Problem Fix
DefaultAzureCredential fails Run az login in the terminal, or verify env vars are set
AZURE_CLIENT_SECRET rejected The secret may have expired — rotate it with az ad sp credential reset
psycopg2.OperationalError Check your PostgreSQL container is running and DATABASE_URL is correct
Empty findings Verify the service principal has Reader role on the subscription
AZ-IDN-002 always fires The service principal needs Policy.Read.All Graph permission — see Step 4
AZ-SC-007/008 never fire Confirm AZURE_DEVOPS_ORG_URL and AZURE_DEVOPS_PROJECT are both set — see Step 10