ASN Karma

ASN Karma is a Go pipeline for building ASN-level risk datasets from observed BlackRoute evidence. It aggregates hostile IP/CIDR records by autonomous system, scores abuse exposure with an auditable rule set, and emits release artifacts for security analytics, fraud/risk enrichment, traffic policy, and network operations.

---

Latest Release

Fresh dataset artifacts are published by the scheduled build. The links below point at the latest GitHub Release assets.

Last dataset build: 2026-08-23T05:12:48Z

Open latest GitHub release

Artifact	Download	Description
index.json	download	Machine-readable release manifest
asn-risk.jsonl	download	Primary JSONL risk dataset
asn-changes.jsonl	download	ASN delta feed since previous build
asn-summary.csv	download	CSV summary for review and reporting
asn-evidence-table.md	download	Markdown table of top ASN evidence counts
asn-history.jsonl.gz	download	Compressed history state used by the next scheduled build
asn-profiles.tar.gz	download	Per-ASN JSON profiles
source-impact.csv	download	Source contribution breakdown
country-risk.csv	download	Country-level operational rollup
high-risk-asn-critical.txt	download	Critical ASN tier
high-risk-asn-high.txt	download	High ASN tier
high-risk-asn-watch.txt	download	Watch ASN tier
high-risk-asn-prefixes-critical.txt	download	Derived critical ASN announced prefixes
high-risk-asn-prefixes-high.txt	download	Derived high ASN announced prefixes
high-risk-asn-prefixes-watch.txt	download	Derived watch ASN announced prefixes
report.md	download	Markdown dataset report
release-notes.md	download	Release summary and top ASN table
run_stats.json	download	Build metadata and tier counts
checksums.txt	download	SHA256 checksums for release artifacts

Overview

ASN Karma consumes BlackRoute JSONL records and produces an ASN risk layer designed for operational use. The output is intentionally explainable: each ASN record includes score, tier, observed record counts, source diversity, top threat labels, and build metadata.

The project treats ASN expansion as derived intelligence. Source evidence comes from observed IP/CIDR records only; generated ASN prefix lists are output artifacts, not feedback into the evidence stream.

System Behavior

BlackRoute JSONL
  -> parse observed IP/CIDR evidence
  -> enrich records without ASN via Team Cymru bulk whois
  -> aggregate records by ASN
  -> compute source diversity and threat label distribution
  -> apply scoring policy from configs/scoring.json
  -> write JSONL, CSV, TXT tiers, and run statistics

Stage	Responsibility	Current implementation
Ingest	Read BlackRoute-style JSONL with tolerant field mapping	internal/blackroute
Enrich	Map observed IP/CIDR records to ASN, country, and routed prefix	internal/enrich
Model	Normalize observed records and aggregate by ASN	internal/model
Scoring	Apply deterministic score and tier policy	internal/scoring
Output	Emit release artifacts for machines and operators	internal/output
Automation	Build and publish artifacts from GitHub Actions	.github/workflows/build.yml

Features

• Go CLI with no runtime service dependency.
• Team Cymru bulk whois enrichment for upstream records without ASN metadata.
• Deterministic ASN scoring from local configuration.
• JSONL primary output for downstream data pipelines.
• CSV summary for analyst workflows.
• Text tier files for infrastructure policy integration.
• 7/30/90 day history signals for persistence and trend.
• Confidence scoring alongside risk scoring.
• Per-ASN profile archive and derived announced-prefix artifacts.
• SHA256 checksums for release artifacts.
• GitHub Actions workflow for scheduled dataset builds.
• Explicit expanded_prefixes_are_evidence: false field in risk records.
• Local smoke-test fixture under data/blackroute.example.jsonl.

Quick Start

go test ./...
go run ./cmd/asn-karma \
  -input data/blackroute.example.jsonl \
  -out release \
  -readme README.md

The command writes release artifacts into release/.

release/
  index.json
  asn-risk.jsonl
  asn-changes.jsonl
  asn-summary.csv
  asn-evidence-table.md
  asn-profiles.tar.gz
  source-impact.csv
  country-risk.csv
  high-risk-asn-critical.txt
  high-risk-asn-high.txt
  high-risk-asn-watch.txt
  high-risk-asn-prefixes-critical.txt
  high-risk-asn-prefixes-high.txt
  high-risk-asn-prefixes-watch.txt
  report.md
  release-notes.md
  run_stats.json
  checksums.txt

Installation

From Source

git clone https://github.com/ipanalytics/ASN-Karma.git
cd ASN-Karma
go build -o bin/asn-karma ./cmd/asn-karma

Requirements

Component	Version
Go	1.22 or newer
Input dataset	BlackRoute JSONL
Runtime	Linux, macOS, or containerized CI

Usage

Run against a local BlackRoute export:

asn-karma \
  -input data/blackroute.jsonl \
  -config configs/scoring.json \
  -out release

ASN enrichment is enabled by default. For offline parser tests against data that already contains ASN fields:

asn-karma \
  -input data/blackroute.example.jsonl \
  -out release \
  -asn-enrich=false

Use a fixed build timestamp for reproducible test output:

asn-karma \
  -input data/blackroute.example.jsonl \
  -out /tmp/asn-karma-release \
  -built-at 2026-06-15T00:00:00Z

Run directly with Go:

go run ./cmd/asn-karma -input data/blackroute.jsonl -out release

Outputs

Artifact	Format	Purpose
index.json	JSON	Machine-readable release manifest with sizes and SHA256 hashes
asn-risk.jsonl	JSONL	Primary machine-readable ASN risk dataset
asn-changes.jsonl	JSONL	Delta feed since previous build
asn-summary.csv	CSV	Compact review and reporting table
asn-evidence-table.md	Markdown	Top ASN evidence table used by README and release notes
asn-profiles.tar.gz	tar.gz	Per-ASN JSON profiles with risk, history, confidence, and derived prefixes
source-impact.csv	CSV	Source contribution and ASN impact summary
country-risk.csv	CSV	Country-level operational rollup
high-risk-asn-critical.txt	TXT	Strict action tier
high-risk-asn-high.txt	TXT	Challenge or rate-limit tier
high-risk-asn-watch.txt	TXT	Enrichment and logging tier
high-risk-asn-prefixes-critical.txt	TXT	Derived announced prefixes for critical ASN tier
high-risk-asn-prefixes-high.txt	TXT	Derived announced prefixes for high ASN tier
high-risk-asn-prefixes-watch.txt	TXT	Derived announced prefixes for watch ASN tier
report.md	Markdown	Rendered release report with deltas, countries, and source impact
release-notes.md	Markdown	GitHub Release body with run summary and top ASN table
run_stats.json	JSON	Build metadata and tier counts
checksums.txt	TXT	SHA256 checksums for release artifacts

Changes Since Previous Build

The scheduled build updates this table from asn-changes.jsonl. It shows the largest ASN-level deltas compared with the previous persisted history snapshot.

Last updated: 2026-08-23T05:12:48Z

ASN	Name	Country	Change	Previous	Current	Evidence Delta
AS2518	BIGLOBE - BIGLOBE Inc., JP	JP	evidence_increased	356	2901	+2545
AS203020	HostRoyale - HostRoyale Technologies Pvt Ltd, IN	US	evidence_increased	16177	18472	+2295
AS14061	DIGITALOCEAN-ASN - DigitalOcean, LLC, US	US	evidence_increased	174200	176479	+2279
AS31898	ORACLE-BMC-31898 - Oracle Corporation, US	US	evidence_increased	33324	34690	+1366
AS2914	NTT-DATA-2914 - NTT America, Inc., US	US	evidence_increased	8238	9121	+883
AS24282	KIR - KAGOYA JAPAN Inc., JP	JP	risk_level_changed	19	589	+570
AS45090	TENCENT-NET-AP - Shenzhen Tencent Computer Systems Company Limited, CN	CN	evidence_increased	37500	37947	+447
AS200019	AlexHost - ALEXHOST SRL, MD	RU	evidence_increased	2348	2700	+352
AS4134	CHINANET-BACKBONE - No.31,Jin-rong Street, CN	CN	evidence_decreased	155663	155330	-333
AS216472	HS-SYR - High Speed For Internet Services L.L.C, SY	TR	evidence_increased	617	902	+285
AS29066	VELIANET-AS - velia.net Internetdienste GmbH, DE	DE	evidence_increased	1693	1972	+279
AS398101	GO-DADDY-COM-LLC - GoDaddy.com, LLC, US	US	evidence_increased	1508	1753	+245
AS140292	CHINATELECOM-JIANGSU-SUZHOU-5G-NETWORK - CHINATELECOM Jiangsu province Suzhou 5G network, CN	CN	evidence_increased	6354	6584	+230
AS2519	VECTANT - ARTERIA Networks Corporation, JP	JP	evidence_increased	1368	1566	+198
AS265919	AS265919 - BB Host LTDA, BR	BR	evidence_increased	46	237	+191
AS30058	FDCSERVERS - FDCservers.net, US	US	evidence_increased	3271	3446	+175
AS7684	SAKURA-A - SAKURA Internet Inc., JP	JP	evidence_increased	289	456	+167
AS8075	MICROSOFT-CORP-MSN-AS-BLOCK - Microsoft Corporation, US	US	evidence_increased	50725	50888	+163
AS264381	AS264381 - VIBE PROVEDOR DE INTERNET LTDA, BR	BR	risk_level_changed	30	189	+159
AS2907	SINET-AS - Research Organization of Information and Systems, National Institute of Informatics, JP	JP	evidence_increased	138	296	+158
AS213702	QWINS-LTD - QWINS LTD, GB	CY	evidence_increased	564	717	+153
AS52485	AS52485 - networksdelmanana.com, HN	HN	evidence_increased	339	471	+132
AS152194	CTGSERVERLIMITED-AS-AP - CTG Server Limited, HK	SG	evidence_increased	883	1006	+123
AS154177	LIGHT4-AS-AP - LIGHT NODE LIMITED, HK	US	evidence_increased	1818	1932	+114
AS262744	AS262744 - ICENET TELECOMUNICACOES LTDA - ME, BR	BR	risk_level_changed	92	203	+111

Risk Record

When ASN records are available, asn-risk.jsonl contains one JSON object per ASN:

{
  "asn": 64500,
  "asn_name": "Example Hosting",
  "country": "US",
  "risk_score": 39,
  "risk_level": "low",
  "confidence_score": 40,
  "confidence": "low",
  "recommended_action": "no_action",
  "observed_records": 2,
  "unique_observed_cidrs": 2,
  "source_count": 2,
  "source_diversity": 2,
  "top_threat_labels": {
    "c2_ioc": 1,
    "malware_host_active": 1,
    "network_scan_or_abuse": 1
  },
  "evidence_window_days": 30,
  "persistence_days_30d": 1,
  "active_days_7d": 1,
  "active_days_30d": 1,
  "active_days_90d": 1,
  "first_seen": "2026-06-15",
  "last_seen": "2026-06-15",
  "trend": "new",
  "evidence_delta_1d": 2,
  "expanded_prefix_count": 0,
  "expanded_prefixes_are_evidence": false,
  "large_cloud": false,
  "watchlist": false,
  "built_at": "2026-06-15T00:00:00Z"
}

If a build is explicitly allowed to complete with zero ASN records, asn-risk.jsonl contains a single build_status JSON object explaining that no ASN records were produced. Scheduled production builds do not use -allow-empty; an empty ASN dataset fails before release publication.

Data Contracts

Schemas are kept under docs/schema/:

Schema	Covers
docs/schema/asn-risk.schema.json	asn-risk.jsonl records
docs/schema/asn-changes.schema.json	asn-changes.jsonl records
docs/schema/index.schema.json	index.json release manifest
docs/schema/run-stats.schema.json	run_stats.json

Integration Examples

Operational examples are available under examples/:

File	Target
examples/cloudflare-waf.md	Cloudflare WAF ASN policy
examples/nginx-map.md	NGINX enrichment map pattern
examples/opnsense-alias.md	OPNsense firewall aliases
examples/splunk-lookup.md	Splunk CSV lookup
examples/clickhouse-ingest.sql	ClickHouse JSONL ingestion

Scoring Policy

Scoring is configured in configs/scoring.json.

Signal	Role
Source diversity	Rewards corroboration across feeds
Threat severity	Weights labels such as C2, malware hosting, spam, and scanning
Recent activity	Captures observed volume in the build window
Abuse density proxy	Gives smaller concentrated abuse surfaces weight
Cybercrime prefix bonus	Adds weight for severe infrastructure labels
Large cloud penalty	Reduces broad-provider overclassification
Allowlist penalty	Suppresses known infrastructure where appropriate
Watchlist flag	Adds context without turning context into evidence

Risk tiers are emitted as critical, high, watch, or low.

Operational Notes

• Treat asn-risk.jsonl as the canonical artifact.
• Use TXT tier files as policy inputs only after local validation.
• Keep scoring changes reviewable; policy drift should be visible in config diffs.
• Do not feed derived ASN prefix expansion back into source evidence.
• Verify downloaded artifacts with checksums.txt.
• ASNs marked review_required=true are large cloud, backbone, CDN, or major hosting networks; they are capped to review/watch policy unless local telemetry supports enforcement.
• Large cloud and CDN networks need provider-aware handling in production policy.
• Run builds on a schedule after the upstream BlackRoute release has completed.

Project Scope

ASN Karma focuses on ASN-level aggregation, scoring, and artifact generation. It is designed to sit between raw IP reputation feeds and downstream enforcement, enrichment, or analytics systems.

Planned extension points include:

• Optional release signing.
• GitHub Pages dataset index.

Use Cases

• Enrich SIEM, SOAR, and data lake events with ASN risk context.
• Feed WAF, CDN, and edge policy with conservative ASN tiers.
• Track abuse concentration across hosting providers and network operators.
• Support fraud and risk pipelines with infrastructure-level features.
• Build daily ASN exposure reports for security operations.

Limitations

ASN-level scoring is coarse by design. It should be combined with local telemetry, asset context, customer impact analysis, and provider-specific knowledge before enforcement.

Team Cymru enrichment uses current BGP attribution. For historical analysis, run the scorer against input that already carries time-appropriate ASN metadata.

Directory Structure

.
├── cmd/asn-karma/              # CLI entrypoint
├── configs/                    # scoring and policy configuration
├── data/                       # local fixtures and input data
├── data/history/               # persisted daily ASN history state
├── docs/schema/                # JSON schema contracts
├── examples/                   # integration examples
├── internal/blackroute/         # BlackRoute JSONL ingest
├── internal/enrich/             # ASN enrichment adapters
├── internal/model/              # normalized records and aggregation
├── internal/output/             # release artifact writers
├── internal/scoring/            # scoring policy implementation
├── release/                     # generated artifacts
├── site/                        # README and documentation assets
└── .github/workflows/           # scheduled build automation

Deployment

The repository includes a scheduled GitHub Actions workflow:

on:
  schedule:
    - cron: "47 4 * * *"
  workflow_dispatch:

The workflow tests the Go code, downloads the latest BlackRoute JSONL release, builds ASN Karma artifacts, updates the README evidence table, and publishes the generated files as a GitHub release.

For self-hosted deployments, run the CLI from cron, systemd timers, Kubernetes CronJobs, or an existing data orchestration system. The process is batch-oriented and writes immutable output files for each run.

Example Kubernetes CronJob command

command:
  - /usr/local/bin/asn-karma
  - -input
  - /data/blackroute.jsonl
  - -config
  - /config/scoring.json
  - -out
  - /release

License

MIT license.

Disclaimer

ASN Karma provides infrastructure risk signals derived from public abuse evidence. Operators are responsible for applying local policy, validation, and impact controls before enforcement.