jsonstat-go

A Go-native library for the JSON-stat 2.0 format — decode, traverse, transform and encode statistical cubes with a small, idiomatic API that depends only on the Go standard library.

Features

• Decode any 2.0 response class (dataset, collection, dimension) and the pre-2.0 bundle container, with transparent normalisation of the polymorphic parts of the format:
  • dense ("value": [1, 2, null, …]) and sparse ("value": {"0": 1, "2": 3}) value forms,
  • all three status forms (single string, array, sparse object).
• Traverse cubes by flat row-major index, by per-dimension category coordinates, by map[dimID]catID, or lazily with a Go 1.23 iter.Seq2 iterator.
• Transform a cube into four tabular shapes (array of arrays, array of objects, columnar object, Google DataTable) with pivoting, dropping, status, and metadata options.
• Unflatten a cube with a callback that receives each cell's coordinates and value/status — the building block the higher-level transform is built on.
• Encode a model back to canonical JSON-stat bytes with stable property ordering and round-trip fidelity on the bundled fixtures.
• Typed errors with structured context (operation, dimension id, category index, flat index) recoverable through errors.As.

Installation

go get github.com/jsonstat/go

The import path is github.com/jsonstat/go. The module targets Go 1.23 (for the range-over-func iterator used by Dataset.Cells).

Decode

Decode reads from any io.Reader; DecodeBytes is the byte-slice shortcut. A document holds zero, one or many datasets — use Document.SingleDataset for the common case or Document.Datasets for collections and pre-2.0 bundles.

package main

import (
    "fmt"
    "os"

    "github.com/jsonstat/go"
)

func main() {
    f, err := os.Open("oecd.json")
    if err != nil {
        panic(err)
    }
    defer f.Close()

    doc, err := jsonstat.Decode(f)
    if err != nil {
        panic(err)
    }

    ds, err := doc.SingleDataset()
    if err != nil {
        panic(err)
    }

    fmt.Println(ds.Label)                  // "Unemployment rate in the OECD countries 2003-2014"
    fmt.Println(ds.DimensionIDs())         // [concept area year]
    fmt.Println(ds.SizeSlice())            // [1 36 12]
    fmt.Println("cells:", ds.N())          // cells: 432
}

Traverse

Cells are addressed three ways, all returning a Cell that carries its coordinates, value and status:

// By flat row-major index ("what does not change, first"; last dimension
// varies fastest).
c, err := ds.CellByFlat(0)

// By per-dimension category coordinates (one int per dimension, in ID order).
c, err = ds.CellByCoord(0, 0, 0)

// By dimension-ID -> category-ID map.
c, err = ds.CellByLabel(map[string]string{
    "concept": "UNR",
    "area":    "AU",
    "year":    "2003",
})

fmt.Printf("value=%v status=%v coords=%v missing=%t\n",
    c.Value, c.Status, c.Coords, c.Missing)

For bulk iteration prefer the lazy Cells iterator, which allocates no per-cell slice until the callback is reached:

for c, err := range ds.Cells() {
    if err != nil {
        return err
    }
    if c.Missing {
        continue
    }
    total += c.Value
}

Walk is the callback equivalent for pre-1.23 call sites, and DimensionsByRole lets you pick dimensions by JSON-stat role (time, geo, metric):

time, _ := ds.DimensionByRole(jsonstat.RoleTime)
fmt.Println(time.ID, time.Size) // year 12

Transform

Dataset.Transform converts a cube into one of four tabular shapes, configured with functional options:

// 1. Array of arrays (the default). The first row is the header; its column
//    names follow the `field` setting, while category values follow `content`.
//    The array type defaults to field=label, so pass WithField(FieldID) for
//    ID-style column names; pair it with WithContent(ContentID) to also render
//    category IDs instead of labels.
rows, _ := ds.Transform(
    jsonstat.WithStatus(true),
    jsonstat.WithField(jsonstat.FieldID),
    jsonstat.WithContent(jsonstat.ContentID),
)
header := rows.([][]any)[0]
// ["concept", "area", "year", "status", "value"]
dataRow := rows.([][]any)[1]
// ["UNR", "AU", "2003", "", 5.943826289]

// 2. Array of objects (one map per cell). Optional `by` pivots a dimension's
//    categories into columns; `drop` removes single-category dimensions;
//    `prefix` namespaces pivoted column keys.
out, _ := ds.Transform(
    jsonstat.WithTransformType(jsonstat.TransformArrObj),
    jsonstat.WithBy("year"),
    jsonstat.WithDrop("concept"),
    jsonstat.WithPrefix("y_"),
)
records := out.([]map[string]any)

// 3. Columnar object (one parallel array per column).
out, _ = ds.Transform(
    jsonstat.WithTransformType(jsonstat.TransformObjArr),
    jsonstat.WithStatus(true),
)
cols := out.(map[string][]any)
// cols["area"]  == ["AU", "AU", …]
// cols["value"] == [5.943826289, 5.39663128, …]

// 4. Google DataTable ({cols, rows}), suitable for Google Charts.
out, _ = ds.Transform(jsonstat.WithTransformType(jsonstat.TransformObject))
dt := out.(*jsonstat.DataTable)

When meta is true the result is wrapped in a TableWithMeta object carrying the dataset's label, source, updated timestamp, the applied options, and per-dimension category metadata:

out, _ := ds.Transform(
    jsonstat.WithTransformType(jsonstat.TransformArrObj),
    jsonstat.WithStatus(true),
    jsonstat.WithBy("sex"),
    jsonstat.WithDrop("country", "year"),
    jsonstat.WithMeta(true),
)
wrapped := out.(*jsonstat.TableWithMeta)
fmt.Println(wrapped.Meta.Type, wrapped.Meta.By, wrapped.Meta.Drop)

The available options are:

Option	Purpose
[WithTransformType]	Output shape: TransformArray (default), TransformArrObj, TransformObjArr, TransformObject.
[WithStatus]	Include the status column. Ignored when WithBy is in effect.
[WithContent]	Category values are IDs (ContentID) or labels (ContentLabel, default).
[WithField]	Column keys are IDs (FieldID, default) or labels (FieldLabel; the array type defaults to label).
[WithValueLabel]	Rename the value column (default "Value").
[WithStatusLabel]	Rename the status column (default "Status).
[WithMeta]	Wrap the result in TableWithMeta. Not valid with TransformObject.
[WithBy]	Pivot the named dimension's categories into columns. arrobj/objarr only.
[WithPrefix]	Prefix for pivoted column keys. Only honoured when WithBy is active.
[WithDrop]	Omit single-category dimensions from the output.
[WithComma]	Render numeric values as comma-decimal strings. Not valid with TransformObject.

Unflatten

For anything the option-based transform can't express, Dataset.Unflatten hands every cell to a callback with its coordinates, value/status pair and the row being built. Returning nil drops the cell from the output.

// Collect only cells whose value exceeds a threshold, projecting each to a
// custom record type.
type peak struct {
    Area string  `json:"area"`
    Year string  `json:"year"`
    Rate float64 `json:"rate"`
}

rows, err := ds.Unflatten(func(coords jsonstat.Coordinates, dp jsonstat.Datapoint, n int, row []any) any {
    if dp.Missing || dp.Value < 10 {
        return nil
    }
    return peak{Area: coords["area"], Year: coords["year"], Rate: dp.Value}
})
// rows is []any; each non-nil element is a peak.

Encode

MarshalDataset, MarshalDocument and Encode (which writes to a *bytes.Buffer) emit canonical JSON-stat bytes with stable property ordering. Dataset.MarshalJSON and Document.MarshalJSON make the types usable directly from encoding/json:

// Round-trip: decode -> mutate -> re-encode.
doc, _ := jsonstat.DecodeBytes(src)
ds, _ := doc.SingleDataset()
// …traverse or transform ds…

out, err := jsonstat.MarshalDocument(doc)
if err != nil {
    panic(err)
}
_ = os.WriteFile("oecd.out.json", out, 0o644)

Fixtures

The testdata/ directory bundles the canonical JSON-stat sample files (canada.json, oecd.json, order.json, sparse.json, status-array.json, us-unr.json, …) used as round-trip fixtures. They are the easiest way to explore the model:

doc, _ := jsonstat.DecodeBytes(mustRead("testdata/canada.json"))
ds, _  := doc.SingleDataset()
// ds.DimensionIDs() == [country year age concept sex]
// ds.SizeSlice()   == [1 1 20 2 3]
// ds.N()           == 120

License

Apache License 2.0.