BE-797: Add S/M/L identifiers for scale factor

libs/sap-mock-data/README.md

@@ -1,18 +1,18 @@
 # SAP Mock Data
 
 `sap-mock-data` generates deterministic, interconnected SAP-style master and
-transaction data. The pandas generation engine is available through one Python
-API and is shared by the CLI, local notebooks, and Databricks notebooks.
+transaction data. The `generate_dataset` function generates it. The CLI,
+local notebooks, and Databricks notebooks all call that function.
 
-Generation has two layers. Pandas creates and mutates tables. A `TableStore`
-decides where those tables live. Delta Lake is the default local store. The
-Databricks notebook supplies a Unity Catalog adapter.
+Pandas creates and mutates tables. A `TableStore` persists them. Delta Lake
+is the default local store. The Databricks notebook supplies a Unity Catalog
+adapter.
 
 ## Development environment
 
-The library supports Python 3.11 through 3.13 and does not require Nix. Nix is
-an optional way to provision the Python and system tools; Python dependencies
-come from `pyproject.toml` and are locked in `uv.lock`.
+The library supports Python 3.11 through 3.13. Nix is an optional way to
+provision the Python and system tools. Python dependencies come from
+`pyproject.toml` and are locked in `uv.lock`.
 
 ### Without Nix
 
@@ -28,8 +28,7 @@ uv build
 The checked-in `.python-version` selects Python 3.13. By default, uv downloads
 that runtime when it is not already installed.
 
-The package also works with standard Python tooling when the lock file is not
-needed:
+The package also installs with venv and pip, without the lock file:
 
 ```console
 python -m venv .venv
@@ -46,13 +45,13 @@ uv sync --frozen --extra spark --group notebook
 uv run jupyter lab
 ```
 
-Local Spark additionally requires a Java 17 or newer JDK available on `PATH`
-with `JAVA_HOME` configured. Databricks supplies its own Spark and Java runtime.
+Local Spark requires a Java 17 or newer JDK on `PATH` with `JAVA_HOME` set.
+Databricks supplies its own Spark and Java runtime.
 
 ### With Nix
 
-The default Nix shell provides pinned Python and uv executables. It uses the
-same uv-managed Python dependency workflow as the non-Nix setup:
+The default Nix shell provides pinned Python and uv executables. The uv
+commands are the same as without Nix:
 
 ```console
 nix develop
@@ -69,8 +68,8 @@ uv sync --frozen --extra spark --group notebook
 uv run jupyter lab
 ```
 
-PySpark imports are confined to notebooks, so core library and CLI users do not
-need Java or Spark in either environment.
+Only the notebooks import PySpark. The library and CLI run without Java or
+Spark in either environment.
 
 ## Library API
 
@@ -93,9 +92,9 @@ print(result.table_count, result.row_counts)
 Use a new or isolated warehouse path for each run. Generation overwrites its
 tables and preserves unrelated tables in an existing store.
 
-For embedding or quick checks, use `MemoryTableStore`. The generation API
-accepts any pandas-oriented `TableStore`, including the Databricks adapter shown
-in `notebooks/databricks/Generate SAP Mock Data.py`.
+For embedding or quick checks, use `MemoryTableStore`. `generate_dataset`
+accepts any `TableStore` implementation, including the Databricks adapter in
+`notebooks/databricks/Generate SAP Mock Data.py`.
 
 Scenario selection accepts `demo`, `none`, `all`, a comma-separated string, or
 a sequence of IDs. Explicit scenarios need configuration strings:
@@ -107,6 +106,32 @@ GenerationConfig(
 )
 ```
 
+### Dataset size
+
+`scale_factor` is a positive number or one of the size identifiers `S`,
+`M`, `L`, `XL`. For a size identifier, `GenerationConfig` samples each count
+from the ranges below, seeded by `random_seed`. The same seed produces the
+same dataset, apart from the `INJECTED_AT` timestamp in `scenario_metadata`.
+A number multiplies the default order, customer, material, vendor, and site
+counts.
+
+| size | products | suppliers | sites   | BOM depth | raw materials | customers | orders      |
+| ---- | -------- | --------- | ------- | --------- | ------------- | --------- | ----------- |
+| S    | 3        | 3-5       | 1-2     | 1         | 6-10          | 4-8       | 200-400     |
+| M    | 10-50    | 20-30     | 4-5     | 1         | 27-42         | 25-40     | 4000-6000   |
+| L    | 100-200  | 50-100    | 20-40   | 1         | 72-112        | 80-120    | 12000-20000 |
+| XL   | 400-800  | 150-300   | 100-120 | 1         | 180-280       | 250-400   | 40000-80000 |
+
+- The products column includes two fixed BOM parent materials, and the raw
+  materials column includes three fixed BOM components.
+- `num_customers`, `num_finished_goods`, `num_raw_materials`, `num_vendors`,
+  `num_sites`, and `num_orders` set their counts directly, whichever form
+  `scale_factor` takes.
+- The first five sites are fixed plants. Further sites are synthesized with
+  ids from 6000 in steps of 10, and every fifth one is a production plant.
+- The demo scenarios target ids that exist at every size.
+- Generation time grows with products times sites. `XL` takes over an hour.
+
 ### Currency
 
 `GenerationConfig.currency` sets the three-letter currency code used by all
@@ -131,5 +156,5 @@ The manifest records schemas, row and null counts, and canonical content hashes.
   `SparkCatalogStore` and calls the same API.
 - `notebooks/databricks/Node Impact Analysis.py` runs the node-impact analysis.
 
-Spark serves as a storage adapter. Generation logic lives under
-`src/sap_mock_data`; notebooks call that package.
+Spark is a storage adapter. Generation logic is in `src/sap_mock_data`.
+Notebooks call that package.

libs/sap-mock-data/notebooks/databricks/Generate SAP Mock Data.py

@@ -89,10 +89,15 @@ def tables(self) -> list[str]:
     dbutils.widgets.get("catalog"),
     dbutils.widgets.get("schema"),
 )
+scale_widget = dbutils.widgets.get("scale_factor")
+try:
+    scale_factor = float(scale_widget)
+except ValueError:
+    scale_factor = scale_widget
 result = generate_dataset(
     GenerationConfig(
         random_seed=int(dbutils.widgets.get("random_seed")),
-        scale_factor=float(dbutils.widgets.get("scale_factor")),
+        scale_factor=scale_factor,
         scenarios=dbutils.widgets.get("scenarios"),
     ),
     store,

libs/sap-mock-data/pipeline/Mock-up SAP ERP data - Masterdata.py

@@ -5,7 +5,6 @@
 import numpy as np
 from faker import Faker
 import random
-import uuid
 from math import radians, sin, cos, sqrt, asin
 from datetime import datetime, timedelta
 import pyspark.sql.functions as F
@@ -532,7 +531,7 @@ def generate_sapapo_tr_data():
             from_info = PLANT_CONFIG[loc_from]
             to_info = PLANT_CONFIG[loc_to]
 
-            trlid = str(uuid.uuid4()).replace('-', '').upper()[:32]
+            trlid = f"{random.getrandbits(128):032X}"
             lane_name = f"{from_info['city']} -> {to_info['city']}"
 
             data.append({

libs/sap-mock-data/pipeline/Mock-up SAP ERP data - Transactions.py

@@ -281,6 +281,7 @@ def generate_sales_orders(finished_goods, all_customers):
     print(f"Generating {NUMBER_OF_ORDERS} Sales Orders...")
     vbak, vbap, vbep = [], [], []
     base_date = datetime.now()
+    india_customer_exists = CUST_INDIA in all_customers
 
     for i in range(NUMBER_OF_ORDERS):
         vbeln = f'{1000000000 + i:010d}'
@@ -290,7 +291,7 @@ def generate_sales_orders(finished_goods, all_customers):
         req_date = (order_date_dt + timedelta(days=7)).strftime('%Y%m%d')
 
         # Scenario: India Customer
-        if i % 100 == 0: kunnr = CUST_INDIA
+        if i % 100 == 0 and india_customer_exists: kunnr = CUST_INDIA
         else: kunnr = random.choice(all_customers)
 
         order_total = 0.0

libs/sap-mock-data/src/sap_mock_data/cli/main.py

@@ -4,15 +4,33 @@
 
 import argparse
 import json
+import math
 from importlib.metadata import version
 from pathlib import Path
 from typing import Sequence
 
 from .. import GenerationConfig, generate_dataset
+from ..config import SIZE_KNOB_RANGES
 from ..storage import DeltaTableStore
 from ..validation import build_manifest, integrity_report
 
 
+def _scale_factor(value: str) -> float | str:
+    try:
+        number = float(value)
+    except ValueError:
+        letter = value.strip().upper()
+        if letter in SIZE_KNOB_RANGES:
+            return letter
+        raise argparse.ArgumentTypeError(
+            f"must be a positive number or one of {', '.join(SIZE_KNOB_RANGES)}; "
+            f"got {value!r}"
+        )
+    if not (number > 0 and math.isfinite(number)):
+        raise argparse.ArgumentTypeError(f"must be a positive finite number; got {value}")
+    return number
+
+
 def _scenario_configs(values: Sequence[str]) -> dict[str, str]:
     configs: dict[str, str] = {}
     for value in values:
@@ -37,11 +55,18 @@ def _parser() -> argparse.ArgumentParser:
     generate = commands.add_parser("generate", help="generate a dataset")
     generate.add_argument("output", type=Path, help="Delta warehouse directory")
     generate.add_argument("--seed", type=int, default=42)
-    generate.add_argument("--scale-factor", type=float, default=1.0)
+    generate.add_argument(
+        "--scale-factor",
+        type=_scale_factor,
+        default=1.0,
+        help="numeric multiplier, or a dataset size: S, M, L, XL",
+    )
     generate.add_argument("--orders", type=int)
     generate.add_argument("--customers", type=int)
     generate.add_argument("--finished-goods", type=int)
     generate.add_argument("--raw-materials", type=int)
+    generate.add_argument("--vendors", type=int)
+    generate.add_argument("--sites", type=int)
     generate.add_argument("--currency", default="EUR")
     generate.add_argument(
         "--scenarios",
@@ -78,25 +103,28 @@ def _write_json(payload: object, output: Path | None) -> None:
 
 
 def main(argv: Sequence[str] | None = None) -> int:
-    args = _parser().parse_args(argv)
+    parser = _parser()
+    args = parser.parse_args(argv)
     if args.command == "generate":
         try:
             scenario_configs = _scenario_configs(args.scenario_config)
-        except argparse.ArgumentTypeError as error:
-            _parser().error(str(error))
-        config = GenerationConfig(
-            random_seed=args.seed,
-            scale_factor=args.scale_factor,
-            num_orders=args.orders,
-            num_customers=args.customers,
-            num_finished_goods=args.finished_goods,
-            num_raw_materials=args.raw_materials,
-            currency=args.currency,
-            scenarios=args.scenarios,
-            scenario_configs=scenario_configs,
-            generate_dirty_data=args.dirty_data,
-            dirty_data_rate=args.dirty_data_rate,
-        )
+            config = GenerationConfig(
+                random_seed=args.seed,
+                scale_factor=args.scale_factor,
+                num_orders=args.orders,
+                num_customers=args.customers,
+                num_finished_goods=args.finished_goods,
+                num_raw_materials=args.raw_materials,
+                num_vendors=args.vendors,
+                num_sites=args.sites,
+                currency=args.currency,
+                scenarios=args.scenarios,
+                scenario_configs=scenario_configs,
+                generate_dirty_data=args.dirty_data,
+                dirty_data_rate=args.dirty_data_rate,
+            )
+        except (argparse.ArgumentTypeError, ValueError) as error:
+            parser.error(str(error))
         store = DeltaTableStore(args.output)
         result = generate_dataset(config, store)
         print(