Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pages/docs/_meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
},
"golbat": "Golbat",
"dragonite": "Dragonite",
"rotom": "Rotom",
"rotom": "RotomNG",
"--": {
"type": "separator",
"title": ""
Expand Down
4 changes: 2 additions & 2 deletions pages/docs/dragonite/config.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -161,8 +161,8 @@ endpoint = "ws://127.0.0.1:7071"

| Key | Default | Description |
| :-- | :-- | :-- |
| `endpoint` | "ws://127.0.0.1:7071" | This is the URL for your Rotom service. |
| `secret` | "" | When Rotom secret configuration is enabled you must pass a secure token so Dragonite is able to communicate. |
| `endpoint` | "ws://127.0.0.1:7071" | This is the URL for your [RotomNG](/docs/rotom) service. It must point at RotomNG's controller listener (`controller_listener.address`, default port 7071). |
| `secret` | "" | When RotomNG's `controller_listener.secret` is set you must pass the same token here so Dragonite is able to communicate. |

## Logging section

Expand Down
2 changes: 1 addition & 1 deletion pages/docs/other/terminology.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Websockets are a standard communication protocol much like web traffic that occu

##### Worker

Workers are added to a pool inside of Rotom which can be allocated by Dragonite to different Areas. Physical device creates 1-many workers which can then handle different scanning tasks.
Workers are added to a pool inside of RotomNG which can be allocated by Dragonite to different Areas. Physical device creates 1-many workers which can then handle different scanning tasks.

##### GMO

Expand Down
3 changes: 2 additions & 1 deletion pages/docs/rotom/_meta.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
{
"index": "Introduction",
"config": "Configuration"
"config": "Configuration",
"migration": "Migrating from Rotom OG"
}
209 changes: 206 additions & 3 deletions pages/docs/rotom/config.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,210 @@
import { Callout } from 'nextra-theme-docs'

# Rotom Configuration
# RotomNG Configuration

<Callout type="info" emoji="🚧">
This page is still under construction
RotomNG is configured with a TOML file. By default the binary reads `configs/rotom-ng.toml`
relative to the working directory. A different path can be passed as the first argument:

```sh
./rotom-ng /path/to/rotom-ng.toml
```

Start from the example file shipped with the repo:

```sh
cp configs/rotom-ng.toml.example configs/rotom-ng.toml
```

Every section is optional and has sensible defaults, so a working config can be as small as an
empty file. The config can be reloaded without a restart via `PUT /api/config/reload` or the
web UI.

<Callout type="info" emoji="ℹ️">
Coming from the original Node.js Rotom? The `config/local.json` format is gone. Use the
conversion script in the RotomNG repo to translate your old config:

```sh
python3 configs/rotom-og-to-ng.py old-local.json configs/rotom-ng.toml
```

See [Migrating from Rotom OG](/docs/rotom/migration) for the remaining differences.
</Callout>

## Device listener section

Handles connections from MITM devices and their workers.

```toml
[device_listener]
address = ":7070"
#secret = ""
#ping_interval = "30s"
#pong_wait = "30s"
```

| Key | Default | Description |
| :-- | :-- | :-- |
| `address` | ":7070" | Listen address for device/worker websocket connections. |
| `secret` | "" | Optional secret devices must send in the `X-Rotom-Secret` header. Blank disables authentication. |
| `ping_interval` | "30s" | Interval between websocket pings used to enforce the read timeout. |
| `pong_wait` | "30s" | Extra grace beyond `ping_interval` to receive a pong. The effective read timeout is `ping_interval` + `pong_wait`. |

## Controller listener section

Handles connections from Dragonite and any other controllers.

```toml
[controller_listener]
address = ":7071"
#secret = ""
#ping_interval = "30s"
#pong_wait = "30s"
#registration_timeout = "1m"
#data_timeout = "2m"
```

| Key | Default | Description |
| :-- | :-- | :-- |
| `address` | ":7071" | Listen address for controller websocket connections. This is what Dragonite's `rotom.endpoint` points at. |
| `secret` | "" | Optional secret controllers must send in the `X-Rotom-Secret` header. Must match Dragonite's `rotom.secret`. |
| `ping_interval` | "30s" | Interval between websocket pings used to enforce the read timeout. |
| `pong_wait` | "30s" | Extra grace beyond `ping_interval` to receive a pong. |
| `registration_timeout` | "1m" | Max time allowed for the controller registration handshake before falling back to the normal ping read timeout. |
| `data_timeout` | "2m" | The controller connection is considered dead when no data message is received in this period, independent of ping/pong activity. Set to `"0s"` to disable. |

## HTTP listener section

Serves the REST API and the web UI.

```toml
[http_listener]
address = ":7072"
#secret = ""
```

| Key | Default | Description |
| :-- | :-- | :-- |
| `address` | ":7072" | Listen address for the API and web UI. |
| `secret` | "" | Optional secret required in the `X-Rotom-Secret` header on all `/api` endpoints. Requests without it get `401 Unauthorized`. |

## Rate limit section

Controls how frequently a single device's workers can be selected. Disabled by default.

```toml
[rate_limit]
enable = false
max_selections = 10
duration = "1m"
```

| Key | Default | Description |
| :-- | :-- | :-- |
| `enable` | false | Enable device selection rate limiting. |
| `max_selections` | 0 | Maximum number of selections per `duration` for one device. |
| `duration` | "0s" | Time window for rate limiting, e.g. "30s", "1m", "5m". |

<Callout type="info" emoji="ℹ️">
Rate limiting is silently disabled if `max_selections` or `duration` is not a positive value.
</Callout>

## Jobs section

Jobs are shell commands that can be executed on devices from the web UI or API. This is disabled
by default, but can be enabled if your MITM supports it.

```toml
[jobs]
enable = false
path = "./jobs"
```

| Key | Default | Description |
| :-- | :-- | :-- |
| `enable` | false | Enable the jobs system. |
| `path` | "./jobs" | Directory containing job definition files. |

Job files are JSON, either a single object or an array of them:

```json filename="jobs/whoami.json"
{
"id": "whoami",
"description": "execute 'whoami'",
"exec": "whoami"
}
```

Job definitions can be re-read from disk without a restart with the **Reload** button on the
Jobs page (`PUT /api/job/-/reload`).

## Logging section

```toml
[logging]
level = "info"
format = "plain"
no_console_log = false

[logging.file]
#disable = false
#path = "./logs/rotom-ng.log"
#max_size_mb = 512
#max_backups = 30
#max_age_days = 30
#compress = false
#rotate_on_start = false
```

| Key | Default | Description |
| :-- | :-- | :-- |
| `level` | "info" | One of panic, fatal, error, warn, warning, info, debug, trace. |
| `format` | "plain" | Log format: plain or json. |
| `no_console_log` | false | Disable console logging and only write to the log file. |
| `file.disable` | false | Disable file logging entirely. |
| `file.path` | "./logs/rotom-ng.log" | Full path to the log file. |
| `file.max_size_mb` | 0 | Size in MB before rotation. 0 means no limit. |
| `file.max_backups` | 0 | Number of rotated files to keep. 0 keeps all. |
| `file.max_age_days` | 0 | Days to keep rotated files. 0 means no age limit. |
| `file.compress` | false | Compress rotated log files. |
| `file.rotate_on_start` | false | Rotate the log file on startup. |

## Prometheus section

```toml
[prometheus]
enable = false
#namespace = "rotom_ng"
```

| Key | Default | Description |
| :-- | :-- | :-- |
| `enable` | false | Expose metrics at `GET /api/metrics`. When disabled the endpoint returns `404`. |
| `namespace` | "rotom_ng" | Prefix applied to all metric names. |

See [Prometheus](/docs/other/prometheus) for scraping the rest of the stack.

## Tuning section

```toml
[tuning]
disable_worker_stats = false
#profiling = false
```

| Key | Default | Description |
| :-- | :-- | :-- |
| `disable_worker_stats` | false | Slight speedup at the cost of losing worker request stats in Prometheus and the UI. |
| `profiling` | false | Enable the Go pprof endpoints at `GET /api/debug/pprof/*`. |

## Shutdown timeout

A top level key, applying to the device, controller and HTTP listeners. Put it at the very top
of the file, before any section header, or it will be parsed as part of the section above it.

```toml
shutdown_timeout = "5s"
```

| Key | Default | Description |
| :-- | :-- | :-- |
| `shutdown_timeout` | "5s" | How long to wait for connections to drain on shutdown. |
33 changes: 28 additions & 5 deletions pages/docs/rotom/index.mdx
Original file line number Diff line number Diff line change
@@ -1,10 +1,32 @@
# Rotom
import { Callout } from 'nextra-theme-docs'

[**Rotom**](https://github.com/UnownHash/Rotom/) is our device controller. When a MITM client connects to Rotom it will be registered as a device. Devices can register 1 to many workers. A 1 to 1 relationship of device to worker is also completely valid. The connected devices create a pool of workers that can process events. How many workers are available is dependent on your MITM provider, your device constraints and the configuration you provide within your MITM tool.
# RotomNG

You can think of Rotom as a task scheduler, Rotom will pull jobs that need to be processed from Dragonite (backend controller) and will send those request to various workers depending on time, available capacity, and configuration.
[**RotomNG**](https://github.com/UnownHash/RotomNG/) is our device controller. When a MITM client connects to RotomNG it will be registered as a device. Devices can register 1 to many workers. A 1 to 1 relationship of device to worker is also completely valid. The connected devices create a pool of workers that can process events. How many workers are available is dependent on your MITM provider, your device constraints and the configuration you provide within your MITM tool.

Rotom communicates via websockets (`ws://`) to devices and Dragonite. This communication strategy allows for a smarter feedback loop between the backend server and actual worker.
You can think of RotomNG as a task scheduler, RotomNG will pull jobs that need to be processed from Dragonite (backend controller) and will send those request to various workers depending on time, available capacity, and configuration.

RotomNG communicates via websockets (`ws://`) to devices and Dragonite. This communication strategy allows for a smarter feedback loop between the backend server and actual worker.

<Callout type="info" emoji="ℹ️">
RotomNG replaces the original Node.js based Rotom (now referred to as "Rotom OG").
It is a drop-in replacement for devices and Dragonite &mdash; the websocket protocol and
the default ports are unchanged &mdash; but the config file format, the HTTP API and the
Prometheus metric names have changed. See [Migrating from Rotom OG](/docs/rotom/migration).
</Callout>

## Listeners

RotomNG exposes three listeners, all of which can be changed in your config file:

| Listener | Default | Purpose |
| :-- | :-- | :-- |
| Device | `:7070` | MITM devices and workers connect here |
| Controller | `:7071` | Dragonite (and other controllers) connect here |
| HTTP | `:7072` | REST API and the web UI |

The web UI is served from the HTTP listener, so with the defaults it is available at
[http://SERVER_YOUR_IP:7072]().

## Supported MITM clients

Expand All @@ -14,8 +36,9 @@ Rotom communicates via websockets (`ws://`) to devices and Dragonite. This commu
- [--=FurtiF™=-- Tools](https://discord.gg/wtNgst3W64) (Android)


## How MITM clients communicate with Rotom?
## How MITM clients communicate with RotomNG?

This is for MITM developers!

- [RotomProtos](https://github.com/UnownHash/RotomProtos)
- [HTTP API reference](https://github.com/UnownHash/RotomNG/blob/main/docs/RotomNG-API.md)
101 changes: 101 additions & 0 deletions pages/docs/rotom/migration.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
import { Callout } from 'nextra-theme-docs'

# Migrating from Rotom OG

RotomNG is the Go rewrite of the original Node.js Rotom ("Rotom OG"). Devices and Dragonite
talk to it exactly as before &mdash; same websocket protocol, same default ports &mdash; so the
migration is mostly about the config file. The HTTP API and the Prometheus metrics changed too,
which matters if you built tooling or dashboards against them.

The full, field by field guide lives in the repo:
[RotomNG-Vs-OG.md](https://github.com/UnownHash/RotomNG/blob/main/docs/RotomNG-Vs-OG.md).

## Steps

1. Stop Rotom OG.

1. Install RotomNG. See [Docker Setup](/docs/setup/docker) or [Standard Setup](/docs/setup/standard).

1. Convert your old config. The repo ships a conversion script that reads your old
`config/local.json` and writes the new TOML file:

```sh
python3 configs/rotom-og-to-ng.py /path/to/Rotom/config/local.json configs/rotom-ng.toml
```

Review the result against the [configuration reference](/docs/rotom/config).

1. Copy your job files (if you used jobs) into the `jobs` directory and set `jobs.enable = true`.

1. Start RotomNG and reconnect your devices.

<Callout type="info" emoji="ℹ️">
Dragonite needs no changes. Its `[rotom]` section still points at the controller listener
(`ws://HOST:7071`) and the `secret` key still maps to `controller_listener.secret`.
</Callout>

## Configuration

The config format changed from JSON (`config/local.json`) to TOML (`configs/rotom-ng.toml`).
Some highlights:

| Rotom OG | RotomNG |
| :-- | :-- |
| `deviceListener.port` | `device_listener.address` (`":7070"`) |
| `controllerListener.port` | `controller_listener.address` (`":7071"`) |
| `client.port` / `client.host` | `http_listener.address` (`":7072"`) |
| _(n/a)_ | `http_listener.secret` &mdash; optional API authentication |
| _(n/a)_ | `prometheus.enable` &mdash; metrics are now opt-in |

## HTTP API

- **Authentication** &mdash; OG had none. NG supports an optional `X-Rotom-Secret` header on all
`/api` endpoints, configured with `http_listener.secret`.
- **Methods** &mdash; device actions and job execution moved from `POST` to `PUT`.
- **Metrics** &mdash; `GET /metrics` moved to `GET /api/metrics` and must be enabled with
`prometheus.enable = true`.
- **Field names** &mdash; all JSON fields are now `snake_case` (OG used `camelCase`), and
`GET /api/status` nests workers under their device.
- **Removed** &mdash; `GET /api/getPublicIp` (public IP is now on the device object) and
`POST /api/ptcLogin`.
- **Jobs** &mdash; `GET /api/job/list` became `GET /api/job`, job execution became
`PUT /api/job/:jobId/run` with device IDs in the JSON body, and `GET /api/job/status` became
`GET /api/job-instance`.

The complete endpoint list is in
[RotomNG-API.md](https://github.com/UnownHash/RotomNG/blob/main/docs/RotomNG-API.md).

## Prometheus metrics

All metrics moved to the `rotom_ng` namespace (configurable via `prometheus.namespace`), and
the Node.js runtime metrics (`nodejs_*`, `process_*`) were replaced by Go runtime metrics
(`rotom_ng_go_*`, `rotom_ng_process_*`). Renamed metrics:

| Rotom OG | RotomNG |
| :-- | :-- |
| `rotom_devices_alive` | `rotom_ng_devices_connected` (adds an `origin` label) |
| `rotom_devices_total` | `rotom_ng_devices_total` (adds an `origin` label) |
| `rotom_workers_total` | `rotom_ng_workers_connected` |
| `rotom_workers_active` | `rotom_ng_workers_in_use` |
| `rotom_device_memory_free` | `rotom_ng_device_memory_free` |
| `rotom_device_memory_mitm` | `rotom_ng_device_memory_mitm` |
| `rotom_device_memory_start` | `rotom_ng_device_memory_start` |

NG also adds counters and histograms for device commands, registrations, connection accepts,
worker requests/responses, controllers, RPCs and app lifecycle events. Existing Grafana
dashboards built for Rotom OG will need their queries updated.

## Web UI

OG had two pages (Status and Jobs). NG splits things up and adds detail:

| Page | Route | Notes |
| :-- | :-- | :-- |
| Status | `/` | Aggregated controller, device and worker metrics |
| Devices | `/devices` | New dedicated page, expandable rows, enable/disable toggles |
| Controllers | `/controllers` | New &mdash; controller monitoring, reconnect/disconnect |
| Workers | `/workers` | New &mdash; per-worker stats and request rates |
| Jobs | `/jobs` | Reload from disk, start/finish timestamps, clear instances |

Destructive actions (reboot, restart, disconnect) now ask for confirmation, and devices can be
disabled so they are skipped for selection without disconnecting them.
Loading
Loading