[feature] Django admin workflow for mass commands with real-time monitoring

docs/user/intro.rst

@@ -70,9 +70,9 @@ e.g.:
 
 - Sending configuration updates.
 - :doc:`Executing shell commands <shell-commands>`.
-- :doc:`Executing mass commands <shell-commands>`: Run a command on
-  multiple devices at once, see the :ref:`batch command API
-  <controller_batch_command_api>` for details.
+- :ref:`Executing mass commands <mass_commands>`: Run a command on
+  multiple devices at once, from the admin or with the :ref:`batch command
+  API <controller_batch_command_api>`.
 - Perform firmware upgrades via the additional :doc:`firmware upgrade
   module </firmware-upgrader/index>`.
 

docs/user/rest-api.rst

@@ -506,7 +506,8 @@ Parameter        Description
                  ``?type=custom&input=%7B%22command%22%3A%22uptime%22%7D``
 ``devices``      Repeated ``devices`` query parameter, each a device UUID
                  (optional; when provided, ``group`` and ``location`` are
-                 ignored)
+                 ignored; all devices must belong to the same
+                 organization)
 ``group``        Device group UUID (optional)
 ``location``     Location UUID (optional)
 ================ =========================================================
@@ -534,7 +535,8 @@ Parameter        Description
                  (**required**)
 ``notes``        Optional notes (optional)
 ``devices``      List of device UUIDs (optional; when provided, ``group``
-                 and ``location`` are ignored)
+                 and ``location`` are ignored; all devices must belong to
+                 the same organization)
 ``group``        Device group UUID (optional)
 ``location``     Location UUID (optional)
 ================ =========================================================

docs/user/shell-commands.rst

@@ -194,24 +194,86 @@ useful for rebooting all devices in a group, changing passwords across
 multiple devices, or running diagnostics on all devices in an
 organization.
 
-**Targeting options:**
+Sending a Mass Command
+~~~~~~~~~~~~~~~~~~~~~~
 
-- ``organization``: All devices in an organization.
-- ``devices``: Explicit list of device UUIDs.
-- ``group``: Device group UUID.
-- ``location``: Location UUID.
+Open *Network Operations* > *Run Mass command* from the menu. The first
+step asks for:
 
-If ``devices`` is provided, ``group`` and ``location`` are ignored.
-Otherwise, ``group`` and ``location`` can be used together to narrow the
-target set within the organization.
+- the **command type** and its inputs, which change with the type
+  selected;
+- a **label** to identify the mass command later, and optional **notes**;
+- the **targets**: organization, device group and location.
 
-If no targeting options are provided, the command targets all devices in
-the organization. Superusers can omit ``organization`` to target all
-devices across organizations.
+The targets decide which devices are matched. Using more than one narrows
+the selection: a group and a location together match only the devices
+which are in that group *and* at that location.
 
-For superusers, ``organization`` is set automatically when ``group`` or
-``location`` is provided.
+Superusers can leave every target empty to run the command on all the
+devices of the system. Other users must choose at least one target, and
+only see the command types enabled for their organizations (see
+:ref:`openwisp_controller_organization_enabled_commands`).
 
-Refer to the :ref:`Batch Command API <controller_batch_command_api>`
-documentation for the available endpoints, request parameters, and
-examples.
+Reviewing the Devices
+~~~~~~~~~~~~~~~~~~~~~
+
+The second step shows a summary of the command and the list of the devices
+it matched.
+
+Devices can be left out by unchecking them: the counter and the *Execute
+on N devices* button follow the selection, which is kept while paging
+through the list. *Back* returns to the first step with the form still
+filled in.
+
+The mass command starts when the *Execute* button is clicked.
+
+Following the Results
+~~~~~~~~~~~~~~~~~~~~~
+
+After executing, the mass command page opens. It shows the status of the
+mass command, how many devices are affected, the devices which were
+skipped, and one row per device with its status and output.
+
+The rows are updated in real time, so the page does not need to be
+reloaded to follow the progress. The table can be searched by device name
+and filtered by status, device group and location (and by organization for
+superusers).
+
+.. note::
+
+    Commands are executed asynchronously in the background: each device is
+    handled by an independent background task, so how many devices are
+    contacted at the same time depends on the concurrency of the Celery
+    workers. A mass command sent to many devices keeps updating for a
+    while after the page is opened.
+
+Finding Past Mass Commands
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+*Network Operations* > *Mass commands* lists the mass commands which were
+sent, most recent first.
+
+The list can be searched by label, notes, organization, device, location
+and group name, and filtered by organization, status, type, group and
+location. Clicking a mass command opens the page described above.
+
+Skipped Devices
+~~~~~~~~~~~~~~~
+
+A device is skipped when the command cannot be created for it, for example
+when the device has no access credentials, or when the command type is not
+enabled for its organization.
+
+Skipped devices are not executed, but they are shown: the **Skipped
+devices** field summarizes how many there are and why, and each one is
+listed in the results table with the *skipped* status and the reason as
+its output. They can be found with the status filter.
+
+Using the API
+~~~~~~~~~~~~~
+
+The same operations are available over the REST API, which also accepts an
+explicit list of devices instead of the targets described above. All
+explicit devices must belong to the same organization. Refer to the
+:ref:`Batch Command API <controller_batch_command_api>` documentation for
+the available endpoints, request parameters and examples.

docs/user/websocket-api.rst

@@ -17,6 +17,8 @@ All endpoints:
   shown in JavaScript-style notation with inline comments for readability.
 - Push real-time updates after the connection is established.
 - Do not accept client messages: any data sent from the client is ignored.
+  The only exception is the mass command endpoint, which accepts the
+  single request documented below.
 
 Authentication and Authorization
 --------------------------------
@@ -161,3 +163,134 @@ After the connection is established, the server pushes a message every
 time the geometry of any mobile location in a subscribed organization is
 updated. The payload is identical to the one documented for the `2. Single
 Location Updates`_ endpoint.
+
+4. Mass Command Updates
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Connection URL:
+
+::
+
+    wss://<host>/ws/controller/batch-command/<batch_command_id>
+
+Scope
++++++
+
+Progress of a single mass command: its status and the result of every
+device it runs on. See :ref:`mass_commands`.
+
+Authorization
++++++++++++++
+
+A user is authorized if:
+
+- The user is a superuser, OR
+- The user is marked as staff AND has the ``connection.view_batchcommand``
+  or ``connection.change_batchcommand`` permission AND manages the
+  organization of the mass command.
+
+Real-time Updates
++++++++++++++++++
+
+The server pushes a message every time the mass command or one of its
+commands changes. The ``type`` field tells the two apart.
+
+When the mass command itself changes, for example when it moves from
+``idle`` to ``in-progress``:
+
+.. code-block:: javascript
+
+    {
+        "type": "batch_status",
+        "id": "<uuid>",                  // Mass command identifier
+        "label": "<string>",             // Label given when it was sent
+        "notes": "<string>",             // Notes given when it was sent
+        "input": { /* ... */ },          // Command input, masked for "change_password"
+        "organization": "<uuid>",        // Organization, null when system wide
+        "group": "<uuid>",               // Device group target, null when not used
+        "location": "<uuid>",            // Location target, null when not used
+        "status": "<string>",            // "idle", "in-progress", "success" or "failed"
+        "created": "<string>",           // ISO 8601 timestamp
+        "modified": "<string>",          // ISO 8601 timestamp
+        "affected_devices": <integer>,   // Number of devices the command runs on
+        "skipped_count": <integer>,      // Number of devices which were skipped
+        "skipped_preview": [ /* ... */], // First and last skipped devices, with the reason
+        "total_rows": <integer>          // Affected plus skipped devices
+    }
+
+The status and the timestamps are sent as they are stored, without
+translation or formatting, so that each client can render them with its
+own language and time zone. Command rows carry ``modified_display`` as
+well, which is the same timestamp already formatted with the locale and
+the time zone of the server: the admin uses it so that a row updated over
+the websocket reads exactly like the rows rendered with the page.
+
+When the command of one device changes:
+
+.. code-block:: javascript
+
+    {
+        "type": "command_update",
+        "id": "<uuid>",                  // Command identifier
+        "device": "<uuid>",              // Device identifier
+        "device_name": "<string>",       // Device name
+        "connection": "<uuid>",          // Device connection used, may be null
+        "batch_command": "<uuid>",       // Mass command this command belongs to
+        "status": "<string>",            // "in-progress", "success" or "failed"
+        "output": "<string>",            // Output preview: the last line of the
+                                         // output, at most its last 100
+                                         // characters, as in "batch_state"
+        "created": "<string>",           // ISO 8601 timestamp
+        "modified": "<string>",          // ISO 8601 timestamp
+        "modified_display": "<string>",  // Modified, formatted by the server with its
+                                         // own locale and time zone
+        "index": <integer>,              // Position of the row, sent only for new commands
+        "affected_devices": <integer>,   // Commands created so far, sent with "index"
+        "total_rows": <integer>          // Affected plus skipped devices, sent with "index"
+    }
+
+Requesting the Current State
+++++++++++++++++++++++++++++
+
+A client which connects while the mass command is already running can ask
+for the results it missed:
+
+.. code-block:: javascript
+
+    {
+        "type": "request_current_state",
+        "page": 1,                       // Page of results, 20 rows per page
+        "filters": {                     // Optional, the filters the page is showing
+            "q": "<string>",             // Search term matched against the device name
+            "status": "<string>",        // Command status, or "skipped"
+            "location_id": "<uuid>",     // Location of the device
+            "group_id": "<uuid>",        // Device group
+            "organization_id": "<uuid>"  // Organization of the device
+        }
+    }
+
+Every filter is optional and an empty string means the filter is not
+active. The filters are applied before the results are paginated, so a
+client which is showing a filtered table receives the same rows it would
+get by reloading the page.
+
+The server replies with one message holding that page:
+
+.. code-block:: javascript
+
+    {
+        "type": "batch_state",
+        "batch_status": { /* ... */ },   // Batch serializer fields, including
+                                         // the command type, but without
+                                         // "skipped_devices" or "total_rows"
+        "commands": [ /* ... */ ],       // Rows of the requested page. Command
+                                         // rows include the command type and omit
+                                         // event-only fields such as "index".
+                                         // Their "output" is a 100-character tail.
+        "page": <integer>,               // Effective page after bounds checking
+        "total_rows": <integer>          // Rows matching the filters, for the paginator
+    }
+
+Rows of devices which were skipped are included in ``commands`` with
+``is_skipped`` set to ``true``, a ``status`` of ``skipped`` and the reason
+in ``output``.

openwisp_controller/config/base/channels_consumer.py

@@ -10,7 +10,8 @@ class BaseDeviceConsumer(WebsocketConsumer):
     channel_layer_group = "config.device"
 
     def _is_user_authenticated(self):
-        return self.scope["user"].is_authenticated is True
+        user = self.scope.get("user")
+        return user is not None and user.is_authenticated
 
     def is_user_authorized(self):
         user = self.scope["user"]
@@ -32,16 +33,18 @@ def _user_has_permissions(self, add=True, change=True, delete=True):
         return self.scope["user"].has_perms(permissions)
 
     def connect(self):
+        if not self._is_user_authenticated() or not self.is_user_authorized():
+            self.close()
+            return
         try:
-            assert self._is_user_authenticated() and self.is_user_authorized()
             self.pk_ = self.scope["url_route"]["kwargs"]["pk"]
-        except (AssertionError, KeyError):
+        except KeyError:
             self.close()
-        else:
-            async_to_sync(self.channel_layer.group_add)(
-                f"{self.channel_layer_group}-{self.pk_}", self.channel_name
-            )
-            self.accept()
+            return
+        async_to_sync(self.channel_layer.group_add)(
+            f"{self.channel_layer_group}-{self.pk_}", self.channel_name
+        )
+        self.accept()
 
     def disconnect(self, close_code):
         try: