**************************
Web UI WebSocket Protocol
**************************

The built-in LN manager browser UI uses normal HTTP plus a JSON-over-WebSocket
protocol on the same manager TCP port.

This section documents that protocol for users who want to implement their own
websocket-based interfaces.

Transport
=========

The manager-side entry point is the same port configured by the instance
setting ``manager``.

Connection flow:

1. the client performs an HTTP request, typically ``GET / HTTP/1.1``
2. HTTP Basic Auth is required unless the client already has a valid session
   cookie
3. after successful authentication, the client requests a WebSocket upgrade
4. after the upgrade, all application messages are JSON objects in WebSocket
   text frames

The built-in browser UI also uses lightweight focused WebSocket modes selected
via query parameters on the upgrade request:

* ``/ws?single_process=PROCESS/NAME``:
  subscribe only to one process and its output stream
* ``/ws?single_group=GROUP/NAME``:
  subscribe only to one recursive group subtree and the output streams of its
  contained processes
* ``/ws?log_only=1``:
  subscribe only to manager log messages

HTTP Authentication
===================

The built-in browser UI uses the auth file configured by ``webui_auth_file`` or
``LNM_WEBUI_AUTH_FILE``.

Authentication rules:

* if neither is configured, HTTP is refused with a plain-text explanation
* if the auth file is group/other-readable, HTTP is refused with a plain-text
  explanation
* the initial browser login uses HTTP Basic Auth
* after successful authentication, ``ln_manager`` returns a cookie:

  .. code-block:: text

     Set-Cookie: ln_manager_web_session=<random-token>; Path=/; HttpOnly; SameSite=Lax

* that random token is persisted in ``<authfile>.sessions``
* later HTTP and WebSocket requests can authenticate only with that cookie

HTTP keep-alive is supported for plain HTTP requests, so custom clients should
prefer reusing the same TCP connection instead of reconnecting for every asset
or metadata request.

Raw HTTP Log Endpoint
=====================

Besides the browser WebSocket UI, the manager also exposes a plain-text log
endpoint:

.. code-block:: text

   GET /lnm/log?raw

Authentication is the same as for the browser UI: either HTTP Basic Auth or a
valid session cookie.

Supported query parameters:

* ``level=debug|info|warning|error``:
  minimum log level to return
* ``limit=N``:
  return only the last ``N`` matching in-memory log entries
* ``follow``:
  keep the HTTP response open and stream future matching log entries using
  HTTP chunked transfer encoding

Without ``limit``, the manager returns all currently retained in-memory log
entries matching the requested level.

Example one-shot fetch:

.. code-block:: bash

   wget -q --user=USER --password=PASSWORD -O - \
     "http://<IP>:<PORT>/lnm/log?raw&level=debug&limit=15"

Example follow mode:

.. code-block:: bash

   wget -q --user=USER --password=PASSWORD -O - \
     "http://<IP>:<PORT>/lnm/log?raw&level=info&follow"

Save-All HTTP Endpoint
======================

The diagnostics archive is deliberately exposed as plain HTTP, not as a
WebSocket frame:

.. code-block:: text

   GET /save-all

Authentication is the same as the browser UI. The response body is binary
``.tar.bz2`` data and includes a ``Content-Disposition: attachment`` header so
browsers treat it as a file download.

Supported query parameters:

* ``name=NAME``:
  optional sanitized name hint for the top-level archive prefix and suggested
  download filename

Example:

.. code-block:: bash

   wget --user=USER --password=PASSWORD -O debug.tar.bz2 \
     "http://<IP>:<PORT>/save-all?name=my_debug_report"

LN Daemon Logger HTTP Endpoint
==============================

LN daemon logger downloads are plain HTTP downloads, not WebSocket frames.
They retrieve data from daemon-side per-port ring buffers and are unrelated to
``lnrecorder``:

.. code-block:: text

   GET /lnm/logger-download?name=LOGGER&format=pickle&stop=1

Authentication is the same as the browser UI.

Supported query parameters:

* ``name=LOGGER``:
  name of the manager-side LN daemon logger object
* ``format=pickle|matlab|raw``:
  output format; ``pickle`` is the default
* ``stop=1``:
  stop the logger before collecting and downloading data

The response body is binary data with a ``Content-Disposition: attachment``
header. The server sends large downloads asynchronously in chunks so the
manager does not fail on short non-blocking socket writes.

JSON Message Format
===================

After WebSocket upgrade, every application frame is a JSON object with a
top-level ``type`` field.

Manager to client messages
--------------------------

Common manager-to-client frame types are:

* ``snapshot``
* ``process_tree``
* ``process_upsert``
* ``process_remove``
* ``process_output``
* ``process_output_snapshot``
* ``clients_snapshot``
* ``client_upsert``
* ``client_remove``
* ``process_list``
* ``process_list_task_set_result``
* ``process_attribute_set_result``
* ``process_startup_commands``
* ``topic_upsert``
* ``topic_remove``
* ``service_upsert``
* ``service_remove``
* ``services_snapshot``
* ``parameters_snapshot``
* ``parameter_override_result``
* ``parameter_open_scope_result``
* ``log_batch``
* ``loggers_snapshot``
* ``logger_upsert``
* ``logger_remove``
* ``logger_action_result``
* ``network_stats``
* ``network_topology``
* ``error``

Client to manager messages
--------------------------

Common client-to-manager frame types are:

* ``get_snapshot``
* ``start_process``
* ``stop_process``
* ``reload_config``
* ``get_process_output``
* ``subscribe_process_output``
* ``unsubscribe_process_output``
* ``send_stdin``
* ``send_winch``
* ``start_group``
* ``stop_group``
* ``set_process_current_alternate_command``
* ``set_process_command_override``
* ``set_process_alternate_command_override``
* ``set_process_no_skip_display``
* ``set_process_low_delay_output``
* ``set_process_attribute``
* ``dump_startup_commands``
* ``request_clients``
* ``request_process_list``
* ``set_process_list_task``
* ``request_logs``
* ``set_log_min_level``
* ``request_topics_stat``
* ``request_topic_detail``
* ``request_topic_scheduling``
* ``apply_topic_scheduling``
* ``request_parameters``
* ``set_parameter_override``
* ``open_parameter_scope``
* ``request_network_stats``
* ``request_network_topology``
* ``request_loggers``
* ``configure_logger``
* ``start_logger``
* ``stop_logger``
* ``delete_logger``
* ``request_config_file``
* ``save_config_file``

Process Snapshot Example
========================

Initial state is typically fetched with:

.. code-block:: json

   {"type": "get_snapshot"}

The manager answers with a large ``snapshot`` object. A shortened example:

.. code-block:: json

   {
     "type": "snapshot",
     "manager": {
       "name": "test instance",
       "host": "rmc-lx0031"
     },
     "hosts": ["rmc-lx0031"],
     "host_list": [
       {
         "name": "rmc-lx0031",
         "aliases": ["rmc-lx0031.example.org"],
         "node_aliases": ["localhost"],
         "all_names": ["localhost", "rmc-lx0031", "rmc-lx0031.example.org"],
         "daemon_ready": true
       }
     ],
     "start_tools": [
       {"value": null, "label": "default"},
       {"value": "gdb", "label": "gdb"},
       {"value": "valgrind", "label": "valgrind"}
     ],
     "process_tree": [
       {
         "type": "group",
         "name": "all",
         "display_name": "all",
         "opened": true,
         "children": [
           {"type": "process", "name": "all/publish"},
           {"type": "process", "name": "all/subscribe"}
         ]
       }
     ],
     "processes": [
       {
         "name": "all/publish",
         "display_name": "publish",
         "state": "ready",
         "requested_state": "start",
         "pid": 3208721,
         "host": "rmc-lx0031",
         "node": "localhost",
         "command": "/usr/bin/python3 publish.py 100",
         "command_override": null,
         "alternate_commands": [
           ["default", "/usr/bin/python3 publish.py 100"],
           ["htop", "/usr/bin/htop"]
         ],
         "alternate_command_overrides": {},
         "current_alternate_command": "default",
         "start_tool": null,
         "ln_debug": false,
         "current_command": "/usr/bin/python3 publish.py 100",
         "has_ready_state": true,
         "start_time": "2026-04-21T14:11:38.521003",
         "stop_time": null
       }
     ],
     "clients": [
       {
         "id": 1,
         "name": "trace_test43",
         "pid": 3208721,
         "host": "rmc-lx0031",
         "address": "10.0.0.7:46122",
         "library_version": 16,
         "svn_revision": 1234,
         "modified": false,
         "ln_debug": false,
         "process_name": "all/publish",
         "published_topics": ["/topic/demo"],
         "subscribed_topics": [],
         "provided_services": []
       }
     ]
   }

Note that ``process_upsert`` does not contain accumulated process output. That
data is intentionally streamed separately.

``hosts`` is the compatibility list of canonical host names. Prefer
``host_list`` for new clients that need host-name aliases, node-name aliases,
or current daemon readiness.

Daemon Process Lists
====================

The browser UI exposes daemon process/task listings through
``/lnm/process-list``. PID/TID links in the UI open that route with query
parameters such as:

.. code-block:: text

   /lnm/process-list?host=HOST&pid=PID&tid=TID

The underlying WebSocket request is:

.. code-block:: json

   {
     "type": "request_process_list",
     "host": "HOST",
     "show_threads": true,
     "show_env": false,
     "filter": ""
   }

The manager answers with ``process_list``. The payload contains ordered column
metadata and row values with both raw and formatted values. These lists can be
large; clients should avoid rendering all rows synchronously.

``filter`` is passed to the daemon as a structured process-list filter, not as
free text. Each item is encoded as ``key:op:value`` and multiple items are
separated by NUL bytes. For example, use ``tid:==:1234`` to request one Linux
task/thread.

Scheduling, priority, and affinity changes use:

.. code-block:: json

   {
     "type": "set_process_list_task",
     "host": "HOST",
     "pid": 1234,
     "tid": 1234,
     "policy": 1,
     "prio": 20,
     "affinity": "0,1"
   }

Clients
========

Client state is also available explicitly:

.. code-block:: json

   {"type": "request_clients"}

The manager responds with:

.. code-block:: json

   {
     "type": "clients_snapshot",
     "clients": []
   }

Incremental changes arrive as ``client_upsert`` and ``client_remove``.

Parameters
==========

LN parameter providers can be queried explicitly:

.. code-block:: json

   {
     "type": "request_parameters",
     "pattern": "*"
   }

The manager responds with formatted values:

.. code-block:: json

   {
     "type": "parameters_snapshot",
     "pattern": "*",
     "parameters": [
       {
         "id": "7:ln.parameters.example.cpp.query_dict:ln.parameters.example.cpp.sig1",
         "name": "ln.parameters.example.cpp.sig1",
         "service": "ln.parameters.example.cpp.query_dict",
         "provider": "ln_parameters_example",
         "provider_id": 7,
         "description": "sig1 desc",
         "input": "1.0",
         "output": "1.0",
         "override_value": "1.0",
         "override_enabled": false
       }
     ],
     "errors": []
   }

Set or clear one override with:

.. code-block:: json

   {
     "type": "set_parameter_override",
     "service": "ln.parameters.example.cpp.query_dict",
     "provider_id": 7,
     "parameter": "ln.parameters.example.cpp.sig1",
     "enabled": true,
     "value": "42.0"
   }

The result frame is ``parameter_override_result``.

Request publication of one parameter as a topic and open it in ``ln_scope``:

.. code-block:: json

   {
     "type": "open_parameter_scope",
     "service": "ln.parameters.example.cpp.query_dict",
     "provider_id": 7,
     "parameter": "ln.parameters.example.cpp.sig1"
   }

The manager calls the provider's matching ``request_topic`` service and then
opens ``<parameter>.output`` in the configured scope process. The result frame is
``parameter_open_scope_result``.

The stdlib-only reference client exposes convenience wrappers for these
messages:

.. code-block:: python

   from links_and_nodes_manager_client import ManagerClient

   async with ManagerClient("http://localhost:43809") as client:
       parameters = await client.get_parameters("*")
       parameter = parameters[0]

       await client.set_parameter_override(
           parameter["service"],
           parameter["name"],
           True,
           "42.0",
           provider_id=parameter["provider_id"],
       )

       await client.open_parameter_scope(
           parameter["service"],
           parameter["name"],
           provider_id=parameter["provider_id"],
       )

Process Control Examples
========================

Start a process:

.. code-block:: json

   {"type": "start_process", "name": "all/publish"}

Stop a process:

.. code-block:: json

   {"type": "stop_process", "name": "all/publish"}

Start all processes in a group:

.. code-block:: json

   {"type": "start_group", "name": "all/subgroup1"}

Stop all processes in a group:

.. code-block:: json

   {"type": "stop_group", "name": "all/subgroup1"}

Select an alternate command:

.. code-block:: json

   {
     "type": "set_process_current_alternate_command",
     "name": "all/publish",
     "which": "htop"
   }

Override the primary command:

.. code-block:: json

   {
     "type": "set_process_command_override",
     "name": "all/publish",
     "override": "/usr/bin/python3 publish.py 200"
   }

Override a non-default alternate command:

.. code-block:: json

   {
     "type": "set_process_alternate_command_override",
     "name": "all/publish",
     "which": "htop",
     "override": "/usr/bin/htop -d 20"
   }

Set process UI/runtime attributes such as the selected start tool or
``LN_DEBUG``:

.. code-block:: json

   {
     "type": "set_process_attribute",
     "name": "all/publish",
     "attribute": "start_tool",
     "value": "gdb"
   }

``attribute`` is currently limited to ``start_tool`` and ``ln_debug``.

To inspect the generated startup command sequence without injecting it into the
live terminal:

.. code-block:: json

   {"type": "dump_startup_commands", "name": "all/publish"}

Terminal Output and Input
=========================

To fetch initial terminal backlog for a process:

.. code-block:: json

   {
     "type": "get_process_output",
     "name": "all/publish",
     "lines": 500
   }

The manager answers with a complete snapshot for that process terminal:

.. code-block:: json

   {
     "type": "process_output_snapshot",
     "name": "all/publish",
     "pid": 3208721,
     "output": "ready\r\n"
   }

This snapshot request does not subscribe the connection to future live output.
Live process output is opt-in per WebSocket connection. Subscribe with one or
more ``fnmatch.fnmatch`` process-name patterns:

.. code-block:: json

   {
     "type": "subscribe_process_output",
     "patterns": ["all/publish", "tools/*"]
   }

Exact process names are valid patterns. A client that wants all live process
output can subscribe to ``"*"``. To stop receiving matching live output,
remove the same patterns:

.. code-block:: json

   {
     "type": "unsubscribe_process_output",
     "patterns": ["tools/*"]
   }

After subscription, live output for matching process names arrives
incrementally:

.. code-block:: json

   {
     "type": "process_output",
     "name": "all/publish",
     "output": "new line of output\r\n"
   }

Keyboard input is sent to the process via:

.. code-block:: json

   {
     "type": "send_stdin",
     "name": "all/publish",
     "text": "q"
   }

Terminal resize is sent with:

.. code-block:: json

   {
     "type": "send_winch",
     "name": "all/publish",
     "rows": 40,
     "cols": 120
   }

The built-in web UI uses these messages to mirror the GTK VTE behavior.

Python Reference Client
=======================

``python/links_and_nodes_manager_client`` contains the stdlib-only Python
3.11+ reference implementation for this HTTP/WebSocket interface. It is meant
both as a usable asyncio client and as demo code for custom integrations. Keep
it feature-complete when adding new HTTP routes or WebSocket message types.

It can also be installed with ``pip`` directly from this repository subdir:

.. code-block:: bash

   python3 -m pip install ./python/links_and_nodes_manager_client

After that, both the importable package and the ``lnm-client`` console script
are available.

Example:

.. code-block:: python

   import asyncio
   from links_and_nodes_manager_client import ManagerClient

   async def main():
       async with ManagerClient("localhost:43809") as lnm:
           await lnm.get_snapshot()
           await lnm.start_process("all/publish")
           output = await lnm.get_process_output("all/publish")
           print(output["output"])

   asyncio.run(main())

For simple scripts that should not manage an asyncio loop, use
``SyncManagerClient``:

.. code-block:: python

   from links_and_nodes_manager_client import SyncManagerClient

   with SyncManagerClient("localhost:43809") as lnm:
       snapshot = lnm.get_snapshot()
       print("processes:", len(snapshot.get("processes", [])))
       output = lnm.get_process_output("all/publish")
       print(output["output"])

``SyncManagerClient`` is a generated wrapper around ``ManagerClient``. It does
not implement a second HTTP/WebSocket protocol stack; each synchronous method
calls the corresponding async method and runs an internal asyncio event loop
until that call is complete. This makes it easier for beginners and small
automation scripts while keeping ``ManagerClient`` the maintained reference
implementation. Code that already runs inside asyncio should use
``ManagerClient`` directly instead.

The package also provides a demo CLI:

.. code-block:: bash

   export LN_MANAGER=localhost:43809
   python3 -m links_and_nodes_manager_client host-list
   python3 -m links_and_nodes_manager_client processes --watch
   python3 -m links_and_nodes_manager_client process-list localhost --filter htop
   python3 -m links_and_nodes_manager_client output -f all/publish

The CLI and library can use explicit credentials, ``LN_MANAGER_USER`` /
``LN_MANAGER_PASSWORD``, or ``LNM_WEBUI_AUTH_FILE``. When only a username is
specified and the auth file exists, the password is read from that file. When
no username is specified, the first auth-file entry is used. The file must not
be readable by group or others.

Proxy support
-------------

The reference client honors ``http_proxy`` / ``https_proxy`` / ``all_proxy``
and supports HTTP CONNECT plus SOCKS4/SOCKS4A/SOCKS5 proxies without external
dependencies. This is useful for reaching a manager port through an SSH jump
host:

.. code-block:: bash

   ssh -N -D 127.0.0.1:1080 user@jump-host

In another shell:

.. code-block:: bash

   export all_proxy=socks5h://127.0.0.1:1080
   export LN_MANAGER=manager-host-visible-from-jump:43809
   python3 -m links_and_nodes_manager_client processes
   python3 -m links_and_nodes_manager_client logs -f

``socks5h://`` asks the proxy side to resolve the manager hostname. Use
``socks5://`` only when local DNS resolution is desired. To bypass the proxy
for directly reachable hosts, use ``no_proxy``:

.. code-block:: bash

   export no_proxy=localhost,127.0.0.1,manager-on-local-network

Topics, Services and Logs
=========================

Topic updates:

.. code-block:: json

   {
     "type": "topic_upsert",
     "topic": {
       "name": "topic1",
       "publisher": "publish",
       "publisher_host": "rmc-lx0031",
       "subscribers": 1,
       "publisher_rate": 100.0,
       "published_packets": 95006,
       "message_definition": "uint32_t",
       "message_size": 1984
     }
   }

Service updates:

.. code-block:: json

   {
     "type": "service_upsert",
     "service": {
       "name": "publish.request_topic",
       "interface": "service/request_topic",
       "provider": "publish",
       "provider_host": "rmc-lx0031",
       "port": 46321,
       "unix_socket": null
     }
   }

Log batches:

.. code-block:: json

   {
     "type": "log_batch",
     "entries": [
       {
         "id": 184,
         "ts": "2026-04-21T14:11:39.771621",
         "level": "info",
         "source": "Manager",
         "msg": "process 'all/publish' changes state from 'started' to 'ready'"
       }
     ]
   }

LN daemon logger snapshot:

.. code-block:: json

   {
     "type": "loggers_snapshot",
     "loggers": [
       {
         "name": "experiment1",
         "enabled": true,
         "active_topics": ["topic1"],
         "topics": [
           {
             "name": "topic1",
             "log_size": 10000,
             "divisor": 1,
             "only_ts": false,
             "enabled": true,
             "active": true,
             "n_samples": 321,
             "md_name": "uint32_t",
             "port": "shm:1234 on rmc-lx0031"
           }
         ]
       }
     ]
   }

Configure an LN daemon logger:

.. code-block:: json

   {
     "type": "configure_logger",
     "name": "experiment1",
     "only_ts": false,
     "topics": [
       {"name": "topic1", "log_size": 10000, "divisor": 1}
     ]
   }

Start or stop it:

.. code-block:: json

   {"type": "start_logger", "name": "experiment1"}
   {"type": "stop_logger", "name": "experiment1"}

Changes arrive as ``logger_upsert`` / ``logger_remove``. Command results arrive
as ``logger_action_result``.

Network usage sampling:

.. code-block:: json

   {"type": "request_network_stats", "sample_seconds": 2.0, "sample_mode": "interval"}

The manager answers asynchronously with ``network_stats``. ``sample_seconds``
controls the packet/byte counter sampling interval. ``sample_mode`` can be
``interval`` or ``previous``; ``previous`` compares against the last manager-side
network sample and does not sleep. If no previous sample exists yet, the manager
falls back to a 2 second interval sample. The payload includes
combined logical host links, measured per-interface traffic, per-row
``children`` breakdowns for topics, service-call accounting, TCP forwards,
daemon-manager connections, client-manager connections, ``lnm_remote``
GUI/console connections, and web UI HTTP/WS traffic, and possible problems when
measured traffic is more than 5 percent below the theoretical value.
``links`` is an endpoint view: it attributes traffic to the logical source and
destination hosts and does not include intermediate forwarding hops as separate
host-link rows. Physical hop/interface traffic is represented by the
``interfaces`` rows instead.
``links_theoretical`` and ``links_measured`` are still sent for compatibility,
but new clients should use ``links``:

.. code-block:: json

   {
     "type": "network_stats",
     "ok": true,
     "stats": {
       "sample_seconds": 2.0,
       "links": [
         {
           "from_host": "host1",
           "to_host": "host2",
           "bytes_s": 1048576.0,
           "packets_s": 1000.0,
           "measured_bytes_s": 996147.0,
           "measured_packets_s": 950.0,
           "children": [
             {
               "kind": "topic",
               "name": "topic1",
               "bytes_s": 1048576.0,
               "packets_s": 1000.0,
               "measured_bytes_s": 996147.0,
               "measured_packets_s": 950.0
             }
           ]
         }
       ],
       "interfaces": [],
       "problems": [
         {
           "severity": "warning",
           "message": "the link between host1 and host2 seems to be saturated, overused, or lossy"
         }
       ]
     }
   }

Network topology:

.. code-block:: json

   {"type": "request_network_topology"}

The manager answers with ``network_topology`` containing configured networks and
gateway hosts:

.. code-block:: json

   {
     "type": "network_topology",
     "ok": true,
     "topology": {
       "networks": [
         {
           "name": "net1",
           "netmasks": ["10.0.0.0/24"],
           "interfaces": [{"host": "host1", "interface": "eth0", "ip": "10.0.0.1"}]
         }
       ],
       "gateways": []
     }
   }

Error Frames
============

Protocol or execution errors are reported as:

.. code-block:: json

   {
     "type": "error",
     "message": "unknown action 'frobnicate'"
   }

Implementation Notes
====================

The built-in implementation currently lives in:

* manager side: ``python/links_and_nodes_manager/web_server/``
* websocket helper: ``python/links_and_nodes_base/websocket_protocol.py``
* browser UI: ``python/links_and_nodes_manager/resources/html/app.js``

That code is the source of truth if the documentation and implementation ever
diverge.