diff --git a/app/queue-server/doc/images/connection_widget.png b/app/queue-server/doc/images/connection_widget.png new file mode 100644 index 0000000000..13b3f8969e Binary files /dev/null and b/app/queue-server/doc/images/connection_widget.png differ diff --git a/app/queue-server/doc/images/console_monitor.png b/app/queue-server/doc/images/console_monitor.png new file mode 100644 index 0000000000..830be4f942 Binary files /dev/null and b/app/queue-server/doc/images/console_monitor.png differ diff --git a/app/queue-server/doc/images/edit_and_control_queue.png b/app/queue-server/doc/images/edit_and_control_queue.png new file mode 100644 index 0000000000..5ce57fd247 Binary files /dev/null and b/app/queue-server/doc/images/edit_and_control_queue.png differ diff --git a/app/queue-server/doc/images/environment_controls.png b/app/queue-server/doc/images/environment_controls.png new file mode 100644 index 0000000000..86ab255c31 Binary files /dev/null and b/app/queue-server/doc/images/environment_controls.png differ diff --git a/app/queue-server/doc/images/execution_controls.png b/app/queue-server/doc/images/execution_controls.png new file mode 100644 index 0000000000..707f86214c Binary files /dev/null and b/app/queue-server/doc/images/execution_controls.png differ diff --git a/app/queue-server/doc/images/plan_editor.png b/app/queue-server/doc/images/plan_editor.png new file mode 100644 index 0000000000..2b60c1dc80 Binary files /dev/null and b/app/queue-server/doc/images/plan_editor.png differ diff --git a/app/queue-server/doc/images/plan_history.png b/app/queue-server/doc/images/plan_history.png new file mode 100644 index 0000000000..3806e85087 Binary files /dev/null and b/app/queue-server/doc/images/plan_history.png differ diff --git a/app/queue-server/doc/images/plan_manager.png b/app/queue-server/doc/images/plan_manager.png new file mode 100644 index 0000000000..eb9281839b Binary files /dev/null and b/app/queue-server/doc/images/plan_manager.png differ diff --git a/app/queue-server/doc/images/plan_queue.png b/app/queue-server/doc/images/plan_queue.png new file mode 100644 index 0000000000..021989eeac Binary files /dev/null and b/app/queue-server/doc/images/plan_queue.png differ diff --git a/app/queue-server/doc/images/plan_viewer.png b/app/queue-server/doc/images/plan_viewer.png new file mode 100644 index 0000000000..51c49d032d Binary files /dev/null and b/app/queue-server/doc/images/plan_viewer.png differ diff --git a/app/queue-server/doc/images/queue_controls.png b/app/queue-server/doc/images/queue_controls.png new file mode 100644 index 0000000000..f221f600cc Binary files /dev/null and b/app/queue-server/doc/images/queue_controls.png differ diff --git a/app/queue-server/doc/images/queue_monitor.png b/app/queue-server/doc/images/queue_monitor.png new file mode 100644 index 0000000000..4988f0c106 Binary files /dev/null and b/app/queue-server/doc/images/queue_monitor.png differ diff --git a/app/queue-server/doc/images/running_plan.png b/app/queue-server/doc/images/running_plan.png new file mode 100644 index 0000000000..106bb28720 Binary files /dev/null and b/app/queue-server/doc/images/running_plan.png differ diff --git a/app/queue-server/doc/images/status_monitor.png b/app/queue-server/doc/images/status_monitor.png new file mode 100644 index 0000000000..c1bb9c13c6 Binary files /dev/null and b/app/queue-server/doc/images/status_monitor.png differ diff --git a/app/queue-server/doc/index.rst b/app/queue-server/doc/index.rst new file mode 100644 index 0000000000..3f7b9c38f2 --- /dev/null +++ b/app/queue-server/doc/index.rst @@ -0,0 +1,401 @@ +Queue Server +============ + +Overview +-------- + +The Queue Server application provides a graphical interface for the +`Bluesky Queue Server `_, +allowing users to manage experiment queues, edit plans, control execution, +and monitor status at beamlines and scientific facilities. + +It communicates with the Queue Server's HTTP API and optionally +uses WebSocket connections for real-time status and console streaming. + +.. image:: images/edit_and_control_queue.png + :width: 80% + :align: center + +The application provides two independent views, each available +as a separate Phoebus application: + +Edit & Control Queue + Full queue management, plan editing, and execution controls. + +Queue Monitor + Read-only monitoring with collapsible panels and live console output. + + +Getting Started +--------------- + +To open the Edit & Control Queue: + + 1. Invoke the menu ``Applications``, ``Utility``, ``Edit & Control Queue``. + +To open the Queue Monitor: + + 1. Invoke the menu ``Applications``, ``Utility``, ``Queue Monitor``. + +Both applications connect to the same Queue Server instance +and share connection state. Connecting in one application +automatically connects the other. + + +Configuration +^^^^^^^^^^^^^ + +Set environment variables before launching Phoebus to configure +the connection: + +.. code-block:: bash + + # Queue Server HTTP address (default: http://localhost:60610) + export QSERVER_HTTP_SERVER_URI=http://localhost:60610 + + # API key for authentication + export QSERVER_HTTP_SERVER_API_KEY=your_api_key + + # Or point to a file containing the API key + export QSERVER_HTTP_SERVER_API_KEYFILE=~/.phoebus/qserver_api_key.txt + +Alternatively, configure via **Edit > Preferences > Queue Server**. + + +Starting a Local Queue Server +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Use the provided Docker setup for local development: + +.. code-block:: bash + + cd services/bluesky-services + docker-compose --profile container-redis up -d + +This starts: + +- Bluesky Queue Server (RE Manager) on ports 60615/60625 +- HTTP Server REST API on port 60610 +- Redis database on port 6380 + +For details, see ``services/bluesky-services/README.md``. + + +Edit & Control Queue +-------------------- + +.. image:: images/edit_and_control_queue.png + :width: 80% + :align: center + +The Edit & Control Queue provides the full interface for managing +the plan queue and controlling experiment execution. + +The top of the window contains four control panels arranged +left to right: + +- **Environment** -- Open, Close, and Destroy the RE environment +- **Queue** -- Start/Stop queue execution with autostart checkbox +- **Plan Execution** -- Pause, Resume, Ctrl-C, Stop, Abort, and Halt controls +- **RE Manager Status** -- Live status grid showing environment state, + manager state, RE state, queue/history counts, autostart, stop pending, + and loop mode + +Below the control panels, the window is split into two columns: + +Left Column +^^^^^^^^^^^ + +**Plan Manager** (top-left) -- A tabbed panel with two tabs: + +- **Plan Viewer** tab: Displays the parameters of the selected queue item + in a table with Parameter, checkbox, and Value columns. + An "All Parameters" checkbox toggles between showing only modified + parameters and all parameters including defaults. + Buttons at the bottom: ``Copy to Queue`` and ``Edit``. + +- **Plan Editor** tab: Select between Plan and Instruction using radio buttons, + then choose a specific plan from the dropdown. Edit parameter values + in the table. Buttons at the bottom: ``Batch Upload``, ``Add to Queue``, + ``Save``, ``Reset``, and ``Cancel``. + +.. image:: images/plan_manager.png + :width: 400 + :align: center + +**Plan Queue** (bottom-left) -- Table showing queued items with columns: +Index, Type, Name, Parameters, USER, GROUP. +Toolbar buttons: ``Up``, ``Down``, ``Top``, ``Bottom``, ``Deselect``, +``Clear``, ``Loop`` (toggle), ``Delete``, ``Duplicate``. + +.. image:: images/plan_queue.png + :width: 80% + :align: center + +Right Column +^^^^^^^^^^^^ + +**Running Plan** (top-right) -- Shows the currently executing plan +as formatted text. Buttons: ``Update Environment`` and ``Copy to Queue``. + +.. image:: images/running_plan.png + :width: 400 + :align: center + +**Plan History** (bottom-right) -- Table of completed plans with columns: +Index, Type, Name, STATUS, Parameters, USER, GROUP. +Toolbar buttons: ``Copy to Queue``, ``Deselect All``, ``Clear History``, +and ``Export`` (split button with TXT, JSON, and YAML options). + +.. image:: images/plan_history.png + :width: 80% + :align: center + + +Queue Monitor +------------- + +.. image:: images/queue_monitor.png + :width: 80% + :align: center + +The Queue Monitor provides a compact, read-only view of the queue state, +suitable for wall displays or passive monitoring. + +The top of the window shows: + +- **Queue Server** connection widget (left) with status label and + Connect/Disconnect toggle +- **RE Manager Status** grid (right) with the same status fields + as the Edit & Control view + +Below the top bar, four collapsible ``TitledPane`` sections +are stacked vertically: + +- **Running Plan** (collapsed by default) -- Same running plan display +- **Plan Queue** (collapsed by default) -- Read-only queue table +- **Plan History** (collapsed by default) -- Read-only history table +- **Console Monitor** (expanded by default) -- Live console output + +Expand or collapse each section by clicking its header. + + +Connection Widget +----------------- + +.. image:: images/connection_widget.png + :width: 120 + :align: center + +The connection widget shows the current connection status and provides +a toggle button to connect or disconnect. + +The connection status label changes color to indicate the current state: + ++----------------+--------+---------------------------------------------+ +| Label | Color | Meaning | ++================+========+=============================================+ +| OFFLINE | Grey | Disconnected, user toggled off | ++----------------+--------+---------------------------------------------+ +| CONNECTING | Grey | Attempting to establish connection | ++----------------+--------+---------------------------------------------+ +| NETWORK | Blue | Cannot reach the server (auth/network issue) | ++----------------+--------+---------------------------------------------+ +| STATUS | Red | Connected but no status from RE Manager | ++----------------+--------+---------------------------------------------+ +| CONNECTED | Green | Connected and receiving status updates | ++----------------+--------+---------------------------------------------+ + +By default, the widget auto-connects on startup. If the connection +is lost, it automatically attempts to reconnect every 5 seconds. + +When both Edit & Control Queue and Queue Monitor are open, +they share the same connection. Toggling the connection in either +application affects both. + + +Environment Controls +-------------------- + +.. image:: images/environment_controls.png + :width: 120 + :align: center + +The Environment panel manages the RunEngine environment: + +- **Open**: Initialize the RE environment (required before running plans) +- **Close**: Gracefully shut down the RE environment +- **Destroy**: Force-terminate the environment (use if Close hangs) + +Buttons are enabled or disabled based on the current environment state. + + +Queue Controls +-------------- + +.. image:: images/queue_controls.png + :width: 170 + :align: center + +The Queue panel controls queue execution: + +- **Start**: Begin executing queued plans +- **Stop**: Stop after the current plan completes +- **auto**: Checkbox to enable autostart when the environment opens + +The status label shows the current queue execution state +(e.g., STOPPED, RUNNING). + + +Execution Controls +------------------ + +.. image:: images/execution_controls.png + :width: 290 + :align: center + +The Plan Execution panel provides fine-grained control over +the currently running plan: + +- **Pause: Deferred** -- Pause at the next checkpoint +- **Pause: Immediate** -- Pause immediately +- **Resume** -- Resume a paused plan +- **Ctrl-C** -- Send interrupt signal +- **Stop** -- Stop the plan gracefully +- **Abort** -- Abort the plan immediately +- **Halt** -- Emergency halt (no cleanup) + + +RE Manager Status +----------------- + +.. image:: images/status_monitor.png + :width: 300 + :align: center + +The status panel displays a grid of live information: + ++---------------------+----------------------+ +| Left Column | Right Column | ++=====================+======================+ +| RE Environment | Queue AUTOSTART | ++---------------------+----------------------+ +| Manager state | Queue STOP pending | ++---------------------+----------------------+ +| RE state | Queue LOOP mode | ++---------------------+----------------------+ +| Items in history | Items in queue | ++---------------------+----------------------+ + +Values update in real-time as status messages arrive from the server. + + +Console Monitor +--------------- + +.. image:: images/console_monitor.png + :width: 80% + :align: center + +The Console Monitor streams live output from the Queue Server +via WebSocket (or HTTP polling as fallback). + +Controls: + +- **Autoscroll**: Checkbox to auto-scroll to the latest output (enabled by default) +- **Max. Lines**: Text field to limit the number of displayed lines +- **Clear**: Clear the console output + + +Plan Editor +----------- + +.. image:: images/plan_editor.png + :width: 400 + :align: center + +The Plan Editor tab allows creating new plans or editing existing ones. + +1. Select **Plan** or **Instruction** using the radio buttons. +2. Choose a plan from the dropdown (populated from the Queue Server's + list of allowed plans). +3. Edit parameters in the table. Each row shows the parameter name, + a checkbox to include it, and an editable value field. + +Parameter values support Python types: strings, integers, floats, +lists, dictionaries, booleans, and ``None``. + +Buttons: + +- **Batch Upload**: Upload plans from a file +- **Add to Queue**: Add the configured plan to the queue +- **Save**: Save edits to an existing queue item +- **Reset**: Restore default parameter values +- **Cancel**: Discard changes + + +Plan Viewer +----------- + +.. image:: images/plan_viewer.png + :width: 400 + :align: center + +The Plan Viewer tab displays the parameters of the selected queue +or history item in a read-only table. + +- **All Parameters**: Toggle to show all parameters including defaults +- **Copy to Queue**: Duplicate the item back into the queue +- **Edit**: Switch to the Plan Editor tab with this item loaded + + +Preferences +----------- + +Configure via **Edit > Preferences > Queue Server**. + ++------------------------+------------------------------------------+----------------------------+ +| Preference | Environment Variable | Default | ++========================+==========================================+============================+ +| ``queue_server_url`` | ``QSERVER_HTTP_SERVER_URI`` | ``http://localhost:60610`` | ++------------------------+------------------------------------------+----------------------------+ +| ``api_key`` | ``QSERVER_HTTP_SERVER_API_KEY`` | *(none)* | ++------------------------+------------------------------------------+----------------------------+ +| ``api_key_file`` | ``QSERVER_HTTP_SERVER_API_KEYFILE`` | *(none)* | ++------------------------+------------------------------------------+----------------------------+ +| ``use_websockets`` | | ``true`` | ++------------------------+------------------------------------------+----------------------------+ +| ``update_interval_ms`` | | ``500`` | ++------------------------+------------------------------------------+----------------------------+ +| ``connectTimeout`` | | ``5000`` | ++------------------------+------------------------------------------+----------------------------+ +| ``debug`` | | ``false`` | ++------------------------+------------------------------------------+----------------------------+ + +Environment variables override preference file values. + + +Troubleshooting +--------------- + +**Connection stays at NETWORK** + +- Verify the Queue Server HTTP API is running and reachable +- Check the ``queue_server_url`` preference or ``QSERVER_HTTP_SERVER_URI`` +- Verify the API key is correct + +**Connection shows STATUS (red)** + +- The WebSocket connected but the RE Manager is not sending status updates +- Check that the RE Manager process is running on the server side + +**Plans dropdown is empty** + +- The environment must be open for the list of allowed plans to load +- Check the console output for error messages + +**Console output not streaming** + +- WebSocket streaming is preferred; verify ``use_websockets=true`` +- If WebSocket fails, the application falls back to HTTP polling + at the interval set by ``update_interval_ms``