Add a dashboard to visualise the Job queue and explore logs

.github/workflows/dashboard.yml

@@ -0,0 +1,69 @@
+name: Deploy Dashboard
+
+on:
+  push:
+    branches: [master]
+    paths: ['dashboard/**']
+  pull_request:
+    types: [opened, reopened, synchronize, closed]
+    paths: ['dashboard/**']
+
+permissions:
+  contents: write
+  pull-requests: write
+
+concurrency: pages-${{ github.ref }}
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Setup Node
+        if: github.event.action != 'closed'
+        uses: actions/setup-node@v4
+        with:
+          node-version: 22
+
+      - name: Install tools
+        if: github.event.action != 'closed'
+        run: npm install --no-save purescript@0.15.15 spago@1.0.3 esbuild
+
+      # TODO: this is temporary until we fix a bug in spago for bundling single packages:
+      # there is a `purs graph` call that fails when doing that because it's trying to use
+      # all the dependencies of the package, which might not be there. So we build everything.
+      - name: Build whole repo
+        if: github.event.action != 'closed'
+        run: npx spago build
+
+      - name: Build dashboard
+        if: github.event.action != 'closed'
+        run: npm run dashboard:build
+
+      - name: Verify bundle
+        if: github.event.action != 'closed'
+        run: test -f dashboard/app.js
+
+      - name: Prepare deploy directory
+        if: github.event.action != 'closed'
+        run: |
+          mkdir -p _site
+          cp dashboard/index.html _site/
+          cp dashboard/app.js _site/
+          cp -r dashboard/static _site/
+
+      - name: Deploy to Pages
+        if: github.ref == 'refs/heads/master'
+        uses: JamesIves/github-pages-deploy-action@v4
+        with:
+          folder: _site
+          clean-exclude: pr-preview
+
+      # On 'closed' events this removes the preview directory from gh-pages;
+      # on all other PR events it deploys the build to pr-preview/pr-<number>/.
+      - name: Deploy PR preview
+        if: github.event_name == 'pull_request'
+        uses: rossjrw/pr-preview-action@v1
+        with:
+          source-dir: ./_site/

.gitignore

@@ -4,6 +4,7 @@
 /scratch
 /.vscode
 /scripts/analysis
+/generated-docs
 
 result*
 
@@ -20,5 +21,8 @@ result*
 TODO.md
 .spec-results
 
+# Generated bundle
+dashboard/app.js
+
 # Keep it secret, keep it safe.
 .env

CONTRIBUTING.md

@@ -36,6 +36,10 @@ There are two additional PureScript directories focused on testing only:
 - `app-e2e` contains tests that exercise the server API, which requires that a server and associated wiremock services are running
 - `test-utils` contains utility code intended only for tests
 
+There is one additional directory for the web dashboard:
+
+- `dashboard` contains the static HTML/CSS/JS dashboard for monitoring registry jobs. It is deployed independently to GitHub Pages and requires no build step.
+
 There are three more directories containing code for the registry.
 
 - `db` contains schemas and migrations for the sqlite3 database used by the server.
@@ -253,6 +257,36 @@ services.wiremock-github-api = {
 
 It is also possible to include specific files that should be returned to requests via the `files` key. Here's another short example of setting up an S3 mock, in which we copy files from the fixtures into the wiremock service's working directory given a particular file name, and then write request/response mappings that respond to requests by reading the file at path given by `bodyFileName`.
 
+## Dashboard Development
+
+The `dashboard/` directory contains a Halogen (PureScript) application for monitoring registry jobs. It is deployed to GitHub Pages and calls the registry API cross-origin.
+
+### Building
+
+To produce a browser JS bundle:
+
+```sh
+npm run dashboard:build
+```
+
+This outputs `dashboard/app.js`, which `dashboard/index.html` loads via `<script src="./app.js">`.
+
+### Local Development
+
+For iterative development with live rebuild and a local dev server:
+
+```sh
+npm run dashboard:dev
+```
+
+This starts esbuild in watch mode: it serves the dashboard at `http://localhost:8000` and automatically rebuilds `app.js` when PureScript source files change (after running `spago build`).
+
+The API base URL is hardcoded in `dashboard/src/Dashboard/API.purs` (`defaultConfig`) to the production registry URL. The server includes CORS headers with `Access-Control-Allow-Origin: *`, so cross-origin requests from the dashboard work without any additional configuration.
+
+### Deployment
+
+The dashboard is deployed automatically by the `.github/workflows/dashboard.yml` workflow. Pushing changes to `dashboard/**` on `master` triggers a GitHub Pages deployment.
+
 ## Deployment
 
 The registry is continuously deployed. The [deploy.yml](./.github/workflows/deploy.yml) file defines a GitHub Actions workflow to auto-deploy the server when a new commit is pushed to `master` and test workflows have passed.

app/spago.yaml

@@ -21,6 +21,7 @@ package:
     - dotenv
     - effect
     - either
+    - enums
     - exceptions
     - exists
     - fetch
@@ -33,7 +34,6 @@ package:
     - httpurple
     - identity
     - integers
-    - js-date
     - js-fetch
     - js-promise-aff
     - js-uri
@@ -51,7 +51,6 @@ package:
     - nullable
     - numbers
     - ordered-collections
-    - orders
     - parallel
     - parsing
     - partial
@@ -64,14 +63,12 @@ package:
     - run
     - safe-coerce
     - strings
-    - these
     - transformers
     - tuples
     - typelevel-prelude
     - unicode
     - unsafe-coerce
     - uuidv4
-    - variant
   test:
     main: Test.Registry.Main
     dependencies:

dashboard/index.html

@@ -0,0 +1,15 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <title>PureScript Registry Dashboard</title>
+  <link href="https://fonts.googleapis.com/css?family=Roboto+Mono|Roboto:300,400,400i,700,700i" type="text/css" rel="stylesheet">
+  <link rel="stylesheet" href="static/style.css">
+  <link rel="icon" href="data:,">
+</head>
+<body>
+  <noscript>This dashboard requires JavaScript to run.</noscript>
+  <script src="./app.js"></script>
+</body>
+</html>

dashboard/spago.yaml

@@ -0,0 +1,35 @@
+package:
+  name: registry-dashboard
+  publish:
+    license: BSD-3-Clause
+    version: 0.0.1
+  dependencies:
+    - aff
+    - arrays
+    - codec-json
+    - const
+    - control
+    - datetime
+    - effect
+    - either
+    - exceptions
+    - fetch
+    - foldable-traversable
+    - formatters
+    - halogen
+    - halogen-subscriptions
+    - integers
+    - json
+    - lists
+    - maybe
+    - newtype
+    - now
+    - parallel
+    - prelude
+    - registry-lib
+    - routing-duplex
+    - strings
+    - tailrec
+    - web-events
+    - web-html
+    - web-uievents

dashboard/src/Dashboard/API.purs

@@ -0,0 +1,128 @@
+-- | HTTP client for making requests to the registry server from the dashboard.
+-- | Provides typed helpers for fetching job data from the Registry API.
+module Dashboard.API
+  ( ApiConfig
+  , ApiError(..)
+  , defaultConfig
+  , fetchJobs
+  , fetchJob
+  , printApiError
+  ) where
+
+import Prelude
+
+import Codec.JSON.DecodeError as CJ.DecodeError
+import Control.Alt ((<|>))
+import Control.Parallel (parallel, sequential)
+import Data.Codec.JSON as CJ
+import Data.DateTime (DateTime)
+import Data.Either (Either(..))
+import Data.Maybe (Maybe(..))
+import Data.String as String
+import Effect.Aff (Aff, Milliseconds(..))
+import Effect.Aff as Aff
+import Effect.Exception as Exception
+import Fetch (Method(..))
+import Fetch as Fetch
+import JSON as JSON
+import Registry.API.V1 (Job, JobId, LogLevel, Route(..), SortOrder)
+import Registry.API.V1 as V1
+import Routing.Duplex as Routing
+
+-- | Configuration for the API client.
+type ApiConfig =
+  { baseUrl :: String
+  }
+
+-- | Default API configuration pointing to the production registry server.
+defaultConfig :: ApiConfig
+defaultConfig =
+  { baseUrl: "https://registry.purescript.org"
+  }
+
+-- | Errors that can occur when making API requests.
+data ApiError
+  = HttpError { status :: Int, body :: String }
+  | ParseError { message :: String, raw :: String }
+
+-- | Render an ApiError as a human-readable string.
+printApiError :: ApiError -> String
+printApiError = case _ of
+  HttpError { status, body } ->
+    "HTTP " <> show status <> ": " <> body
+  ParseError { message, raw } ->
+    "Parse error: " <> message <> "\nResponse: " <> String.take 500 raw
+
+-- | Print a V1 Route to its URL path string.
+printRoute :: Route -> String
+printRoute = Routing.print V1.routes
+
+-- | Parse a JSON string using a codec, returning Either ApiError.
+parseJson :: forall a. CJ.Codec a -> String -> Either ApiError a
+parseJson codec str = case JSON.parse str of
+  Left jsonErr ->
+    Left $ ParseError { message: "JSON: " <> jsonErr, raw: str }
+  Right json -> case CJ.decode codec json of
+    Left decodeErr ->
+      Left $ ParseError { message: CJ.DecodeError.print decodeErr, raw: str }
+    Right a ->
+      Right a
+
+-- | Request timeout in milliseconds.
+requestTimeout :: Milliseconds
+requestTimeout = Milliseconds 10000.0
+
+-- | Run an Aff action with a timeout. Returns Nothing if the action does not
+-- | complete within the given duration, or Just the result if it does.
+timeout :: forall a. Milliseconds -> Aff a -> Aff (Maybe a)
+timeout ms action = sequential do
+  parallel (Just <$> action) <|> parallel (Nothing <$ Aff.delay ms)
+
+-- | Make a GET request to the given URL path and decode the response body.
+get :: forall a. CJ.Codec a -> ApiConfig -> String -> Aff (Either ApiError a)
+get codec config path = do
+  result <- Aff.try $ timeout requestTimeout do
+    response <- Fetch.fetch (config.baseUrl <> path) { method: GET }
+    body <- response.text
+    pure { status: response.status, body }
+  case result of
+    Left err ->
+      pure $ Left $ HttpError { status: 0, body: Exception.message err }
+    Right Nothing ->
+      pure $ Left $ HttpError { status: 0, body: "Request timed out" }
+    Right (Just { status, body })
+      | status >= 200 && status < 300 ->
+          pure $ parseJson codec body
+      | otherwise ->
+          pure $ Left $ HttpError { status, body }
+
+-- | Fetch the list of jobs from the registry server.
+-- |
+-- | Parameters:
+-- | - `since`: Only return jobs created after this time
+-- | - `until`: Only return jobs created before this time
+-- | - `order`: Sort order for results (ASC or DESC)
+-- | - `includeCompleted`: When true, include finished jobs in the results
+fetchJobs
+  :: ApiConfig
+  -> { since :: Maybe DateTime, until :: Maybe DateTime, order :: Maybe SortOrder, includeCompleted :: Maybe Boolean }
+  -> Aff (Either ApiError (Array Job))
+fetchJobs config params = do
+  let route = Jobs { since: params.since, until: params.until, order: params.order, include_completed: params.includeCompleted }
+  get (CJ.array V1.jobCodec) config (printRoute route)
+
+-- | Fetch a single job by its ID.
+-- |
+-- | Parameters:
+-- | - `level`: Minimum log level to include in the response
+-- | - `since`: Only return log lines after this time
+-- | - `until`: Only return log lines before this time
+-- | - `order`: Sort order for log lines (ASC or DESC)
+fetchJob
+  :: ApiConfig
+  -> JobId
+  -> { level :: Maybe LogLevel, since :: Maybe DateTime, until :: Maybe DateTime, order :: Maybe SortOrder }
+  -> Aff (Either ApiError Job)
+fetchJob config jobId params = do
+  let route = Job jobId { level: params.level, since: params.since, until: params.until, order: params.order }
+  get V1.jobCodec config (printRoute route)