Skip to content

fix(telemetry): consistent invocation ID per run and telemetry acceptance tests #271

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
leggetter merged 8 commits into from
Mar 19, 2026
Merged

fix(telemetry): consistent invocation ID per run and telemetry acceptance tests #271

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
26bcf0a
fix(telemetry): consistent invocation ID per run and add telemetry te…
leggetter Mar 19, 2026
763bfb7
docs(cli): use 'hookdeck gateway connection' in connection command ex…
leggetter Mar 19, 2026
f67928c
chore(docs): regenerate REFERENCE.md from Cobra metadata
leggetter Mar 19, 2026
ec62347
chore(acceptance): run telemetry tests in slice 1, remove unreproduci…
leggetter Mar 19, 2026
8f52178
test(acceptance): move listen telemetry proxy test under telemetry tag
leggetter Mar 19, 2026
74e34c4
fix(ci): force telemetry on for acceptance-telemetry job
leggetter Mar 19, 2026
dad166b
fix(acceptance): tag JSON config tests so telemetry job runs telemetr…
leggetter Mar 19, 2026
04036b1
acceptance: retry CLIRunner on transient HTTP 502 with logging
leggetter Mar 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/workflows/test-acceptance.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ jobs:
env:
ACCEPTANCE_SLICE: ${{ matrix.slice }}
HOOKDECK_CLI_TESTING_API_KEY: ${{ secrets[matrix.api_key_secret] }}
HOOKDECK_CLI_TELEMETRY_DISABLED: "1"
steps:
- name: Check out code
uses: actions/checkout@v3
Expand All @@ -36,3 +37,25 @@ jobs:

- name: Run Go Acceptance Tests (slice ${{ matrix.slice }})
run: go test -tags="${{ matrix.tags }}" ./test/acceptance/... -v -timeout 12m

# Telemetry proxy tests require the real telemetry header; matrix jobs set
# HOOKDECK_CLI_TELEMETRY_DISABLED=1. This job runs -tags=telemetry with
# HOOKDECK_CLI_TELEMETRY_DISABLED=0 so telemetry is on even if the repo sets a global opt-out.
acceptance-telemetry:
runs-on: ubuntu-latest
env:
ACCEPTANCE_SLICE: "0"
HOOKDECK_CLI_TESTING_API_KEY: ${{ secrets.HOOKDECK_CLI_TESTING_API_KEY }}
# Explicitly allow telemetry: repo/org env may set HOOKDECK_CLI_TELEMETRY_DISABLED=1 globally.
HOOKDECK_CLI_TELEMETRY_DISABLED: "0"
steps:
- name: Check out code
uses: actions/checkout@v3

- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: "1.24.9"

- name: Run telemetry acceptance tests
run: go test -tags=telemetry ./test/acceptance/... -v -timeout 12m
6 changes: 5 additions & 1 deletion AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -274,6 +274,10 @@ go run cmd/hookdeck/main.go login --help
- **Always run tests** when changing code: unit tests (`go test ./pkg/...`) and, for CLI-facing changes, acceptance tests (`go test ./test/acceptance/...`). Do not skip tests to avoid failures.
- When running commands (build, test, acceptance tests), if you see **TLS/certificate errors** (e.g. `x509: certificate verify failed`, `tls: failed to verify certificate`), **permission errors** (e.g. `operation not permitted` when writing to the Go module cache), or similar failures that look environment-related, the command is likely running inside a **sandbox**. **Prompt the user** and **re-run the command with elevated permissions** (e.g. `required_permissions: ["network"]` for tests that need API access, or `["all"]` to disable the sandbox) so the operation can succeed. Do not treat a build or test as passed if stderr shows these errors, even when the process exit code is 0.

### GitHub CLI (`gh`)
- Use the **[GitHub CLI](https://cli.github.com/) (`gh`)** to read GitHub data and perform actions from the shell: **workflow runs and job logs** (e.g. `gh run list`, `gh run view <run-id> --log-failed`, `gh run view <run-id> --job <job-id> --log`), **PRs and checks** (`gh pr view`, `gh pr checks`, `gh pr diff`), **API access** (`gh api`), and creating or updating PRs, issues, and releases.
- Install and authenticate `gh` where needed (e.g. `gh auth login`). If `gh` fails with TLS, network, or permission errors, re-run with **network** or **all** permissions when the agent sandbox may be blocking access.

## 6. Documentation Standards

### Command help text (Short and Long)
Expand Down Expand Up @@ -355,7 +359,7 @@ if apiErr, ok := err.(*hookdeck.APIError); ok {
Acceptance tests require a Hookdeck API key. See [`test/acceptance/README.md`](test/acceptance/README.md) for full details. Quick setup: create `test/acceptance/.env` with `HOOKDECK_CLI_TESTING_API_KEY=<key>`. The `.env` file is git-ignored and must never be committed.

### Acceptance tests and feature tags
Acceptance tests in `test/acceptance/` are partitioned by **feature build tags** so they can run in parallel (two slices in CI and locally). Each test file must have exactly one feature tag (e.g. `//go:build connection`, `//go:build request`). The runner (CI workflow or `run_parallel.sh`) maps features to slices and passes the corresponding `-tags="..."`; see [test/acceptance/README.md](test/acceptance/README.md) for slice mapping and setup. **Every new acceptance test file must have a feature tag**; otherwise it is included in every build and runs in both slices (duplicated). Use tags to balance and parallelize; same commands and env for local and CI.
Acceptance tests in `test/acceptance/` are partitioned by **feature build tags** so they can run in parallel (matrix slices plus a separate `acceptance-telemetry` job in CI; see [test/acceptance/README.md](test/acceptance/README.md)). Each `*_test.go` file must have exactly one feature tag (e.g. `//go:build connection`, `//go:build request`, `//go:build telemetry`). **Untagged test files are included in every `-tags=...` build**, including `-tags=telemetry` only, so non-telemetry tests would run in the telemetry job—do not add untagged `*_test.go` files. Use tags to balance and parallelize; same commands and env for local and CI.

### Unit Testing
- Test validation logic thoroughly
Expand Down
128 changes: 86 additions & 42 deletions REFERENCE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,12 +59,14 @@ hookdeck login [flags]
| Flag | Type | Description |
|------|------|-------------|
| `-i, --interactive` | `bool` | Run interactive configuration mode if you cannot open a browser |
| `--local` | `bool` | Save credentials to current directory (.hookdeck/config.toml) |

**Examples:**

```bash
$ hookdeck login
$ hookdeck login -i # interactive mode (no browser)
$ hookdeck login --local # save credentials to .hookdeck/config.toml
```
## Logout

Expand Down Expand Up @@ -103,8 +105,6 @@ hookdeck whoami
```bash
$ hookdeck whoami
```

Output includes the current project type (Gateway, Outpost, or Console).
<!-- GENERATE_END -->
## Projects

Expand All @@ -114,7 +114,7 @@ Output includes the current project type (Gateway, Outpost, or Console).

### hookdeck project list

List and filter projects by organization and project name substrings. Output shows project type (Gateway, Outpost, Console). Outbound projects are excluded from the list.
List and filter projects by organization and project name substrings

**Usage:**

Expand All @@ -126,16 +126,15 @@ hookdeck project list [<organization_substring>] [<project_substring>] [flags]

| Flag | Type | Description |
|------|------|-------------|
| `--output` | `string` | Output format: `json` for machine-readable list (id, org, project, type, current) |
| `--type` | `string` | Filter by project type: `gateway`, `outpost`, or `console` |
| `--output` | `string` | Output format: json |
| `--type` | `string` | Filter by project type: gateway, outpost, console |

**Examples:**

```bash
$ hookdeck project list
Acme / Ecommerce Production (current) | Gateway
Acme / Ecommerce Staging | Gateway

$ hookdeck project list --output json
$ hookdeck project list --type gateway
```
Expand All @@ -161,14 +160,11 @@ hookdeck project use [<organization_name> [<project_name>]] [flags]
$ hookdeck project use
Use the arrow keys to navigate: ↓ ↑ → ←
? Select Project:
▸ [Acme] Ecommerce Production
[Acme] Ecommerce Staging
[Acme] Ecommerce Development

Selecting project [Acme] Ecommerce Staging
▸ Acme / Ecommerce Production (current) | Gateway
Acme / Ecommerce Staging | Gateway

$ hookdeck project use --local
Pinning project [Acme] Ecommerce Staging to current directory
Pinning project to current directory
```
<!-- GENERATE_END -->
## Local Development
Expand Down Expand Up @@ -219,7 +215,6 @@ Commands for managing Event Gateway sources, destinations, connections,
transformations, events, requests, metrics, and MCP server.

The gateway command group provides full access to all Event Gateway resources.
**Gateway commands require the current project to be a Gateway project** (inbound or console). If your project type is Outpost or you have no project selected, run `hookdeck project use` to switch to a Gateway project first.

**Usage:**

Expand Down Expand Up @@ -282,22 +277,22 @@ hookdeck gateway connection list [flags]

```bash
# List all connections
hookdeck connection list
hookdeck gateway connection list

# Filter by connection name
hookdeck connection list --name my-connection
hookdeck gateway connection list --name my-connection

# Filter by source ID
hookdeck connection list --source-id src_abc123
hookdeck gateway connection list --source-id src_abc123

# Filter by destination ID
hookdeck connection list --destination-id dst_def456
hookdeck gateway connection list --destination-id dst_def456

# Include disabled connections
hookdeck connection list --disabled
hookdeck gateway connection list --disabled

# Limit results
hookdeck connection list --limit 10
hookdeck gateway connection list --limit 10
```
### hookdeck gateway connection create

Expand Down Expand Up @@ -353,10 +348,10 @@ hookdeck gateway connection create [flags]
| `--rule-deduplicate-include-fields` | `string` | Comma-separated list of fields to include for deduplication |
| `--rule-deduplicate-window` | `int` | Time window in seconds for deduplication (default "0") |
| `--rule-delay` | `int` | Delay in milliseconds (default "0") |
| `--rule-filter-body` | `string` | JQ expression to filter on request body |
| `--rule-filter-headers` | `string` | JQ expression to filter on request headers |
| `--rule-filter-path` | `string` | JQ expression to filter on request path |
| `--rule-filter-query` | `string` | JQ expression to filter on request query parameters |
| `--rule-filter-body` | `string` | Filter on request body using Hookdeck filter syntax (JSON) |
| `--rule-filter-headers` | `string` | Filter on request headers using Hookdeck filter syntax (JSON) |
| `--rule-filter-path` | `string` | Filter on request path using Hookdeck filter syntax (JSON) |
| `--rule-filter-query` | `string` | Filter on request query parameters using Hookdeck filter syntax (JSON) |
| `--rule-retry-count` | `int` | Number of retry attempts (default "0") |
| `--rule-retry-interval` | `int` | Interval between retries in milliseconds (default "0") |
| `--rule-retry-response-status-codes` | `string` | Comma-separated HTTP status codes to retry on |
Expand Down Expand Up @@ -386,19 +381,19 @@ hookdeck gateway connection create [flags]

```bash
# Create with inline source and destination
hookdeck connection create \
hookdeck gateway connection create \
--name "test-webhooks-to-local" \
--source-type WEBHOOK --source-name "test-webhooks" \
--destination-type CLI --destination-name "local-dev"

# Create with existing resources
hookdeck connection create \
hookdeck gateway connection create \
--name "github-to-api" \
--source-id src_abc123 \
--destination-id dst_def456

# Create with source configuration options
hookdeck connection create \
hookdeck gateway connection create \
--name "api-webhooks" \
--source-type WEBHOOK --source-name "api-source" \
--source-allowed-http-methods "POST,PUT,PATCH" \
Expand All @@ -418,6 +413,12 @@ You can specify either a connection ID or name.
hookdeck gateway connection get <connection-id-or-name> [flags]
```

**Arguments:**

| Argument | Type | Description |
|----------|------|-------------|
| `connection-id-or-name` | `string` | **Required.** Connection ID or name |

**Flags:**

| Flag | Type | Description |
Expand All @@ -430,10 +431,10 @@ hookdeck gateway connection get <connection-id-or-name> [flags]

```bash
# Get connection by ID
hookdeck connection get conn_abc123
hookdeck gateway connection get conn_abc123

# Get connection by name
hookdeck connection get my-connection
hookdeck gateway connection get my-connection
```
### hookdeck gateway connection update

Expand All @@ -448,6 +449,12 @@ and allows changing any field including the connection name.
hookdeck gateway connection update <connection-id> [flags]
```

**Arguments:**

| Argument | Type | Description |
|----------|------|-------------|
| `connection-id` | `string` | **Required.** Connection ID |

**Flags:**

| Flag | Type | Description |
Expand All @@ -460,10 +467,10 @@ hookdeck gateway connection update <connection-id> [flags]
| `--rule-deduplicate-include-fields` | `string` | Comma-separated list of fields to include for deduplication |
| `--rule-deduplicate-window` | `int` | Time window in seconds for deduplication (default "0") |
| `--rule-delay` | `int` | Delay in milliseconds (default "0") |
| `--rule-filter-body` | `string` | JQ expression to filter on request body |
| `--rule-filter-headers` | `string` | JQ expression to filter on request headers |
| `--rule-filter-path` | `string` | JQ expression to filter on request path |
| `--rule-filter-query` | `string` | JQ expression to filter on request query parameters |
| `--rule-filter-body` | `string` | Filter on request body using Hookdeck filter syntax (JSON) |
| `--rule-filter-headers` | `string` | Filter on request headers using Hookdeck filter syntax (JSON) |
| `--rule-filter-path` | `string` | Filter on request path using Hookdeck filter syntax (JSON) |
| `--rule-filter-query` | `string` | Filter on request query parameters using Hookdeck filter syntax (JSON) |
| `--rule-retry-count` | `int` | Number of retry attempts (default "0") |
| `--rule-retry-interval` | `int` | Interval between retries in milliseconds (default "0") |
| `--rule-retry-response-status-codes` | `string` | Comma-separated HTTP status codes to retry on |
Expand Down Expand Up @@ -504,6 +511,12 @@ Delete a connection.
hookdeck gateway connection delete <connection-id> [flags]
```

**Arguments:**

| Argument | Type | Description |
|----------|------|-------------|
| `connection-id` | `string` | **Required.** Connection ID |

**Flags:**

| Flag | Type | Description |
Expand All @@ -514,10 +527,10 @@ hookdeck gateway connection delete <connection-id> [flags]

```bash
# Delete a connection (with confirmation)
hookdeck connection delete conn_abc123
hookdeck gateway connection delete conn_abc123

# Force delete without confirmation
hookdeck connection delete conn_abc123 --force
hookdeck gateway connection delete conn_abc123 --force
```
### hookdeck gateway connection upsert

Expand All @@ -542,6 +555,12 @@ Create a new connection or update an existing one by name (idempotent).
hookdeck gateway connection upsert <name> [flags]
```

**Arguments:**

| Argument | Type | Description |
|----------|------|-------------|
| `name` | `string` | **Required.** Connection name (create or update by name) |

**Flags:**

| Flag | Type | Description |
Expand Down Expand Up @@ -584,10 +603,10 @@ hookdeck gateway connection upsert <name> [flags]
| `--rule-deduplicate-include-fields` | `string` | Comma-separated list of fields to include for deduplication |
| `--rule-deduplicate-window` | `int` | Time window in seconds for deduplication (default "0") |
| `--rule-delay` | `int` | Delay in milliseconds (default "0") |
| `--rule-filter-body` | `string` | JQ expression to filter on request body |
| `--rule-filter-headers` | `string` | JQ expression to filter on request headers |
| `--rule-filter-path` | `string` | JQ expression to filter on request path |
| `--rule-filter-query` | `string` | JQ expression to filter on request query parameters |
| `--rule-filter-body` | `string` | Filter on request body using Hookdeck filter syntax (JSON) |
| `--rule-filter-headers` | `string` | Filter on request headers using Hookdeck filter syntax (JSON) |
| `--rule-filter-path` | `string` | Filter on request path using Hookdeck filter syntax (JSON) |
| `--rule-filter-query` | `string` | Filter on request query parameters using Hookdeck filter syntax (JSON) |
| `--rule-retry-count` | `int` | Number of retry attempts (default "0") |
| `--rule-retry-interval` | `int` | Interval between retries in milliseconds (default "0") |
| `--rule-retry-response-status-codes` | `string` | Comma-separated HTTP status codes to retry on |
Expand Down Expand Up @@ -617,22 +636,22 @@ hookdeck gateway connection upsert <name> [flags]

```bash
# Create or update a connection with inline source and destination
hookdeck connection upsert "my-connection" \
hookdeck gateway connection upsert "my-connection" \
--source-name "stripe-prod" --source-type STRIPE \
--destination-name "my-api" --destination-type HTTP --destination-url https://api.example.com

# Update just the rate limit on an existing connection
hookdeck connection upsert my-connection \
hookdeck gateway connection upsert my-connection \
--destination-rate-limit 100 --destination-rate-limit-period minute

# Update source configuration options
hookdeck connection upsert my-connection \
hookdeck gateway connection upsert my-connection \
--source-allowed-http-methods "POST,PUT,DELETE" \
--source-custom-response-content-type "json" \
--source-custom-response-body '{"status":"received"}'

# Preview changes without applying them
hookdeck connection upsert my-connection \
hookdeck gateway connection upsert my-connection \
--destination-rate-limit 200 --destination-rate-limit-period hour \
--dry-run
```
Expand All @@ -645,6 +664,12 @@ Enable a disabled connection.
```bash
hookdeck gateway connection enable <connection-id>
```

**Arguments:**

| Argument | Type | Description |
|----------|------|-------------|
| `connection-id` | `string` | **Required.** Connection ID |
### hookdeck gateway connection disable

Disable an active connection. It will stop receiving new events until re-enabled.
Expand All @@ -654,6 +679,12 @@ Disable an active connection. It will stop receiving new events until re-enabled
```bash
hookdeck gateway connection disable <connection-id>
```

**Arguments:**

| Argument | Type | Description |
|----------|------|-------------|
| `connection-id` | `string` | **Required.** Connection ID |
### hookdeck gateway connection pause

Pause a connection temporarily.
Expand All @@ -665,6 +696,12 @@ The connection will queue incoming events until unpaused.
```bash
hookdeck gateway connection pause <connection-id>
```

**Arguments:**

| Argument | Type | Description |
|----------|------|-------------|
| `connection-id` | `string` | **Required.** Connection ID |
### hookdeck gateway connection unpause

Resume a paused connection.
Expand All @@ -676,6 +713,12 @@ The connection will start processing queued events.
```bash
hookdeck gateway connection unpause <connection-id>
```

**Arguments:**

| Argument | Type | Description |
|----------|------|-------------|
| `connection-id` | `string` | **Required.** Connection ID |
<!-- GENERATE_END -->
## Sources

Expand Down Expand Up @@ -1864,6 +1907,7 @@ hookdeck ci [flags]
| Flag | Type | Description |
|------|------|-------------|
| `--api-key` | `string` | Your Hookdeck Project API key. The CLI reads from HOOKDECK_API_KEY if not provided. |
| `--local` | `bool` | Save credentials to current directory (.hookdeck/config.toml) |
| `--name` | `string` | Name of the CI run (ex: GITHUB_REF) for identification in the dashboard |

**Examples:**
Expand Down
Loading
Loading