Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions .github/workflows/community-dashboard.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
name: Update Community Dashboard Metrics

on:
schedule:
- cron: "0 6 * * *" # 6 AM UTC daily
workflow_dispatch:

permissions:
contents: write

defaults:
run:
working-directory: community-dashboard/strands-metrics

jobs:
update-metrics:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
lfs: true

- name: Install Rust Toolchain
uses: dtolnay/rust-toolchain@stable

- name: Cache Cargo registry
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
community-dashboard/strands-metrics/target
key: ${{ runner.os }}-cargo-${{ hashFiles('community-dashboard/strands-metrics/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-

- name: Sync GitHub Data
env:
GITHUB_TOKEN: ${{ secrets.METRICS_PAT }}
RUST_LOG: info
run: cargo run --release -- sync

- name: Garbage Collection (Sweep)
env:
GITHUB_TOKEN: ${{ secrets.METRICS_PAT }}
RUST_LOG: info
run: cargo run --release -- sweep

- name: Sync Package Downloads (PyPI/npm)
env:
RUST_LOG: info
run: cargo run --release -- sync-downloads

- name: Load Goals Configuration
run: cargo run --release -- load-goals

- name: Load Team Configuration
run: cargo run --release -- load-team

- name: Commit and Push to Main
working-directory: .
run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "github-actions[bot]@users.noreply.github.com"
git add community-dashboard/metrics.db
git commit -m "chore: update community-dashboard metrics.db for $(date +'%Y-%m-%d')" || echo "No changes to commit"
git push origin main
4 changes: 4 additions & 0 deletions community-dashboard/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
cdk/cdk.out
cdk/node_modules
target/
.git
16 changes: 16 additions & 0 deletions community-dashboard/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
metrics.db
strands-metrics/target/

# CDK
cdk/node_modules/
cdk/cdk.out/
cdk/dist/
cdk/.env

# Docker (ignore DB snapshots but keep the directory)
docker/data/*
!docker/data/.gitkeep

# IDE
.idea/
.DS_Store
276 changes: 276 additions & 0 deletions community-dashboard/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,276 @@
# Community Dashboard

GitHub metrics dashboards for the `strands-agents` organization. Collects data from GitHub, PyPI, and npm into a SQLite database, then visualizes it through pre-built Grafana dashboards.

Deployable locally via Docker Compose or to AWS via CDK (Fargate + EFS + CloudFront).

## Directory Structure

```
community-dashboard/
├── strands-metrics/ # Rust CLI project
│ ├── Cargo.toml
│ ├── Cargo.lock
│ └── src/
│ ├── main.rs # CLI entry point (sync, sweep, query, etc.)
│ ├── client.rs # GitHub API client (octocrab)
│ ├── db.rs # SQLite schema & initialization
│ ├── downloads.rs # PyPI/npm download tracking
│ ├── goals.rs # Goal thresholds & team management
│ └── aggregates.rs # Daily metric computation
├── goals.yaml # Configurable goal thresholds
├── team.yaml # Team members for performance tracking
├── packages.yaml # Package-to-registry mappings
├── docker/
│ ├── Dockerfile # Unified Grafana + metrics-sync image
│ ├── docker-compose.local.yaml # Local dev with auto-sync
│ ├── entrypoint.sh # Container startup script
│ └── sync-all.sh # Full sync pipeline (used by cron)
├── provisioning/
│ ├── datasources/
│ │ └── automatic.yaml # SQLite datasource config
│ └── dashboards/
│ ├── dashboards.yaml # Dashboard folder provider config
│ ├── general/ # Top-level dashboards
│ │ ├── executive.json # Executive Summary
│ │ └── health.json # Org Health
│ ├── sdks/ # SDK-specific dashboards
│ │ ├── evals.json # Evaluations
│ │ ├── python-sdk.json # Python SDK
│ │ └── typescript-sdk.json # TypeScript SDK
│ └── operations/ # Operations dashboards
│ ├── team.json # Team Performance
│ └── triage.json # Triage
└── cdk/ # AWS CDK deployment stack
├── bin/app.ts
├── lib/community-dashboard-stack.ts
├── package.json
└── cdk.json
```

## Quick Start

### Local dev with auto-sync (Docker)

Build the unified image that syncs GitHub data automatically:

```bash
GITHUB_TOKEN=ghp_xxx docker compose -f docker/docker-compose.local.yaml up --build
```

Open [http://localhost:3000](http://localhost:3000). This builds the Rust CLI, runs an initial sync on startup, and schedules daily updates at 06:00 UTC via supercronic.

### Local dev with auto-sync (Podman)

Podman works as a drop-in replacement. Build and run manually:

```bash
# Build the image
podman build -t community-dashboard -f docker/Dockerfile .

# Run the container
podman run -d --name community-dashboard \
-p 3000:3000 \
-e GITHUB_TOKEN=ghp_xxx \
-v community-dashboard-data:/var/lib/grafana/data \
community-dashboard
```

Or use `podman-compose` with the existing compose file:

```bash
GITHUB_TOKEN=ghp_xxx podman-compose -f docker/docker-compose.local.yaml up --build
```

To persist data across container restarts, the named volume `community-dashboard-data` stores `metrics.db` at `/var/lib/grafana/data`. You can also bind-mount a local directory instead:

```bash
mkdir -p docker/data
podman run -d --name community-dashboard \
-p 3000:3000 \
-e GITHUB_TOKEN=ghp_xxx \
-v ./docker/data:/var/lib/grafana/data:Z \
community-dashboard
```

The `:Z` suffix is needed on SELinux-enabled systems (Fedora, RHEL) to relabel the mount for container access.

### Standalone CLI

Build and run `strands-metrics` directly:

```bash
cd strands-metrics

# Build
cargo build --release

# Sync GitHub data (PRs, issues, stars, commits, CI runs, reviews)
GITHUB_TOKEN=ghp_xxx cargo run --release -- sync

# Garbage collection (reconcile stale open items)
GITHUB_TOKEN=ghp_xxx cargo run --release -- sweep

# Sync PyPI/npm download stats
cargo run --release -- sync-downloads

# Load goal thresholds into the database
cargo run --release -- load-goals

# Load team members for the Team dashboard
cargo run --release -- load-team

# Backfill historical downloads (PyPI: ~180 days, npm: ~365 days)
cargo run --release -- backfill-downloads

# Run arbitrary SQL queries
cargo run --release -- query "SELECT repo, SUM(prs_merged) FROM daily_metrics GROUP BY repo"
```

## CLI Commands

| Command | Description |
|---------|-------------|
| `sync` | Incremental sync of GitHub data (PRs, issues, stars, commits, CI, reviews, comments) |
| `sweep` | Garbage collection -- checks open items against GitHub and marks missing ones as deleted |
| `query <sql>` | Run raw SQL against the metrics database |
| `load-goals [path]` | Load goal thresholds from YAML into the database (default: `../goals.yaml`) |
| `list-goals` | Display all configured goal thresholds |
| `load-team [path]` | Load team members from YAML or `--members alice,bob` (default: `../team.yaml`) |
| `sync-downloads` | Sync recent package downloads from PyPI and npm (default: 30 days) |
| `backfill-downloads` | Backfill historical download data (PyPI: ~180 days, npm: ~365 days) |

### Global flags

| Flag | Default | Description |
|------|---------|-------------|
| `--db-path` / `-d` | `../metrics.db` | Path to the SQLite database file |

## Dashboards

### General

- **Executive Summary** -- High-level org overview: total stars, open PRs/issues, stale PR count, contributor trends
- **Health** -- Org health metrics with goal lines: merge time, cycle time, CI failure rate, community PR %, contributor retention, response times

### SDKs

- **Python SDK** -- Python SDK-specific metrics: PRs, issues, stars, downloads from PyPI
- **TypeScript SDK** -- TypeScript SDK metrics with npm download tracking
- **Evaluations** -- Evals framework metrics

### Operations

- **Team Performance** -- Per-member activity tracking: PRs opened/merged, reviews given, issues closed
- **Triage** -- Open issues and PRs requiring attention, sorted by staleness

## Configuration

### goals.yaml

Defines target thresholds that appear as goal lines on Health dashboard panels:

```yaml
goals:
avg_merge_time_hours:
value: 24
label: "Goal (24h)"
direction: lower_is_better # green below, red above
# warning_ratio: 0.75 # optional, default varies by direction
```

Each goal requires:
- `value` -- The target threshold
- `label` -- Display label for the goal line
- `direction` -- `lower_is_better` or `higher_is_better`
- `warning_ratio` -- (optional) Multiplier for warning threshold (default: 0.75 for lower, 0.70 for higher)

### team.yaml

Lists team members tracked in the Team Performance dashboard:

```yaml
members:
- username: alice
- username: bob
```

### packages.yaml

Maps GitHub repos to their published packages for download tracking:

```yaml
repo_mappings:
sdk-python:
- package: strands-agents
registry: pypi
sdk-typescript:
- package: "@strands-agents/sdk"
registry: npm
```

## AWS Deployment (CDK)

### Prerequisites

1. AWS CLI configured with appropriate credentials
2. Node.js 18+
3. A GitHub PAT stored in AWS Secrets Manager:
```bash
aws secretsmanager create-secret \
--name strands-grafana/github-token \
--secret-string "ghp_xxx" \
--region us-west-2
```

### Deploy

```bash
cd cdk
cp .env.example .env
# Edit .env with your GITHUB_SECRET_ARN

npm install
npx cdk deploy
```

### Architecture

```
CloudFront (HTTPS) -> ALB (HTTP:80) -> ECS Fargate -> Grafana + strands-metrics -> EFS (metrics.db)
```

- **CloudFront** for HTTPS without needing ACM + custom domain
- **EFS with RETAIN policy** so metrics survive redeployments
- **Fargate** (0.5 vCPU / 1 GB) with daily cron via supercronic
- **Anonymous viewer-only** Grafana with `ALLOW_EMBEDDING=true` for iframes

## GitHub Actions Workflow

The included workflow (`.github/workflows/community-dashboard.yaml`) runs daily at 06:00 UTC:

1. Syncs GitHub data (PRs, issues, stars, commits, CI, reviews)
2. Runs garbage collection (sweep)
3. Syncs PyPI/npm download stats
4. Loads goals and team configuration
5. Commits the updated `metrics.db` back to the repository

Required secret: `METRICS_PAT` -- a GitHub PAT with read access to the `strands-agents` org.

## Data Flow

```
GitHub API (octocrab) PyPI Stats API npm Registry API
| | |
v v v
strands-metrics CLI (Rust)
|
v
metrics.db (SQLite)
|
v
Grafana (SQLite datasource plugin)
|
v
7 pre-built dashboards in 3 folders
```
3 changes: 3 additions & 0 deletions community-dashboard/cdk/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# ARN of the Secrets Manager secret holding the GitHub personal access token.
# The secret value should be a plain-text token (not JSON).
GITHUB_SECRET_ARN=arn:aws:secretsmanager:us-west-2:ACCOUNT:secret:strands-grafana/github-token
24 changes: 24 additions & 0 deletions community-dashboard/cdk/bin/app.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/usr/bin/env node
import "source-map-support/register";
import * as dotenv from "dotenv";
import * as cdk from "aws-cdk-lib";
import { CommunityDashboardStack } from "../lib/community-dashboard-stack";

// Load environment variables from .env file (if present)
dotenv.config();

const app = new cdk.App();

const env = {
account: process.env.CDK_DEFAULT_ACCOUNT,
region: process.env.CDK_DEFAULT_REGION ?? process.env.AWS_REGION ?? "us-west-2",
};

new CommunityDashboardStack(app, "CommunityDashboardStack", {
env,
description:
"Community Dashboard — GitHub metrics collection and dashboards for strands-agents org",
tags: {
Project: "community-dashboard",
},
});
Loading