adobe · szergen · Mar 12, 2026 · Feb 24, 2026 · Feb 25, 2026 · Feb 25, 2026
diff --git a/CLAUDE.md b/CLAUDE.md
@@ -0,0 +1,248 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Commands
+
+```bash
+# Install all dependencies (root + all packages)
+npm install
+
+# Run all tests across all packages
+npm test
+
+# Run tests for a single package
+npm test -w packages/spacecat-shared-utils
+
+# Run integration tests (data-access only, requires Docker)
+npm run test:it -w packages/spacecat-shared-data-access
+
+# Lint all packages
+npm run lint
+
+# Lint a single package
+npm run lint -w packages/spacecat-shared-utils
+
+# Generate API docs
+npm run docs
+
+# Dry-run semantic release (on non-main branches)
+npm run semantic-release-dry
+
+# Clean node_modules and lock file
+npm run clean
+```
+
+Node >=22 <25 is required (see `.nvmrc` for exact version). The project is pure ESM (`"type": "module"`).
+
+Coverage enforcement: **100% lines/statements, 97% branches** per package (see each package's `.nycrc.json`).
+
+---
+
+## Architecture Overview
+
+This is the **foundation library layer** for all SpaceCat Node.js services. It's an npm workspaces monorepo (no lerna/nx) containing 22 packages that provide data access, API clients, utilities, and auth used by `spacecat-api-service`, `spacecat-audit-worker`, `spacecat-import-worker`, and other platform services.
+
+### Monorepo Structure
+
+```
+spacecat-shared/
+├── packages/                     # 22 npm workspace packages
+│   ├── spacecat-shared-data-access/   # Core data layer (PostgREST + Aurora PostgreSQL)
+│   ├── spacecat-shared-http-utils/    # HTTP response helpers + auth middleware
+│   ├── spacecat-shared-utils/         # General utilities (validation, S3, SQS, helpers)
+│   ├── spacecat-shared-rum-api-client/ # RUM Bundler API client
+│   ├── spacecat-shared-vault-secrets/ # HashiCorp Vault secrets (replaces helix-shared-secrets)
+│   ├── spacecat-shared-*-client/      # 13+ API clients (ahrefs, google, slack, etc.)
+│   ├── spacecat-shared-html-analyzer/ # HTML analysis utilities
+│   └── spacecat-shared-example/       # Example/template package
+├── docs/                         # Generated API docs
+├── eslint.config.js              # Root ESLint flat config
+├── .releaserc.cjs                # Root semantic-release config
+└── package.json                  # Monorepo root (private, workspaces: ./packages/*)
+```
+
+### Package Catalog
+
+| Category | Packages | Purpose |
+|----------|----------|---------|
+| **Data layer** | `data-access` | Entity models/collections backed by PostgreSQL via PostgREST |
+| **HTTP/Auth** | `http-utils` | HTTP response helpers, auth handlers (ScopedApiKey, LegacyApiKey, IMS) |
+| **Utilities** | `utils` | Validation, S3/SQS helpers, schemas, date functions |
+| **Secrets** | `vault-secrets` | HashiCorp Vault integration (AppRole auth, KV secrets) |
+| **HTML** | `html-analyzer` | HTML analysis and extraction |
+| **API clients** | `ahrefs-client`, `athena-client`, `brand-client`, `cloud-manager-client`, `content-client`, `drs-client`, `google-client`, `gpt-client`, `ims-client`, `launchdarkly-client`, `rum-api-client`, `scrape-client`, `slack-client`, `splunk-client`, `tier-client`, `tokowaka-client` | External service integrations |
+| **Example** | `example` | Template for creating new packages |
+
+### Package-Level CLAUDE.md Files
+
+Some packages have their own CLAUDE.md with deeper architectural guidance:
+
+- [`packages/spacecat-shared-data-access/CLAUDE.md`](packages/spacecat-shared-data-access/CLAUDE.md) — Entity pattern, SchemaBuilder DSL, PostgREST queries, integration tests
+- [`packages/spacecat-shared-vault-secrets/CLAUDE.md`](packages/spacecat-shared-vault-secrets/CLAUDE.md) — Vault pipeline, AppRole auth, E2E validation
+- [`packages/spacecat-shared-google-client/docs/CLAUDE.md`](packages/spacecat-shared-google-client/docs/CLAUDE.md) — GSC client, OAuth, environment variables
+
+---
+
+## Key Patterns
+
+### Per-Package File Structure
+
+Each package follows this layout:
+
+```
+packages/spacecat-shared-<name>/
+├── src/              # Source code
+│   └── index.js      # Package entry point (barrel exports)
+├── test/             # Tests (mirrors src/ structure)
+├── package.json      # Package metadata, dependencies, scripts
+├── .releaserc.cjs    # semantic-release config (extends monorepo)
+├── .nycrc.json       # c8 coverage config
+├── README.md
+└── CHANGELOG.md      # Auto-generated by semantic-release
+```
+
+### Entity Pattern (data-access)
+
+The data-access package defines entities with 4 files per entity:
+
+```
+src/models/<entity>/
+  <entity>.schema.js       # SchemaBuilder definition (attributes, references, indexes)
+  <entity>.model.js        # Extends BaseModel (business logic, constants)
+  <entity>.collection.js   # Extends BaseCollection (custom queries)
+  index.js                 # Re-exports model, collection, schema
+```
+
+See [`packages/spacecat-shared-data-access/CLAUDE.md`](packages/spacecat-shared-data-access/CLAUDE.md) for the SchemaBuilder DSL, attribute options, and field mapping details.
+
+### Field Mapping Convention
+
+Models use **camelCase**, the database uses **snake_case**. Mapping is automatic via `postgrest.utils.js`. Override with `postgrestField: 'custom_name'` on individual attributes.
+
+### TypeScript Declarations
+
+Packages provide `.d.ts` files for public APIs — no TypeScript source code. Type declarations live alongside the source files they describe.
+
+### API Client Pattern
+
+API clients typically follow this structure:
+
+```js
+export default class FooClient {
+  constructor(config, log) { /* ... */ }
+  async someOperation(params) { /* ... */ }
+}
+
+// Factory function exported from index.js
+export function createFooClient(config, log) {
+  return new FooClient(config, log);
+}
+```
+
+---
+
+## Testing
+
+**Framework:** Mocha + Chai + chai-as-promised + sinon + sinon-chai
+**Coverage:** c8 (per-package `.nycrc.json`)
+**HTTP mocking:** nock
+**Reporters:** mocha-multi-reporters (spec + xunit)
+
+### Running Tests
+
+```bash
+# All packages
+npm test
+
+# Single package
+npm test -w packages/spacecat-shared-utils
+
+# Integration tests (data-access only — requires Docker + ECR access)
+npm run test:it -w packages/spacecat-shared-data-access
+```
+
+### Test Conventions
+
+- Tests live in `test/` (or `test/unit/`) and mirror the `src/` directory structure
+- Use `describe`/`it` blocks with `expect` assertions
+- sinon stubs for external dependencies; nock for HTTP mocking
+- Each test file uses a sandbox: `const sandbox = sinon.createSandbox()` with `afterEach(() => sandbox.restore())`
+- Coverage thresholds are enforced per package
+
+### Integration Tests (data-access)
+
+Integration tests require the `mysticat-data-service` Docker image from ECR:
+
+```bash
+# ECR login (one-time)
+aws ecr get-login-password --profile spacecat-dev --region us-east-1 \
+  | docker login --username AWS --password-stdin 682033462621.dkr.ecr.us-east-1.amazonaws.com
+
+npm run test:it -w packages/spacecat-shared-data-access
+```
+
+---
+
+## Working with the Monorepo
+
+### Inter-Package Dependencies
+
+Some packages depend on others in the monorepo (e.g., `data-access` depends on `utils`). These are declared as regular npm dependencies with the published `@adobe/spacecat-shared-*` package name and version.
+
+### Running Commands for a Single Package
+
+Use the `-w` (workspace) flag:
+
+```bash
+npm test -w packages/spacecat-shared-<name>
+npm run lint -w packages/spacecat-shared-<name>
+```
+
+### Adding a New Package
+
+1. Copy `packages/spacecat-shared-example/` as a starting template
+2. Rename the directory to `spacecat-shared-<name>`
+3. Update `package.json`: name (`@adobe/spacecat-shared-<name>`), description, dependencies
+4. Update `.releaserc.cjs` (inherits from root)
+5. Implement source in `src/`, tests in `test/`
+6. Ensure 100% coverage with `.nycrc.json` thresholds
+7. The root `workspaces: ["./packages/*"]` auto-discovers it
+
+### Adding a New Entity (data-access)
+
+1. Create 4 files in `src/models/<entity>/`: `schema.js`, `model.js`, `collection.js`, `index.js`
+2. Register the entity in `src/models/index.js`
+3. The corresponding DB migration must be created in [mysticat-data-service](https://github.com/adobe/mysticat-data-service)
+4. Add unit tests in `test/unit/models/<entity>/`
+
+### Adding a New Import Type (data-access)
+
+1. Update three locations in `src/models/site/config.js`: `IMPORT_TYPES`, `IMPORT_TYPE_SCHEMAS`, `DEFAULT_IMPORT_CONFIGS`
+2. Update three corresponding assertions in `test/unit/models/site/config.test.js`
+
+See [`packages/spacecat-shared-data-access/CLAUDE.md`](packages/spacecat-shared-data-access/CLAUDE.md) for full details including code examples.
+
+---
+
+## Commits and Releases
+
+- **Conventional commits** are required — use `npm run commit` (commitizen) or write them manually
+- **Pre-commit hook** (husky + lint-staged) runs ESLint on staged `.js` files
+- **semantic-release** with `semantic-release-monorepo` handles per-package versioning
+- Release happens automatically on push to `main` — each package is released independently based on commits touching its files
+- **Branch:** `main` is the only release branch
+
+### After Publishing
+
+When a new version of a `spacecat-shared-*` package is published, update the dependency version in all consuming repos (`spacecat-api-service`, `spacecat-audit-worker`, `spacecat-import-worker`, etc.).
+
+---
+
+## Linting
+
+- **Config:** `eslint.config.js` (ESLint flat config)
+- **Base:** `@adobe/eslint-config-helix` (recommended + source + test presets)
+- **Parser:** `@babel/eslint-parser` with ESM + import assertions
+- **Test overrides:** `no-console: off`, `func-names: off`
+- **Ignored:** `.vscode`, `.idea`, `coverage`, `docs`, `vendor`, `dist`, `.releaserc.cjs`, `test/fixtures`
diff --git a/packages/spacecat-shared-data-access/CLAUDE.md b/packages/spacecat-shared-data-access/CLAUDE.md
@@ -214,6 +214,70 @@ const liveSites = await dataAccess.Site.all(
 | `S3_CONFIG_BUCKET` | No | Only for `Configuration` entity |
 | `AWS_REGION` | No | Only for `Configuration` entity |
 
+## Site Config: Import Types
+
+Site configuration lives in `src/models/site/config.js` and defines the available import types, their validation schemas, and default configs. This is one of the most frequently changed files in the package.
+
+### Adding a New Import Type
+
+Update three locations in `src/models/site/config.js`:
+
+**1. Add the constant to `IMPORT_TYPES`:**
+
+```js
+export const IMPORT_TYPES = {
+  // ... existing types
+  MY_NEW_IMPORT: 'my-new-import',
+};
+```
+
+**2. Add the Joi validation schema to `IMPORT_TYPE_SCHEMAS`:**
+
+```js
+export const IMPORT_TYPE_SCHEMAS = {
+  // ... existing schemas
+  [IMPORT_TYPES.MY_NEW_IMPORT]: Joi.object({
+    type: Joi.string().valid(IMPORT_TYPES.MY_NEW_IMPORT).required(),
+    ...IMPORT_BASE_KEYS,
+    // Add optional type-specific fields here (e.g., geo, limit, year, week)
+  }),
+};
+```
+
+`IMPORT_BASE_KEYS` includes `enabled`, `destinations`, and `sources`. Only add extra fields if the import type needs them.
+
+**3. Add the default config to `DEFAULT_IMPORT_CONFIGS`:**
+
+```js
+export const DEFAULT_IMPORT_CONFIGS = {
+  // ... existing defaults
+  'my-new-import': {
+    type: 'my-new-import',
+    destinations: ['default'],
+    sources: ['rum'],   // or 'ahrefs', 'google'
+    enabled: true,
+  },
+};
+```
+
+### Test Updates
+
+In `test/unit/models/site/config.test.js`, add:
+
+1. **Enable import test** — verify `config.enableImport('my-new-import')` produces the expected default config
+2. **Validation error string** — the long `.to.equal(...)` assertion for invalid import type errors must include the new type
+3. **Validation error details** — the `.to.eql([...])` array of expected error detail objects must include the new type
+
+### Available Sources
+
+| Source | Constant | Used by |
+|--------|----------|---------|
+| `'ahrefs'` | `IMPORT_SOURCES.AHREFS` | Keyword, traffic, backlink imports |
+| `'google'` | `IMPORT_SOURCES.GSC` | Google Search Console imports |
+| `'rum'` | `IMPORT_SOURCES.RUM` | RUM/CWV/engagement imports |
+
+---
+
 ## Special Entities
 
 - **Configuration**: S3-backed (not PostgREST). Requires `S3_CONFIG_BUCKET`.

diff --git a/packages/spacecat-shared-data-access/src/models/site/config.js b/packages/spacecat-shared-data-access/src/models/site/config.js
@@ -33,6 +33,8 @@ export const IMPORT_TYPES = {
   TOP_FORMS: 'top-forms',
   CODE: 'code',
   USER_ENGAGEMENT: 'user-engagement',
+  CWV_TRENDS_DAILY: 'cwv-trends-daily',
+  CWV_TRENDS_ONBOARD: 'cwv-trends-onboard',
 };
 
 export const IMPORT_DESTINATIONS = {
@@ -170,6 +172,14 @@ export const IMPORT_TYPE_SCHEMAS = {
     type: Joi.string().valid(IMPORT_TYPES.USER_ENGAGEMENT).required(),
     ...IMPORT_BASE_KEYS,
   }),
+  [IMPORT_TYPES.CWV_TRENDS_DAILY]: Joi.object({
+    type: Joi.string().valid(IMPORT_TYPES.CWV_TRENDS_DAILY).required(),
+    ...IMPORT_BASE_KEYS,
+  }),
+  [IMPORT_TYPES.CWV_TRENDS_ONBOARD]: Joi.object({
+    type: Joi.string().valid(IMPORT_TYPES.CWV_TRENDS_ONBOARD).required(),
+    ...IMPORT_BASE_KEYS,
+  }),
 };
 
 export const DEFAULT_IMPORT_CONFIGS = {
@@ -258,6 +268,18 @@ export const DEFAULT_IMPORT_CONFIGS = {
     sources: ['rum'],
     enabled: true,
   },
+  'cwv-trends-daily': {
+    type: 'cwv-trends-daily',
+    destinations: ['default'],
+    sources: ['rum'],
+    enabled: true,
+  },
+  'cwv-trends-onboard': {
+    type: 'cwv-trends-onboard',
+    destinations: ['default'],
+    sources: ['rum'],
+    enabled: true,
+  },
 };
 
 export const configSchema = Joi.object({