Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .changeset/add-lazy-schema.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
'@data-client/endpoint': minor
'@data-client/rest': minor
'@data-client/graphql': minor
---

Add [schema.Lazy](https://dataclient.io/rest/api/Lazy) for deferred relationship denormalization.

`schema.Lazy` wraps a relationship field so denormalization returns raw primary keys
instead of resolved entities. Use `.query` with [useQuery](/docs/api/useQuery) to
resolve on demand in a separate memo/GC scope.

New exports: `schema.Lazy`, `Lazy`

```ts
class Department extends Entity {
buildings: string[] = [];
static schema = {
buildings: new schema.Lazy([Building]),
};
}

// dept.buildings = ['bldg-1', 'bldg-2'] (raw PKs)
const buildings = useQuery(Department.schema.buildings.query, dept.buildings);
```
11 changes: 10 additions & 1 deletion .changeset/fix-denorm-depth-limit.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,22 @@
---
'@data-client/normalizr': patch
'@data-client/endpoint': patch
'@data-client/core': patch
'@data-client/react': patch
'@data-client/vue': patch
---

Fix stack overflow during denormalization of large bidirectional entity graphs.

Add entity depth limit (128) to prevent `RangeError: Maximum call stack size exceeded`
Add entity depth limit (64) to prevent `RangeError: Maximum call stack size exceeded`
when denormalizing cross-type chains with thousands of unique entities
(e.g., Department → Building → Department → ...). Entities beyond the depth limit
are returned with unresolved ids instead of fully denormalized nested objects.

The limit can be configured per-Entity with [`static maxEntityDepth`](/rest/api/Entity#maxEntityDepth):

```ts
class Department extends Entity {
static maxEntityDepth = 16;
}
```
5 changes: 3 additions & 2 deletions docs/core/api/useQuery.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ import VoteDemo from '../shared/\_VoteDemo.mdx';
Data rendering without the fetch.

Access any [Queryable Schema](/rest/api/schema#queryable)'s store value; like [Entity](/rest/api/Entity), [All](/rest/api/All), [Collection](/rest/api/Collection), [Query](/rest/api/Query),
and [Union](/rest/api/Union). If the value does not exist, returns `undefined`.
and [Union](/rest/api/Union). [Lazy](/rest/api/Lazy) fields also work via their [`.query`](/rest/api/Lazy#query) accessor.
If the value does not exist, returns `undefined`.

`useQuery()` is reactive to data [mutations](../getting-started/mutations.md); rerendering only when necessary. Returns `undefined`
when data is [Invalid](../concepts/expiry-policy#invalid).
Expand Down Expand Up @@ -59,7 +60,7 @@ function useQuery<S extends Queryable>(

[Queryable](/rest/api/schema#queryable) schemas require an `queryKey()` method that returns something. These include
[Entity](/rest/api/Entity), [All](/rest/api/All), [Collection](/rest/api/Collection), [Query](/rest/api/Query),
and [Union](/rest/api/Union).
and [Union](/rest/api/Union). [Lazy](/rest/api/Lazy) fields produce a Queryable via their [`.query`](/rest/api/Lazy#query) accessor.

```ts
interface Queryable {
Expand Down
8 changes: 7 additions & 1 deletion docs/core/shared/_schema_table.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,16 @@
<td align="center">🛑</td>
</tr>
<tr>
<td>any</td>
<td rowSpan={2}>any</td>
<td align="center"></td>
<td>[Query(Queryable)](/rest/api/Query)</td>
<td>memoized custom transforms</td>
<td align="center">✅</td>
</tr>
<tr>
<td align="center"></td>
<td>[Lazy(Schema)](/rest/api/Lazy)</td>
<td>deferred denormalization</td>
<td align="center">🛑</td>
</tr>
</tbody></table>
38 changes: 38 additions & 0 deletions docs/rest/api/Entity.md
Original file line number Diff line number Diff line change
Expand Up @@ -418,6 +418,44 @@ Nested below:
const price = useQuery(LatestPrice, { symbol: 'BTC' });
```

### static maxEntityDepth?: number {#maxEntityDepth}

Limits entity nesting depth during denormalization to prevent stack overflow
in large bidirectional entity graphs. **Default: 128**

When bidirectional relationships create chains with many unique entities
(e.g., `Department → Building → Department → ...`), denormalization can recurse
thousands of levels deep. `maxEntityDepth` truncates resolution at the specified
depth — entities beyond the limit are returned with nested foreign keys left as
unresolved ids rather than fully denormalized objects.

```typescript
class Department extends Entity {
id = '';
name = '';
buildings: Building[] = [];

pk() { return this.id; }
static key = 'Department';
// highlight-next-line
static maxEntityDepth = 16;

static schema = {
buildings: [Building],
};
}
```

:::tip

Set this on entities that participate in deep or wide bidirectional relationships.
Normal entity graphs (depth < 10) never approach the default limit.

For relationships that don't need eager denormalization, [Lazy](/rest/api/Lazy)
skips resolution entirely and lets you resolve on demand via [useQuery](/docs/api/useQuery).

:::

## Lifecycle

import Lifecycle from '../diagrams/\_entity_lifecycle.mdx';
Expand Down
134 changes: 134 additions & 0 deletions docs/rest/api/Lazy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
---
title: Lazy Schema - Deferred Relationship Denormalization
sidebar_label: Lazy
---

# Lazy

`Lazy` wraps a schema to skip eager denormalization of relationship fields. During parent entity denormalization, the field retains its raw normalized value (primary keys/IDs). The relationship can then be resolved on demand via [useQuery](/docs/api/useQuery) using the `.query` accessor.

This is useful for:
- **Large bidirectional graphs** that would overflow the call stack during recursive denormalization
- **Performance optimization** by deferring resolution of relationships that aren't always needed
- **Memoization isolation** — changes to lazy entities don't invalidate the parent's denormalized form

## Constructor

```typescript
new Lazy(innerSchema)
```

- `innerSchema`: Any [Schema](/rest/api/schema) — an [Entity](./Entity.md), an array shorthand like `[MyEntity]`, a [Collection](./Collection.md), etc.

## Usage

### Array relationship (most common)

```typescript
import { Entity, Lazy } from '@data-client/rest';

class Building extends Entity {
id = '';
name = '';
}

class Department extends Entity {
id = '';
name = '';
buildings: string[] = [];

static schema = {
buildings: new Lazy([Building]),
};
}
```

When a `Department` is denormalized, `dept.buildings` will contain raw primary keys (e.g., `['bldg-1', 'bldg-2']`) instead of resolved `Building` instances.

To resolve the buildings, use [useQuery](/docs/api/useQuery) with the `.query` accessor:

```tsx
function DepartmentBuildings({ dept }: { dept: Department }) {
// dept.buildings contains raw IDs: ['bldg-1', 'bldg-2']
const buildings = useQuery(Department.schema.buildings.query, dept.buildings);
// buildings: Building[] | undefined

if (!buildings) return null;
return (
<ul>
{buildings.map(b => <li key={b.id}>{b.name}</li>)}
</ul>
);
}
```

### Single entity relationship

```typescript
class Department extends Entity {
id = '';
name = '';
mainBuilding = '';

static schema = {
mainBuilding: new Lazy(Building),
};
}
```

```tsx
// dept.mainBuilding is a raw PK string: 'bldg-1'
const building = useQuery(
Department.schema.mainBuilding.query,
{ id: dept.mainBuilding },
);
```

When the inner schema is an [Entity](./Entity.md) (or any schema with `queryKey`), `LazyQuery` delegates to its `queryKey` — so you pass the same args you'd use to query that entity directly.

### Collection relationship

```typescript
class Department extends Entity {
id = '';
static schema = {
buildings: new Lazy(buildingsCollection),
};
}
```

```tsx
const buildings = useQuery(
Department.schema.buildings.query,
...collectionArgs,
);
```

## `.query`

Returns a `LazyQuery` instance suitable for [useQuery](/docs/api/useQuery). The `LazyQuery`:

- **`queryKey(args)`** — If the inner schema has a `queryKey` (Entity, Collection, etc.), delegates to it. Otherwise returns `args[0]` directly (for array/object schemas where you pass the raw normalized value).
- **`denormalize(input, args, unvisit)`** — Delegates to the inner schema, resolving IDs into full entity instances.

The `.query` getter always returns the same instance (cached).

## How it works

### Normalization

`Lazy.normalize` delegates to the inner schema. Entities are stored in the normalized entity tables as usual — `Lazy` has no effect on normalization.

### Denormalization (parent path)

`Lazy.denormalize` is a **no-op** — it returns the input unchanged. When `EntityMixin.denormalize` iterates over schema fields and encounters a `Lazy` field, the `unvisit` dispatch calls `Lazy.denormalize`, which simply passes through the raw PKs. No nested entities are visited, no dependencies are registered in the cache.

### Denormalization (useQuery path)

When using `useQuery(lazyField.query, ...)`, `LazyQuery.denormalize` delegates to the inner schema via `unvisit`, resolving IDs into full entity instances through the normal denormalization pipeline. This runs in its own `MemoCache.query()` scope with independent dependency tracking and GC.

## Performance characteristics

- **Parent denormalization**: Fewer dependency hops (lazy entities excluded from deps). Faster cache hits. No invalidation when lazy entities change.
- **useQuery access**: Own memo scope with own `paths` and `countRef`. Changes to lazy entities only re-render components that called `useQuery`, not the parent.
- **No Proxy/getter overhead**: Raw IDs are plain values. Full resolution only happens through `useQuery`, using the normal denormalization path.
1 change: 1 addition & 0 deletions docs/rest/api/Query.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ the same high performance and referential equality guarantees expected of Reacti
[Schema](./schema.md) used to retrieve/denormalize data from the Reactive Data Client cache.
This accepts any [Queryable](/rest/api/schema#queryable) schema: [Entity](./Entity.md), [All](./All.md), [Collection](./Collection.md), [Query](./Query.md),
[Union](./Union.md), and [Object](./Object.md) schemas for joining multiple entities.
[Lazy](./Lazy.md) fields produce a Queryable via their [`.query`](./Lazy.md#query) accessor.

### process(entries, ...args) {#process}

Expand Down
2 changes: 1 addition & 1 deletion docs/rest/api/schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,7 @@ This enables their use in these additional cases:
- Improve performance of [useSuspense](/docs/api/useSuspense), [useDLE](/docs/api/useDLE) by rendering before endpoint resolution

`Querables` include [Entity](./Entity.md), [All](./All.md), [Collection](./Collection.md), [Query](./Query.md),
and [Union](./Union.md).
and [Union](./Union.md). [Lazy](./Lazy.md) fields produce a Queryable via their [`.query`](./Lazy.md#query) accessor.

```ts
interface Queryable {
Expand Down
9 changes: 9 additions & 0 deletions docs/rest/guides/relational-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -600,6 +600,15 @@ export class User extends Entity {
}
```

:::tip

For bidirectional relationships that don't need eager denormalization,
[Lazy](../api/Lazy.md) defers resolution and lets you resolve on demand
via [useQuery](/docs/api/useQuery), avoiding deep recursion and improving
memoization isolation.

:::

[1]: ../api/Entity.md
[2]: /docs/api/useCache
[3]: ../api/Entity.md#schema
1 change: 1 addition & 0 deletions packages/endpoint/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ export {
Query,
Values,
All,
Lazy,
unshift,
} from './schema.js';
// Without this we get 'cannot be named without a reference to' for resource()....why is this?
Expand Down
3 changes: 2 additions & 1 deletion packages/endpoint/src/schema.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import {
default as Entity,
} from './schemas/EntityMixin.js';
import { default as Invalidate } from './schemas/Invalidate.js';
import { default as Lazy } from './schemas/Lazy.js';
import { default as Query } from './schemas/Query.js';
import type {
CollectionConstructor,
Expand All @@ -34,7 +35,7 @@ import type {
UnionResult,
} from './schemaTypes.js';

export { EntityMap, Invalidate, Query, EntityMixin, Entity };
export { EntityMap, Invalidate, Query, Lazy, EntityMixin, Entity };

export type { SchemaClass };

Expand Down
1 change: 1 addition & 0 deletions packages/endpoint/src/schema.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ export {
default as Entity,
} from './schemas/EntityMixin.js';
export { default as Query } from './schemas/Query.js';
export { default as Lazy } from './schemas/Lazy.js';
7 changes: 7 additions & 0 deletions packages/endpoint/src/schemas/EntityMixin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,13 @@ export default function EntityMixin<TBase extends Constructor>(
/** Defines indexes to enable lookup by */
declare static indexes?: readonly string[];

/** Maximum entity nesting depth for denormalization (default: 128)
*
* Set a lower value to truncate deep bidirectional entity graphs earlier.
* @see https://dataclient.io/rest/api/Entity#maxEntityDepth
*/
declare static maxEntityDepth?: number;

/**
* A unique identifier for each Entity
*
Expand Down
6 changes: 6 additions & 0 deletions packages/endpoint/src/schemas/EntityTypes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,12 @@ export interface IEntityClass<TBase extends Constructor = any> {
* @see https://dataclient.io/rest/api/Entity#indexes
*/
indexes?: readonly string[] | undefined;
/** Maximum entity nesting depth for denormalization (default: 128)
*
* Set a lower value to truncate deep bidirectional entity graphs earlier.
* @see https://dataclient.io/rest/api/Entity#maxEntityDepth
*/
maxEntityDepth?: number | undefined;
/**
* A unique identifier for each Entity
*
Expand Down
Loading
Loading