feat: cache TMDB API responses in database#39
Conversation
- New tmdb_cache table (Supabase migration) - fetchTmdbData checks cache first, returns cached data if <30 days old - Cache key: IMDB ID or SHA256 of cleaned title - Misses cached too (avoids repeated lookups for non-existent content) - Cache writes are non-blocking and failure-tolerant - Extracted cleanTitleForSearch as reusable function Saves 2-3 TMDB API calls per page view for previously-seen content.
There was a problem hiding this comment.
Pull request overview
Adds a database-backed cache for TMDB enrichment results so torrent/DHT detail pages avoid repeating the same 2–3 TMDB API calls on subsequent loads.
Changes:
- Introduces a new Supabase table (
tmdb_data) intended to store TMDB lookup results keyed by IMDB ID or title-hash. - Updates
fetchTmdbData()to read from the cache first and to upsert results (including misses) back into the cache. - Refactors title-cleaning logic into a helper for TMDB title search fallback.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| supabase/migrations/20260303030000_tmdb_data.sql | Creates the TMDB cache table and RLS policies/index intended to support cached lookups. |
| src/lib/imdb/tmdb.ts | Implements cache keying, Supabase read/write, and integrates cache lookup + persistence into TMDB enrichment. |
Comments suppressed due to low confidence (7)
src/lib/imdb/tmdb.ts:179
- This miss-path caches
EMPTYusingawait setCache(...), which adds a DB round-trip to the request even though the cache write is non-essential. If you want misses cached without impacting response latency, consider not awaiting this write (or performing it in a background job).
if (!tmdbId) {
// Cache the miss too (avoid repeated lookups for non-existent content)
await setCache(cacheKey, EMPTY);
return EMPTY;
supabase/migrations/20260303030000_tmdb_data.sql:20
updated_atis intended for freshness/cleanup, but this migration doesn't add an update trigger (and the app upsert doesn't setupdated_at). As a result,updated_atwill not advance on updates, making TTL/cleanup inaccurate. Consider adding aBEFORE UPDATEtrigger using the existingupdate_updated_at_column()helper (or otherwise ensureupdated_atis set on every update/upsert).
-- Track freshness
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now()
);
-- Index for cleanup of stale entries
CREATE INDEX idx_tmdb_data_updated_at ON tmdb_data (updated_at);
supabase/migrations/20260303030000_tmdb_data.sql:6
- PR description/migration naming look inconsistent: the PR describes a new
tmdb_cachetable, but this migration createstmdb_dataand the code reads/writestmdb_data. Please align the PR description and/or table name so deploy/migration/docs match what the app expects.
CREATE TABLE IF NOT EXISTS tmdb_data (
-- Lookup key: either an IMDB ID (tt1234567) or a cleaned title hash
lookup_key text PRIMARY KEY,
tmdb_id integer,
src/lib/imdb/tmdb.ts:31
- The cache key for title-only lookups hashes
titleHintdirectly, but the TMDB fallback search uses a cleaned title (cleanTitleForSearch). This mismatch means semantically identical titles with different filename noise will generate different cache keys and reduce cache hit rate (and it diverges from the stated “cleaned title hash” strategy). Consider hashing the cleaned title (and/or normalizing IMDB IDs) when building the cache key.
function getCacheKey(imdbId: string, titleHint?: string): string {
if (imdbId) return imdbId;
if (titleHint) return 'title:' + createHash('sha256').update(titleHint.toLowerCase().trim()).digest('hex').slice(0, 32);
return '';
src/lib/imdb/tmdb.ts:52
- Cache TTL isn’t enforced:
getCached()returns any row it finds without checkingcreated_at/updated_atfreshness, so entries can live forever (including cached misses). If the intended behavior is “<30 days old”, select the timestamp and ignore/overwrite stale rows (and/or add DB-side cleanup).
async function getCached(key: string): Promise<TmdbData | null> {
if (!key) return null;
try {
const supabase = getSupabaseClient();
if (!supabase) return null;
const { data } = await supabase
.from('tmdb_data')
.select('*')
.eq('lookup_key', key)
.single();
if (!data) return null;
src/lib/imdb/tmdb.ts:85
setCache()upserts without settingupdated_at, so even if you later add freshness checks based onupdated_at, updates won’t refresh the timestamp unless there’s a trigger. Setupdated_atexplicitly on upsert (or add the table trigger) so the cache entry’s age reflects the last write.
async function setCache(key: string, data: TmdbData): Promise<void> {
if (!key) return;
try {
const supabase = getSupabaseClient();
if (!supabase) return;
await supabase
.from('tmdb_data')
.upsert({
lookup_key: key,
tmdb_id: data.tmdbId,
poster_url: data.posterUrl,
backdrop_url: data.backdropUrl,
overview: data.overview,
tagline: data.tagline,
cast_names: data.cast,
writers: data.writers,
content_rating: data.contentRating,
}, { onConflict: 'lookup_key' });
src/lib/imdb/tmdb.ts:226
- Cache writes are awaited (
await setCache(...)) on both hits and misses, which makes the request wait for an extra database round-trip after the TMDB calls. If the intent is “non-blocking cache writes”, consider firing the write without awaiting it (or queueing it) so page/API latency isn’t increased by the cache update.
const result: TmdbData = { posterUrl, backdropUrl, overview, tagline, cast, writers, contentRating, tmdbId };
// Cache the result
await setCache(cacheKey, result);
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| -- Public read/write (no user-scoping needed, this is shared cache) | ||
| CREATE POLICY "tmdb_data_public_read" ON tmdb_data FOR SELECT USING (true); | ||
| CREATE POLICY "tmdb_data_service_write" ON tmdb_data FOR ALL USING (true); |
There was a problem hiding this comment.
The write policy tmdb_data_service_write is FOR ALL USING (true), which effectively allows any role with table privileges (including anon/authenticated) to modify this shared table. To keep this a server-managed cache, restrict writes to the service role (e.g., USING (auth.role() = 'service_role') and add WITH CHECK), and keep the public policy as SELECT-only.
| -- Public read/write (no user-scoping needed, this is shared cache) | |
| CREATE POLICY "tmdb_data_public_read" ON tmdb_data FOR SELECT USING (true); | |
| CREATE POLICY "tmdb_data_service_write" ON tmdb_data FOR ALL USING (true); | |
| -- Public read (no user-scoping needed, this is shared cache) | |
| CREATE POLICY "tmdb_data_public_read" ON tmdb_data FOR SELECT USING (true); | |
| CREATE POLICY "tmdb_data_service_write" | |
| ON tmdb_data | |
| FOR INSERT, UPDATE, DELETE | |
| USING (auth.role() = 'service_role') | |
| WITH CHECK (auth.role() = 'service_role'); |
src/lib/imdb/tmdb.ts
Outdated
| const url = process.env.NEXT_PUBLIC_SUPABASE_URL || process.env.SUPABASE_URL; | ||
| const key = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY; |
There was a problem hiding this comment.
getSupabaseClient() falls back to NEXT_PUBLIC_SUPABASE_ANON_KEY when the service role key is absent. In combination with a permissive RLS policy, this enables public writes to the cache table. Since this module is used server-side, it should require the service role key (or use the repo’s existing server client helper) and avoid any NEXT_PUBLIC_* key fallback for writes.
| const url = process.env.NEXT_PUBLIC_SUPABASE_URL || process.env.SUPABASE_URL; | |
| const key = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY; | |
| const url = process.env.SUPABASE_URL || process.env.NEXT_PUBLIC_SUPABASE_URL; | |
| const key = process.env.SUPABASE_SERVICE_ROLE_KEY; |
Addresses Copilot review: 1. RLS policy now restricts writes to service_role only 2. getSupabaseClient() requires SUPABASE_SERVICE_ROLE_KEY (no anon key fallback)
Summary
Caches TMDB API responses in a new
tmdb_cacheSupabase table to avoid repeated API calls for the same content. Every /dht/ and /torrents/ detail page was making 2-3 TMDB API calls on every load.How it works
fetchTmdbData()checkstmdb_cachetable firstCache key strategy
tt1234567) when availableMigration
supabase/migrations/20260303030000_tmdb_cache.sql— already applied to production.Impact
Testing