Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/nine-eagles-fetch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@core/sync-service': patch
---

Fix Materializer crash that was happening when processing changes with PK updates.
Original file line number Diff line number Diff line change
Expand Up @@ -377,24 +377,28 @@ defmodule Electric.Shapes.Consumer.Materializer do

%Changes.UpdatedRecord{
key: key,
old_key: old_key,
record: record,
move_tags: move_tags,
removed_move_tags: removed_move_tags
},
{{index, tag_indices}, counts_and_events} ->
# When the primary key doesn't change, old_key may be nil; default to key
old_key = old_key || key

# TODO: this is written as if it supports multiple selected columns, but it doesn't for now
columns_present = Enum.any?(state.columns, &is_map_key(record, &1))
has_tag_updates = removed_move_tags != []

if columns_present or has_tag_updates do
tag_indices =
tag_indices
|> remove_row_from_tag_indices(key, removed_move_tags)
|> remove_row_from_tag_indices(old_key, removed_move_tags)
|> add_row_to_tag_indices(key, move_tags)
Comment on lines +395 to 396

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Drop old key from unchanged tags on PK updates

When an UpdatedRecord changes the primary key but keeps the same move tag(s), removed_move_tags is empty, so this branch never removes old_key from tag_indices and only adds key. That leaves stale keys in the tag index while index has already been rewritten to the new key (Map.pop!/Map.put below), so a later move-out event can attempt Map.pop!(index, old_key) and crash the materializer. This affects shapes with dependency tags where PK updates occur without tag changes.

Useful? React with 👍 / 👎.


if columns_present do
{value, original_string} = cast!(record, state)
old_value = Map.fetch!(index, key)
{old_value, index} = Map.pop!(index, old_key)
index = Map.put(index, key, value)

# Skip decrement/increment dance if value hasn't changed to avoid
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -350,6 +350,60 @@ defmodule Electric.Shapes.Consumer.MaterializerTest do
end)
end

@tag snapshot_data: {
[%Changes.NewRecord{record: %{"id" => "1", "value" => "10"}}],
[pk_cols: ["id"]]
}
test "update that changes the primary key is handled correctly", ctx do
ctx = with_materializer(ctx)

assert Materializer.get_link_values(ctx) == MapSet.new([10])

# Update where the PK changes from "1" to "2"
Materializer.new_changes(
ctx,
[
%Changes.UpdatedRecord{
record: %{"id" => "2", "value" => "20"},
old_record: %{"id" => "1", "value" => "10"}
}
]
|> prep_changes()
)

assert Materializer.get_link_values(ctx) == MapSet.new([20])

assert_receive {:materializer_changes, _, %{move_out: [{10, "10"}], move_in: [{20, "20"}]}}
end

@tag snapshot_data: {
[%Changes.NewRecord{record: %{"id" => "1", "value" => "10"}}],
[pk_cols: ["id"]]
}
test "update that changes the primary key but keeps the same value", ctx do
ctx = with_materializer(ctx)

assert Materializer.get_link_values(ctx) == MapSet.new([10])

# Update where the PK changes but tracked value stays the same
Materializer.new_changes(
ctx,
[
%Changes.UpdatedRecord{
record: %{"id" => "2", "value" => "10"},
old_record: %{"id" => "1", "value" => "10"}
}
]
|> prep_changes()
)

# Value should still be present
assert Materializer.get_link_values(ctx) == MapSet.new([10])

# No events since the tracked value didn't change
refute_received {:materializer_changes, _, _}
end

test "events are accumulated across uncommitted fragments", ctx do
ctx = with_materializer(ctx)

Expand Down