fix: handle non-sequential unfinalized blocks with backfill#3016
Merged
fix: handle non-sequential unfinalized blocks with backfill#3016
Conversation
📝 WalkthroughWalkthroughAdds backfill and fork-detection logic to UnfinalizedBlocksService, integrates best-height tracking via events, changes registerUnfinalizedBlock to return a fork Header when detected, and expands tests to cover many backfill/fork scenarios and error paths. Changes
Sequence Diagram(s)sequenceDiagram
participant Unfinalized as UnfinalizedBlocksService
participant Chain as BlockchainService
participant Store as Store/Cache
participant Event as IndexerEventBus
Unfinalized->>Unfinalized: updateBestHeight({height})
Unfinalized->>Unfinalized: registerUnfinalizedBlock(header)
alt header non-sequential (gap)
Unfinalized->>Chain: getHeaderForHeight(range) (backfill)
Chain-->>Unfinalized: headers / error
alt fetch error
Unfinalized->>Unfinalized: propagate error
else headers returned
Unfinalized->>Unfinalized: validate parentHash chain
alt fork detected
Unfinalized->>Event: emit ForkDetected(forkHeader)
Unfinalized->>Store: prepare reset / drop below safeHeight
Unfinalized-->>Unfinalized: return forkHeader
else backfill success
Unfinalized->>Store: append backfilled headers
Unfinalized->>Store: append triggering header
Unfinalized->>Event: emit BlockRegistered(header)
end
end
else sequential
Unfinalized->>Store: append header
Unfinalized->>Event: emit BlockRegistered(header)
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 41c60ddf6f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
139
to
140
| if (lastUnfinalized && lastUnfinalized.blockHash !== header.parentHash) { | ||
| return header; |
There was a problem hiding this comment.
This new sequential-path check flags a fork whenever header.parentHash differs from the previous hash, but Header.parentHash is allowed to be undefined and some callers/tests pass empty values, so valid sequential blocks are now treated as forks after the first block. In those environments the indexer will repeatedly return rewind targets instead of progressing. Only perform this fork check when the new header actually has a parent hash to validate.
Useful? React with 👍 / 👎.
Comment on lines
+318
to
+320
| assert( | ||
| bestHeader.blockHeight === checkingHeader.blockHeight, | ||
| 'Expect best header and checking header to be at the same height' |
There was a problem hiding this comment.
The new assertion assumes checkingHeader and the first bestVerifiableBlocks entry always share a height, but after early fork detection in registerUnfinalizedBlock we can enter getLastCorrectFinalizedBlock with a header above finalized height (for example, latest unfinalized block has a bad parent while finalized has advanced), so this assertion throws and crashes instead of finding a valid rewind point. The previous parent-walk logic handled this case and should not be blocked by a strict height equality check.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
packages/node-core/src/indexer/unfinalizedBlocks.service.spec.ts (1)
389-394: Minor: clarify the comment about block count.The comment says
// 111 + 139 backfilled + 250but it's actually 1 (block 111) + 138 (blocks 112-249) + 1 (block 250) = 140. The math works out, but the "139 backfilled" is off by one. Consider updating for clarity.📝 Proposed fix
- expect(unfinalizedBlocks.length).toBe(140); // 111 + 139 backfilled + 250 + expect(unfinalizedBlocks.length).toBe(140); // block 111 + 138 backfilled (112-249) + block 250🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/node-core/src/indexer/unfinalizedBlocks.service.spec.ts` around lines 389 - 394, The comment on the assertion for unfinalizedBlocks length is misleading: update the inline comment next to the expect(unfinalizedBlocks.length).toBe(140) to correctly describe the components as "1 (block 111) + 138 (blocks 112-249) + 1 (block 250) = 140" so it matches the actual ranges; reference the test's unfinalizedBlocks variable in unfinalizedBlocks.service.spec.ts when making this change.packages/node-core/src/indexer/unfinalizedBlocks.service.ts (2)
173-180: Consider adding a clarifying comment about the intentional push before fork check.The header is pushed to
unfinalizedBlocks(line 174) before checking if backfill detected a fork. If a fork was detected, the blocks aren't saved (line 179 is skipped), leaving an inconsistent in-memory state. This works because the caller will trigger a reindex that resets the blocks, but the intent isn't immediately obvious.📝 Proposed comment for clarity
// Add the original header after successful backfill + // Note: We push even if fork was detected. The in-memory state will be reset + // when the caller processes the fork and triggers a reindex. this.unfinalizedBlocks.push(header);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/node-core/src/indexer/unfinalizedBlocks.service.ts` around lines 173 - 180, The code pushes the new header into this.unfinalizedBlocks before checking backfillResult.forkDetected, leaving in-memory state modified even when a fork is detected and saveUnfinalizedBlocks is skipped; add a clarifying inline comment above the this.unfinalizedBlocks.push(header) (or move the push if you prefer) stating that the push is intentional to preserve the original header for caller-triggered reindex/reset on fork detection, and reference backfillResult.forkDetected and saveUnfinalizedBlocks in the comment so future readers understand why the in-memory list is left unchanged on fork and that persistence only occurs when no fork is detected.
216-219: Useunknowninstead ofanyfor caught errors.TypeScript best practice is to use
unknownfor caught exceptions and perform type narrowing.♻️ Proposed fix
- } catch (e: any) { - logger.error(`Failed to fetch block ${height} during backfill: ${e.message}`); - throw new Error(`Failed to backfill missing unfinalized block at height ${height}: ${e.message}`); + } catch (e: unknown) { + const message = e instanceof Error ? e.message : String(e); + logger.error(`Failed to fetch block ${height} during backfill: ${message}`); + throw new Error(`Failed to backfill missing unfinalized block at height ${height}: ${message}`); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/node-core/src/indexer/unfinalizedBlocks.service.ts` around lines 216 - 219, In the catch block in unfinalizedBlocks.service.ts (the try/catch that logs and rethrows failed backfill for a given height), change the catch parameter from e: any to e: unknown and narrow it before accessing message; e.g., determine a message via `const msg = e instanceof Error ? e.message : String(e)` and then use msg in both logger.error and the thrown Error to avoid unsafe any usage while preserving the original error text.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/node-core/src/indexer/unfinalizedBlocks.service.ts`:
- Around line 318-321: The assertion in getLastCorrectFinalizedBlock wrongly
assumes bestHeader.blockHeight === checkingHeader.blockHeight; remove that
assert and instead align heights by walking the taller header backward until
heights match (use parentHash lookups in unfinalizedBlocks or the same
height-walking logic from hasForked), then compare headers; ensure you reference
forkedHeader/checkingHeader, bestVerifiableBlocks and unfinalizedBlocks when
performing the backward-walk and final parentHash comparisons so the function
handles cases where finalizedBlockNumber advanced and unfinalizedBlocks contain
mixed heights.
---
Nitpick comments:
In `@packages/node-core/src/indexer/unfinalizedBlocks.service.spec.ts`:
- Around line 389-394: The comment on the assertion for unfinalizedBlocks length
is misleading: update the inline comment next to the
expect(unfinalizedBlocks.length).toBe(140) to correctly describe the components
as "1 (block 111) + 138 (blocks 112-249) + 1 (block 250) = 140" so it matches
the actual ranges; reference the test's unfinalizedBlocks variable in
unfinalizedBlocks.service.spec.ts when making this change.
In `@packages/node-core/src/indexer/unfinalizedBlocks.service.ts`:
- Around line 173-180: The code pushes the new header into
this.unfinalizedBlocks before checking backfillResult.forkDetected, leaving
in-memory state modified even when a fork is detected and saveUnfinalizedBlocks
is skipped; add a clarifying inline comment above the
this.unfinalizedBlocks.push(header) (or move the push if you prefer) stating
that the push is intentional to preserve the original header for
caller-triggered reindex/reset on fork detection, and reference
backfillResult.forkDetected and saveUnfinalizedBlocks in the comment so future
readers understand why the in-memory list is left unchanged on fork and that
persistence only occurs when no fork is detected.
- Around line 216-219: In the catch block in unfinalizedBlocks.service.ts (the
try/catch that logs and rethrows failed backfill for a given height), change the
catch parameter from e: any to e: unknown and narrow it before accessing
message; e.g., determine a message via `const msg = e instanceof Error ?
e.message : String(e)` and then use msg in both logger.error and the thrown
Error to avoid unsafe any usage while preserving the original error text.
ℹ️ Review info
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
packages/node-core/src/indexer/unfinalizedBlocks.service.spec.tspackages/node-core/src/indexer/unfinalizedBlocks.service.ts
Comment on lines
+318
to
+321
| assert( | ||
| bestHeader.blockHeight === checkingHeader.blockHeight, | ||
| 'Expect best header and checking header to be at the same height' | ||
| ); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, get the file and look at the relevant lines and surrounding context
wc -l packages/node-core/src/indexer/unfinalizedBlocks.service.tsRepository: subquery/subql
Length of output: 121
🏁 Script executed:
# Read the file to understand the getLastCorrectFinalizedBlock method
cat -n packages/node-core/src/indexer/unfinalizedBlocks.service.ts | sed -n '300,340p'Repository: subquery/subql
Length of output: 1901
🏁 Script executed:
# Look at the entire getLastCorrectFinalizedBlock method - find its start
rg -n "getLastCorrectFinalizedBlock" packages/node-core/src/indexer/unfinalizedBlocks.service.ts -B 5 -A 30Repository: subquery/subql
Length of output: 2974
🏁 Script executed:
# Find registerUnfinalizedBlock method and lines 140 and 177
rg -n "registerUnfinalizedBlock" packages/node-core/src/indexer/unfinalizedBlocks.service.ts -A 50 | head -100Repository: subquery/subql
Length of output: 3361
🏁 Script executed:
# Find hasForked method to compare with assertion
rg -n "hasForked" packages/node-core/src/indexer/unfinalizedBlocks.service.ts -A 30Repository: subquery/subql
Length of output: 2828
🏁 Script executed:
# Find where getLastCorrectFinalizedBlock is called and what is passed
rg -n "getLastCorrectFinalizedBlock\(" packages/node-core/src/indexer/unfinalizedBlocks.service.ts -B 5 -A 2Repository: subquery/subql
Length of output: 644
🏁 Script executed:
# Check backfillBlocks and what heights it can return
rg -n "backfillBlocks" packages/node-core/src/indexer/unfinalizedBlocks.service.ts -A 40 | head -80Repository: subquery/subql
Length of output: 2893
🏁 Script executed:
# Also check what forkHeader could be from backfillResult
rg -n "forkHeader" packages/node-core/src/indexer/unfinalizedBlocks.service.ts -B 2 -A 2Repository: subquery/subql
Length of output: 826
🏁 Script executed:
# Check if there are any tests that exercise this scenario
fd -t f "\.test\.(ts|js)" packages/node-core | xargs grep -l "unfinalizedBlocks\|getLastCorrectFinalizedBlock" 2>/dev/nullRepository: subquery/subql
Length of output: 42
🏁 Script executed:
# Check the UNFINALIZED_THRESHOLD constant value
rg -n "UNFINALIZED_THRESHOLD" packages/node-core/src/indexer/unfinalizedBlocks.service.tsRepository: subquery/subql
Length of output: 260
🏁 Script executed:
# Look at the complete flow: where registerUnfinalizedBlock is called and how forkedHeader is used
rg -n "processUnfinalizedBlockHeader" packages/node-core/src/indexer/unfinalizedBlocks.service.ts -B 5 -A 25 | head -60Repository: subquery/subql
Length of output: 2445
🏁 Script executed:
# Check if there's any other code path that might adjust heights before calling getLastCorrectFinalizedBlock
rg -n "getLastCorrectFinalizedBlock" packages/node-core/src/indexer/unfinalizedBlocks.service.ts -B 10Repository: subquery/subql
Length of output: 852
🏁 Script executed:
# Check deleteFinalizedBlock to understand when blocks <= finalizedBlockNumber remain in unfinalizedBlocks
rg -n "deleteFinalizedBlock" packages/node-core/src/indexer/unfinalizedBlocks.service.ts -A 20Repository: subquery/subql
Length of output: 1916
🏁 Script executed:
# Also check the order of operations when finalizedBlockNumber changes
rg -n "finalizedBlockNumber\|_finalizedHeader" packages/node-core/src/indexer/unfinalizedBlocks.service.ts | head -30Repository: subquery/subql
Length of output: 40
🏁 Script executed:
# Get the complete flow to understand when finalizedBlockNumber is set vs when getLastCorrectFinalizedBlock is called
cat -n packages/node-core/src/indexer/unfinalizedBlocks.service.ts | sed -n '94,127p'Repository: subquery/subql
Length of output: 1345
🏁 Script executed:
# Check if registerFinalizedBlock is called before or after processUnfinalizedBlockHeader
rg -n "registerFinalizedBlock\|processUnfinalizedBlockHeader" packages/node-core/src/indexer -A 2 -B 2 --type=ts | head -100Repository: subquery/subql
Length of output: 40
🏁 Script executed:
# Final check: Look at the overall flow and understand the exact scenario
# Get more context around processUnfinalizedBlockHeader and when it's called
cat -n packages/node-core/src/indexer/unfinalizedBlocks.service.ts | sed -n '99,250p'Repository: subquery/subql
Length of output: 7288
🏁 Script executed:
# Verify: When registerUnfinalizedBlock returns a header, is it guaranteed to be > finalizedBlockNumber?
cat -n packages/node-core/src/indexer/unfinalizedBlocks.service.ts | sed -n '130,145p'Repository: subquery/subql
Length of output: 875
Assertion failure in getLastCorrectFinalizedBlock when fork detected via registerUnfinalizedBlock at lines 318-321.
The assertion assumes bestHeader and forkedHeader are at the same height, but this invariant is not guaranteed:
registerUnfinalizedBlock(line 131) only processes headers withblockHeight > finalizedBlockNumberand returns them asforkedHeader- When
finalizedBlockNumberincreases, blocks inunfinalizedBlocksmay become<= finalizedBlockNumberbut remain undeleted (cleanup only happens on line 241 when no fork is detected) getLastCorrectFinalizedBlockreceivesforkedHeader(height >finalizedBlockNumber) and filtersbestVerifiableBlocksto heights<= finalizedBlockNumber- First loop iteration:
bestHeader.blockHeight <= finalizedBlockNumberwhilecheckingHeader.blockHeight > finalizedBlockNumber→ assertion always fails
Unlike hasForked() which explicitly walks backward to match heights (lines 289–291), registerUnfinalizedBlock paths provide no such guarantee.
Scenario:
- Unfinalized blocks: [100, 101, 102, 103], finalized = 99
finalizedBlockNumberincreases to 102- Block 104 arrives with wrong parentHash → returned as
forkedHeader bestVerifiableBlocks= [100, 101, 102] (filtered fromunfinalizedBlocks)- Loop iteration 1:
bestHeader.blockHeight = 102,checkingHeader.blockHeight = 104→ assertion fails
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/node-core/src/indexer/unfinalizedBlocks.service.ts` around lines 318
- 321, The assertion in getLastCorrectFinalizedBlock wrongly assumes
bestHeader.blockHeight === checkingHeader.blockHeight; remove that assert and
instead align heights by walking the taller header backward until heights match
(use parentHash lookups in unfinalizedBlocks or the same height-walking logic
from hasForked), then compare headers; ensure you reference
forkedHeader/checkingHeader, bestVerifiableBlocks and unfinalizedBlocks when
performing the backward-walk and final parentHash comparisons so the function
handles cases where finalizedBlockNumber advanced and unfinalizedBlocks contain
mixed heights.
Contributor
Coverage report for
|
St.❔ |
Category | Percentage | Covered / Total |
|---|---|---|---|
| 🟢 | Statements | 87.39% (+17.28% 🔼) |
71001/81250 |
| 🟢 | Branches | 80.39% (+2.1% 🔼) |
3365/4186 |
| 🟡 | Functions | 65.79% (+3.29% 🔼) |
1258/1912 |
| 🟢 | Lines | 87.39% (+17.28% 🔼) |
71001/81250 |
⚠️ Details were not displayed: the report size has exceeded the limit.
Test suite run failed
Failed tests: 20/723. Failed suites: 6/116.
● MultiChain Rewind Service › The situation where notifyHandle controls the state › should handle multiple concurrent rewind requests
expect(received).toBe(expected) // Object.is equality
Expected: "complete"
Received: "normal"
304 | await tx.commit();
305 | // This can fail because the notification has already come in and the status is back to normal
> 306 | expect(multiChainRewindService2.status).toBe(MultiChainRewindStatus.Complete);
| ^
307 |
308 | await delay(notifyHandleDelay);
309 | // The last chain rollback is complete, all chains have finished rolling back.
at Object.<anonymous> (packages/node-core/src/indexer/multiChainRewind.service.test.ts:306:47)
● BaseProjectService › initializing services › succeeds with an unfinalized blocks rewind
expect(received).resolves.not.toThrow()
Received promise rejected instead of resolved
Rejected to value: [AssertionError: Expect best header and checking header to be at the same height]
483 | const reindexSpy = jest.spyOn(service, 'reindex');
484 |
> 485 | await expect(service.init()).resolves.not.toThrow();
| ^
486 |
487 | expect(reindexSpy).toHaveReturnedTimes(1);
488 | });
at expect (node_modules/expect/build/index.js:105:15)
at Object.<anonymous> (packages/node-core/src/indexer/project.service.spec.ts:485:13)
● Dictionary V1 › coreDictionary › set startHeight of this dictionary
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Dictionary V1 › coreDictionary › validateChainMeta and useDictionary
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Dictionary V1 › coreDictionary › validate dictionary with a height
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Dictionary V1 › coreDictionary › able to build queryEntryMap
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Dictionary V1 › coreDictionary › can use scoped dictionary query
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Dictionary V1 › coreDictionary › able to getDicitonaryQueryEntries
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Dictionary V1 › get metadata
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Dictionary V1 › init metadata and get metadata
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Dictionary V1 › return dictionary query result
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Dictionary V1 › should return undefined startblock height greater than dictionary last processed height
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Dictionary V1 › should use metadata last process height at end of query height
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Individual dictionary V1 test › return undefined when dictionary api failed
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Individual dictionary V1 test › limits the dictionary query to that block range
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Individual dictionary V1 test › test query the correct range
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Individual dictionary V1 test › use minimum value of event/extrinsic returned block as batch end block
ApolloError: Response not successful: Received status code 500
at new ApolloError (packages/node-core/node_modules/@apollo/client/errors/index.js:59:28)
at packages/node-core/node_modules/@apollo/client/core/QueryManager.js:823:71
at both (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:35:31)
at packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:72
at Object.then (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:24:24)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/asyncMap.js:37:49)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:68
at Array.forEach (<anonymous>)
at iterateObserversSafely (packages/node-core/node_modules/@apollo/client/utilities/observables/iteration.js:11:25)
at Object.error (packages/node-core/node_modules/@apollo/client/utilities/observables/Concast.js:76:21)
at notifySubscription (node_modules/zen-observable/lib/Observable.js:140:18)
at onNotify (node_modules/zen-observable/lib/Observable.js:179:3)
at SubscriptionObserver.error (node_modules/zen-observable/lib/Observable.js:240:7)
at handleError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:170:14)
at packages/node-core/node_modules/@apollo/client/link/http/createHttpLink.js:148:17
Cause:
ServerError: Response not successful: Received status code 500
at Object.throwServerError (packages/node-core/node_modules/@apollo/client/link/utils/throwServerError.js:19:17)
at throwServerError (packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:120:9)
at packages/node-core/node_modules/@apollo/client/link/http/parseAndCheckHttpResponse.js:176:48
● Substrate DictionaryService › should return all specVersion
expect(received).toBeGreaterThan(expected)
Matcher error: received value must be a number or bigint
Received has value: undefined
63 | const specVersions = await dictionaryService.getSpecVersions();
64 |
> 65 | expect(specVersions?.length).toBeGreaterThan(0);
| ^
66 | dictionaryService.onApplicationShutdown();
67 | }, 50000);
68 | });
at Object.<anonymous> (packages/node/src/indexer/dictionary/v1/substrateDictionaryV1.spec.ts:65:34)
● etherscan api › Deployment information › can get the deploy height of a contract
Etherscan API error: Free API access is not supported for this chain. Please upgrade your api plan for full chain coverage. https://etherscan.io/apis
24 | return undefined;
25 | }
> 26 | throw new Error(`Etherscan API error: ${data.result}`);
| ^
27 | }
28 |
29 | return data.result;
at runRequest (packages/cli/src/utils/etherscan.ts:26:11)
at fetchContractDeployHeight (packages/cli/src/utils/etherscan.ts:69:18)
at Object.<anonymous> (packages/cli/src/utils/etherscan.spec.ts:41:22)
● Promise Utils › utils.promise delay()
expect(received).toBeLessThan(expected)
Expected: < 1050
Received: 1164
10 | const millsecDiff = new Date().getTime() - start.getTime();
11 | expect(millsecDiff).toBeGreaterThanOrEqual(1000);
> 12 | expect(millsecDiff).toBeLessThan(1050);
| ^
13 | });
14 |
15 | it('utils.promise timeout()', async () => {
at Object.<anonymous> (packages/node-core/src/utils/promise.spec.ts:12:25)
Report generated by 🧪jest coverage report action from 1c4a497
There was a problem hiding this comment.
♻️ Duplicate comments (1)
packages/node-core/src/indexer/unfinalizedBlocks.service.ts (1)
337-340:⚠️ Potential issue | 🔴 CriticalHeight-equality assert can still crash fork recovery.
At Line 337,
bestHeader.blockHeight === checkingHeader.blockHeightis not guaranteed:checkingHeadermay come fromregisterUnfinalizedBlock()/backfill at a higher height than anybestVerifiableBlocksentry. This assert can terminate the process on the exact non-sequential/fork paths this PR targets.Proposed fix
- for (const bestHeader of bestVerifiableBlocks.reverse()) { - assert( - bestHeader.blockHeight === checkingHeader.blockHeight, - 'Expect best header and checking header to be at the same height' - ); + for (const bestHeader of bestVerifiableBlocks.reverse()) { + while (checkingHeader.blockHeight > bestHeader.blockHeight) { + assert(checkingHeader.parentHash, 'Expect checking header parentHash to be exist'); + checkingHeader = await this.blockchainService.getHeaderForHash(checkingHeader.parentHash); + } + + if (checkingHeader.blockHeight < bestHeader.blockHeight) { + continue; + } + if (bestHeader.blockHash === checkingHeader.blockHash || bestHeader.blockHash === checkingHeader.parentHash) { return bestHeader; } // Get the new parent assert(checkingHeader.parentHash, 'Expect checking header parentHash to be exist'); checkingHeader = await this.blockchainService.getHeaderForHash(checkingHeader.parentHash); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/node-core/src/indexer/unfinalizedBlocks.service.ts` around lines 337 - 340, The assert comparing bestHeader.blockHeight and checkingHeader.blockHeight can crash on fork/non-sequential paths (checkingHeader may be from registerUnfinalizedBlock/backfill); replace the assert with a non-fatal conditional: if heights differ, do not throw—handle gracefully by logging a warning and returning early (or otherwise following the fork-recovery branch) instead of asserting, so the code can continue processing bestVerifiableBlocks and unfinalized block backfill; update the block of logic that currently uses bestHeader and checkingHeader to guard on equality and choose the appropriate recovery path when they differ.
🧹 Nitpick comments (1)
packages/node-core/src/indexer/unfinalizedBlocks.service.spec.ts (1)
18-49: Best-height pruning test is not exercising the new event-driven path.Line 279 updates
mockBestHeight, but the service’s new logic reads best height fromupdateBestHeight()(BlockBest event), notblockchainService.getBestHeight(). This test can pass without validating the intended behavior.Suggested test adjustment
- mockBestHeight = 800; + unfinalizedBlocksService.updateBestHeight({height: 800});Also applies to: 271-286
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/node-core/src/indexer/unfinalizedBlocks.service.spec.ts` around lines 18 - 49, The test currently mutates mockBestHeight (used by BlockchainService.getBestHeight) which bypasses the service’s new event-driven path; instead, modify the test to emit the BlockBest event or call the service method updateBestHeight() with the new height so the UnfinalizedBlocks service reacts via its event handler; locate uses of mockBestHeight and replace the direct mutation (around the existing assertions) with an explicit BlockBest emission or invocation of updateBestHeight() and then await the service/state update to assert pruning behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@packages/node-core/src/indexer/unfinalizedBlocks.service.ts`:
- Around line 337-340: The assert comparing bestHeader.blockHeight and
checkingHeader.blockHeight can crash on fork/non-sequential paths
(checkingHeader may be from registerUnfinalizedBlock/backfill); replace the
assert with a non-fatal conditional: if heights differ, do not throw—handle
gracefully by logging a warning and returning early (or otherwise following the
fork-recovery branch) instead of asserting, so the code can continue processing
bestVerifiableBlocks and unfinalized block backfill; update the block of logic
that currently uses bestHeader and checkingHeader to guard on equality and
choose the appropriate recovery path when they differ.
---
Nitpick comments:
In `@packages/node-core/src/indexer/unfinalizedBlocks.service.spec.ts`:
- Around line 18-49: The test currently mutates mockBestHeight (used by
BlockchainService.getBestHeight) which bypasses the service’s new event-driven
path; instead, modify the test to emit the BlockBest event or call the service
method updateBestHeight() with the new height so the UnfinalizedBlocks service
reacts via its event handler; locate uses of mockBestHeight and replace the
direct mutation (around the existing assertions) with an explicit BlockBest
emission or invocation of updateBestHeight() and then await the service/state
update to assert pruning behavior.
ℹ️ Review info
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
packages/node-core/src/indexer/unfinalizedBlocks.service.spec.tspackages/node-core/src/indexer/unfinalizedBlocks.service.ts
There was a problem hiding this comment.
♻️ Duplicate comments (1)
packages/node-core/src/indexer/unfinalizedBlocks.service.ts (1)
336-339:⚠️ Potential issue | 🔴 CriticalRemove the same-height assert; it can crash valid fork handling paths.
Line 336 assumes
checkingHeader.blockHeight === bestHeader.blockHeight, butcheckingHeadermay start abovefinalizedBlockNumber(e.g., fromregisterUnfinalizedBlock), so this assert can fail before reconciliation.🔧 Suggested fix
- for (const bestHeader of bestVerifiableBlocks.reverse()) { - assert( - bestHeader.blockHeight === checkingHeader.blockHeight, - 'Expect best header and checking header to be at the same height' - ); + for (const bestHeader of bestVerifiableBlocks.reverse()) { + while (checkingHeader.blockHeight > bestHeader.blockHeight) { + assert(checkingHeader.parentHash, 'Expect checking header parentHash to be exist'); + checkingHeader = await this.blockchainService.getHeaderForHash(checkingHeader.parentHash); + } + + if (checkingHeader.blockHeight < bestHeader.blockHeight) { + continue; + } + if (bestHeader.blockHash === checkingHeader.blockHash || bestHeader.blockHash === checkingHeader.parentHash) { return bestHeader; }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/node-core/src/indexer/unfinalizedBlocks.service.ts` around lines 336 - 339, Remove the hard assert that compares bestHeader.blockHeight and checkingHeader.blockHeight and instead handle height mismatches gracefully: in the reconciliation logic that currently uses assert(bestHeader.blockHeight === checkingHeader.blockHeight) (references: bestHeader, checkingHeader, finalizedBlockNumber, registerUnfinalizedBlock), replace the assert with conditional logic that allows checkingHeader to be above the finalizedBlockNumber (e.g., if checkingHeader.blockHeight !== bestHeader.blockHeight then either advance/choose the higher header or skip/continue reconciliation for that branch and optionally log a debug message) so valid fork handling paths do not crash.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@packages/node-core/src/indexer/unfinalizedBlocks.service.ts`:
- Around line 336-339: Remove the hard assert that compares
bestHeader.blockHeight and checkingHeader.blockHeight and instead handle height
mismatches gracefully: in the reconciliation logic that currently uses
assert(bestHeader.blockHeight === checkingHeader.blockHeight) (references:
bestHeader, checkingHeader, finalizedBlockNumber, registerUnfinalizedBlock),
replace the assert with conditional logic that allows checkingHeader to be above
the finalizedBlockNumber (e.g., if checkingHeader.blockHeight !==
bestHeader.blockHeight then either advance/choose the higher header or
skip/continue reconciliation for that branch and optionally log a debug message)
so valid fork handling paths do not crash.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Previously, when dictionary returned non-sequential blocks that exceeded the finalized height,
the the indexer would crash with error "Unfinalized block is not sequential".
This fix:
UNFINALIZED_THRESHOLD(200) to determine if gap is too large to backfillFixes # (issue)
Type of change
Please delete options that are not relevant.
Checklist
Summary by CodeRabbit
New Features
Bug Fixes
Tests
Style