fix: wider spaces between leaderboard copy pipes to prevent hitting queue limits#3868
fix: wider spaces between leaderboard copy pipes to prevent hitting queue limits#3868
Conversation
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
3 similar comments
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| TARGET_DATASOURCE leaderboards_copy_ds | ||
| COPY_MODE append | ||
| COPY_SCHEDULE 15 1 * * * | ||
| COPY_SCHEDULE 0 2 * * * |
There was a problem hiding this comment.
New schedule collision with existing daily copy pipe
Medium Severity
leaderboards_merge_time.pipe was moved to COPY_SCHEDULE 0 2 * * *, which is the exact same schedule as the existing project_insights_copy.pipe (also 0 2 * * *). This introduces a new concurrent execution that didn't exist before (it was previously at 15 1 * * *), working against the PR's goal of reducing queue pressure. Several hourly copy pipes also fire at :00, compounding the overlap at the 2:00 AM slot.
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
| Push only files containing "leaderboards_" in their name | ||
|
|
||
| EOF | ||
| exit 0 |
There was a problem hiding this comment.
Error paths exit with success status code
Low Severity
show_help unconditionally calls exit 0, but it's also invoked from the error paths for unknown options and missing --match argument. This causes the script to report success even when it fails to parse arguments and performs no work. If used in automation or chained with &&, the silent success on error could mask a failure to push any pipes.
Additional Locations (2)
2 participants
Note
Low Risk
Only changes scheduled execution times for COPY pipes and adds a developer script; risk is mainly mis-timing or missed cadence if cron expressions are wrong.
Overview
Adjusts multiple leaderboard Tinybird pipes to spread out
COPY_SCHEDULEs (moving several jobs from the same minute/hour to later minutes and into subsequent hours) to avoid copy queue contention/limits.Adds
services/libs/tinybird/scripts/push.sh, a small CLI helper to runtb pushover thepipes/directory with an optional--matchfilter for pushing only a subset of pipes.Written by Cursor Bugbot for commit 0746782. This will update automatically on new commits. Configure here.