Add heatmap analytics for job activity visualization#463
Merged
dereuromark merged 2 commits intomasterfrom Mar 18, 2026
Merged
Conversation
- Add getHeatmapData() method to QueuedJobsTable for aggregating jobs by day/hour - Add heatmap action to QueuedJobsController with filters for metric, days, and job type - Add heatmapColor() helper method for color gradient - Create heatmap template with summary stats and interactive grid - Add navigation links between stats and heatmap views
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #463 +/- ##
============================================
+ Coverage 76.50% 77.54% +1.03%
- Complexity 853 898 +45
============================================
Files 44 44
Lines 2933 3095 +162
============================================
+ Hits 2244 2400 +156
- Misses 689 695 +6 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an admin “heatmap” view for Queue job activity to visualize throughput by day-of-week and hour, alongside existing time-series statistics.
Changes:
- Adds a new
QueuedJobsController::heatmap()action and a correspondingtemplates/Admin/QueuedJobs/heatmap.phpview with filters and summary stats. - Implements
QueuedJobsTable::getHeatmapData()to aggregate counts into a 7×24 grid across multiple DB drivers. - Adds navigation links (dashboard + stats page) and a
QueueHelper::heatmapColor()helper for GitHub-style intensity coloring.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| templates/Admin/QueuedJobs/stats.php | Adds navigation card to switch between time series and heatmap views. |
| templates/Admin/QueuedJobs/heatmap.php | New heatmap visualization page with filters, legend, and tooltip-enabled grid. |
| templates/Admin/Queue/index.php | Adds quick links to “Time Series” and “Heatmap” from the dashboard. |
| src/View/Helper/QueueHelper.php | Adds heatmapColor() to compute color buckets based on intensity. |
| src/Model/Table/QueuedJobsTable.php | Adds getHeatmapData() aggregation and summary computation. |
| src/Controller/Admin/QueuedJobsController.php | Adds heatmap() endpoint with query param validation and view variables. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+865
to
+1001
| /** | ||
| * Returns heatmap data aggregated by day of week and hour. | ||
| * | ||
| * @param string $field Field to aggregate on ('created' or 'completed') | ||
| * @param int $days Number of days to look back | ||
| * @param string|null $jobTask Filter by specific job task | ||
| * | ||
| * @return array{grid: array<int, array<int, int>>, summary: array<string, mixed>} | ||
| */ | ||
| public function getHeatmapData(string $field = 'created', int $days = 30, ?string $jobTask = null): array { | ||
| $driverName = $this->getDriverName(); | ||
| $since = (new DateTime())->subDays($days); | ||
|
|
||
| $query = $this->find(); | ||
|
|
||
| // Build day of week and hour expressions based on driver | ||
| switch ($driverName) { | ||
| case static::DRIVER_POSTGRES: | ||
| $dayOfWeek = $query->newExpr("EXTRACT(DOW FROM {$field})"); | ||
| $hourOfDay = $query->newExpr("EXTRACT(HOUR FROM {$field})"); | ||
|
|
||
| break; | ||
| case static::DRIVER_SQLSERVER: | ||
| $dayOfWeek = $query->newExpr("DATEPART(WEEKDAY, {$field}) - 1"); | ||
| $hourOfDay = $query->newExpr("DATEPART(HOUR, {$field})"); | ||
|
|
||
| break; | ||
| case static::DRIVER_SQLITE: | ||
| $dayOfWeek = $query->newExpr("CAST(strftime('%w', {$field}) AS INTEGER)"); | ||
| $hourOfDay = $query->newExpr("CAST(strftime('%H', {$field}) AS INTEGER)"); | ||
|
|
||
| break; | ||
| default: // MySQL | ||
| $dayOfWeek = $query->newExpr("DAYOFWEEK({$field})"); | ||
| $hourOfDay = $query->newExpr("HOUR({$field})"); | ||
|
|
||
| break; | ||
| } | ||
|
|
||
| $conditions = [ | ||
| $field . ' IS NOT' => null, | ||
| $field . ' >=' => $since, | ||
| ]; | ||
| if ($jobTask) { | ||
| $conditions['job_task'] = $jobTask; | ||
| } | ||
|
|
||
| /** @var array<array{day_of_week: int, hour_of_day: int, job_count: int}> $results */ | ||
| $results = $query | ||
| ->select([ | ||
| 'day_of_week' => $dayOfWeek, | ||
| 'hour_of_day' => $hourOfDay, | ||
| 'job_count' => $query->func()->count('*'), | ||
| ]) | ||
| ->where($conditions) | ||
| ->groupBy(['day_of_week', 'hour_of_day']) | ||
| ->disableHydration() | ||
| ->toArray(); | ||
|
|
||
| // Initialize 7x24 grid (days x hours) with zeros | ||
| $grid = []; | ||
| for ($day = 0; $day < 7; $day++) { | ||
| $grid[$day] = array_fill(0, 24, 0); | ||
| } | ||
|
|
||
| // Populate grid with results | ||
| $totalJobs = 0; | ||
| $peakCount = 0; | ||
| $peakDay = 0; | ||
| $peakHour = 0; | ||
|
|
||
| foreach ($results as $row) { | ||
| $day = (int)$row['day_of_week']; | ||
| $hour = (int)$row['hour_of_day']; | ||
| $count = (int)$row['job_count']; | ||
|
|
||
| // MySQL DAYOFWEEK returns 1=Sunday, 2=Monday, etc. Adjust to 0=Sunday | ||
| if ($driverName === static::DRIVER_MYSQL) { | ||
| $day = $day - 1; | ||
| } | ||
|
|
||
| // Ensure day is in valid range (0-6) | ||
| $day = max(0, min(6, $day)); | ||
|
|
||
| $grid[$day][$hour] = $count; | ||
| $totalJobs += $count; | ||
|
|
||
| if ($count > $peakCount) { | ||
| $peakCount = $count; | ||
| $peakDay = $day; | ||
| $peakHour = $hour; | ||
| } | ||
| } | ||
|
|
||
| // Calculate summary statistics | ||
| $dayNames = ['Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday']; | ||
| $busiestDay = $dayNames[0]; | ||
| $dayCounts = []; | ||
| foreach ($grid as $day => $hours) { | ||
| $dayCounts[$day] = array_sum($hours); | ||
| } | ||
| $maxDayCount = 0; | ||
| foreach ($dayCounts as $day => $count) { | ||
| if ($count > $maxDayCount) { | ||
| $maxDayCount = $count; | ||
| $busiestDay = $dayNames[$day]; | ||
| } | ||
| } | ||
|
|
||
| // Find quietest hour | ||
| $quietestCount = PHP_INT_MAX; | ||
| $quietestDay = 0; | ||
| $quietestHour = 0; | ||
| foreach ($grid as $day => $hours) { | ||
| foreach ($hours as $hour => $count) { | ||
| if ($count < $quietestCount) { | ||
| $quietestCount = $count; | ||
| $quietestDay = $day; | ||
| $quietestHour = $hour; | ||
| } | ||
| } | ||
| } | ||
|
|
||
| return [ | ||
| 'grid' => $grid, | ||
| 'summary' => [ | ||
| 'total' => $totalJobs, | ||
| 'avgPerHour' => $totalJobs > 0 ? round($totalJobs / ($days * 24), 1) : 0, | ||
| 'peakHour' => sprintf('%s %02d:00', $dayNames[$peakDay], $peakHour), | ||
| 'peakCount' => $peakCount, | ||
| 'quietestHour' => sprintf('%s %02d:00', $dayNames[$quietestDay], $quietestHour), | ||
| 'quietestCount' => $quietestCount === PHP_INT_MAX ? 0 : $quietestCount, | ||
| 'busiestDay' => $busiestDay, | ||
| 'days' => $days, | ||
| ], | ||
| ]; | ||
| } |
Comment on lines
+217
to
+242
| /** | ||
| * Returns a color for heatmap visualization based on intensity. | ||
| * | ||
| * Uses a green gradient from light (low activity) to dark (high activity). | ||
| * | ||
| * @param float $intensity Value between 0 and 1 | ||
| * | ||
| * @return string CSS color value | ||
| */ | ||
| public function heatmapColor(float $intensity): string { | ||
| if ($intensity <= 0) { | ||
| return '#ebedf0'; // Empty/no activity | ||
| } | ||
|
|
||
| // Green gradient similar to GitHub contribution graph | ||
| $colors = [ | ||
| '#9be9a8', // Light green (low) | ||
| '#40c463', // Medium green | ||
| '#30a14e', // Darker green | ||
| '#216e39', // Dark green (high) | ||
| ]; | ||
|
|
||
| $index = min((int)floor($intensity * count($colors)), count($colors) - 1); | ||
|
|
||
| return $colors[$index]; | ||
| } |
Comment on lines
+116
to
+146
| public function heatmap(): void { | ||
| if (!Configure::read('Queue.isStatisticEnabled')) { | ||
| throw new NotFoundException('Not enabled'); | ||
| } | ||
|
|
||
| $jobType = $this->request->getQuery('job_type'); | ||
| $metric = $this->request->getQuery('metric', 'created'); | ||
| $days = (int)$this->request->getQuery('days', 30); | ||
|
|
||
| // Validate metric | ||
| if (!in_array($metric, ['created', 'completed'], true)) { | ||
| $metric = 'created'; | ||
| } | ||
|
|
||
| // Validate days range | ||
| if ($days < 7) { | ||
| $days = 7; | ||
| } elseif ($days > 365) { | ||
| $days = 365; | ||
| } | ||
|
|
||
| $heatmapData = $this->QueuedJobs->getHeatmapData($metric, $days, $jobType); | ||
|
|
||
| $jobTypes = $this->QueuedJobs->find()->where()->find( | ||
| 'list', | ||
| keyField: 'job_task', | ||
| valueField: 'job_task', | ||
| )->distinct('job_task')->toArray(); | ||
|
|
||
| $this->set(compact('heatmapData', 'jobTypes', 'jobType', 'metric', 'days')); | ||
| } |
Comment on lines
+97
to
+106
| <div class="heatmap-cell" | ||
| style="background-color: <?= $bgColor ?>; color: <?= $textColor ?>;" | ||
| data-day="<?= $dayNamesFull[$day] ?>" | ||
| data-hour="<?= sprintf('%02d:00-%02d:59', $hour, $hour) ?>" | ||
| data-count="<?= $count ?>" | ||
| data-bs-toggle="tooltip" | ||
| data-bs-placement="top" | ||
| data-bs-html="true" | ||
| title="<strong><?= $dayNamesFull[$day] ?> <?= sprintf('%02d:00', $hour) ?></strong><br><?= number_format($count) ?> <?= __d('queue', 'jobs') ?>"> | ||
| <?php if ($count > 0) { ?> |
Comment on lines
+292
to
+303
|
|
||
| <?php $this->append('script'); ?> | ||
| <script> | ||
| document.addEventListener('DOMContentLoaded', function() { | ||
| // Initialize Bootstrap tooltips | ||
| var tooltipTriggerList = [].slice.call(document.querySelectorAll('[data-bs-toggle="tooltip"]')); | ||
| tooltipTriggerList.forEach(function(tooltipTriggerEl) { | ||
| new bootstrap.Tooltip(tooltipTriggerEl); | ||
| }); | ||
| }); | ||
| </script> | ||
| <?php $this->end(); |
src/Model/Table/QueuedJobsTable.php
Outdated
Comment on lines
+878
to
+899
| $query = $this->find(); | ||
|
|
||
| // Build day of week and hour expressions based on driver | ||
| switch ($driverName) { | ||
| case static::DRIVER_POSTGRES: | ||
| $dayOfWeek = $query->newExpr("EXTRACT(DOW FROM {$field})"); | ||
| $hourOfDay = $query->newExpr("EXTRACT(HOUR FROM {$field})"); | ||
|
|
||
| break; | ||
| case static::DRIVER_SQLSERVER: | ||
| $dayOfWeek = $query->newExpr("DATEPART(WEEKDAY, {$field}) - 1"); | ||
| $hourOfDay = $query->newExpr("DATEPART(HOUR, {$field})"); | ||
|
|
||
| break; | ||
| case static::DRIVER_SQLITE: | ||
| $dayOfWeek = $query->newExpr("CAST(strftime('%w', {$field}) AS INTEGER)"); | ||
| $hourOfDay = $query->newExpr("CAST(strftime('%H', {$field}) AS INTEGER)"); | ||
|
|
||
| break; | ||
| default: // MySQL | ||
| $dayOfWeek = $query->newExpr("DAYOFWEEK({$field})"); | ||
| $hourOfDay = $query->newExpr("HOUR({$field})"); |
src/Model/Table/QueuedJobsTable.php
Outdated
|
|
||
| break; | ||
| case static::DRIVER_SQLSERVER: | ||
| $dayOfWeek = $query->newExpr("DATEPART(WEEKDAY, {$field}) - 1"); |
- Add whitelist validation for $field parameter to prevent SQL injection - Fix SQL Server DATEFIRST issue using DATEDIFF calculation - Fix XSS in tooltip by escaping values with h() - Remove duplicate tooltip initialization (handled in layout) - Use expr() instead of deprecated newExpr() - Add comprehensive tests for getHeatmapData() method - Add tests for heatmapColor() helper method - Add controller tests for heatmap() action
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new heatmap visualization page for analyzing job activity patterns by day of week and hour. This helps identify:
Features
Technical Details