Repository navigation
perf(labs): serve the listing from lab_daily - #2176
Open
alanpeixinho wants to merge 1 commit into
Open
alanpeixinho wants to merge 1 commit into
alanpeixinho wants to merge 1 commit into
Conversation
* Add a lab_daily aggregate and recompute it from builds and tests. * Read the labs listing and lab origins from that table, with no JSON lab fallback. * Cache the listing query, and refresh today hourly and the last 30 days every 6 hours. Assisted-by: Cursor Grok 4.7 Closes kernelci#2165 Signed-off-by: Alan Peixinho <alan.peixinho@profusion.mobi>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What it is
The labs listing and the lab origins list read
lab_dailyinstead of scanningbuildsandtests. Each row is one UTC day, one origin, and one lab, with pass, fail, and inconclusive counts for builds, boots, and tests.recompute_lab_dailyrebuilds a day by deleting that day and inserting the counts again from raw rows. Rows with nolab_idare left out. Dummy builds are left out. The cron rebuilds today every hour, and the last 30 days every 6 hours.Both queries use the normal query cache. Build counts stay at 0 until those builds have a
lab_id.On production, scanning
testsfor the listing took 12.2 s, and scanningtestsfor the origins list took 66 to 75 s. The local database has 2,100,330 tests. The listing scan read 1,478,325 of those and returned 16 labs. The same listing read 80 rows fromlab_daily. The origins scan read 1,599,999 tests and returned 2 origins. The same list read 120 rows fromlab_daily.testslab_dailyHow to test
Run this on a local database.
0021_lab_daily.python manage.py recompute_lab_daily --days=30./labs, or callGET /api/labs/?origin=maestro&interval_in_days=5andGET /api/labs/origins/.lab_idset shows its counts. A lab withlab_idnull does not appear.python manage.py recompute_lab_daily --day=YYYY-MM-DDa second time. The same day is replaced, and the counts stay the same.Closes #2165
Made with Cursor