feat(notifications): add hardware_summary action #1610

ximiali · 2025-11-12T07:37:41Z

Description

Add a new hardware_summary action to the notifications management command.
This action generates hardware reports based on the hardware subscription files.

Changes

Add a new hardware_summary action to the notifications command with CLI argument.
Add functions to parse hardware subscription files, query hardware data, and normalize hardware results.
Add Jinja template for email reports.
Add example and Qualcomm hardware subscription files.

How to Test

Run the following command to generate the hardware summary report:

poetry run python manage.py notifications --action hardware_summary

Related Issue

Closes #1081

MarceloRobert · 2025-11-12T11:57:28Z

Hello and thanks for the contribution! We will check this PR as soon as we can.

For now, please check the lint rules, you can use the linter locally to check for required changes. Commit and PR description looks good!

I glanced over the changes and they look well written, I would just make a nitpick for leaving a \n at the end of the last line on the new files, GitHub marks those problems with a symbol at the end of the file.

padovan · 2025-11-12T14:02:38Z

Hello, Thanks for this. Can you also attach and example output from running this command. I want to review how the report output is coming out.

Add new hardware_summary action in the notifications management command. This action generates hardware reports based on the subscription file. Signed-off-by: Yushan Li <[email protected]>

ximiali · 2025-11-13T09:24:58Z

Hello, Thanks for this. Can you also attach and example output from running this command. I want to review how the report output is coming out.

Thanks for your feedback! Here’s an example report output, showing only a limited number of items for clarity.

Hello,

Status summary for qcs9100-ride

Builds:    43 ✅    0 ❌    0 ⚠️
Boots:       0 ✅    0 ❌    0 ⚠️
Tests:      55 ✅    5 ❌   39 ⚠️
-------------

#kernelci test maestro:685eb3af5c2cf25042e8983b
- Status: FAIL
- Comment: baseline-arm64-qualcomm on qcs9100-ride in lava-kci-qualcomm
- Starttime: 2025-06-27 15:07:27.269000+00:00
- Tree: next/master
- Origin: maestro
- Test_lab: lava-kci-qualcomm
- Commit: 2aeda9592360c200085898a258c4754bfe879921
- Dashboard: https://dashboard.kernelci.org/test/maestro:685eb3af5c2cf25042e8983b
-------------


--
This is an experimental report format. Please send feedback in!
Talk to us at [email protected]

Made with love by the KernelCI team - https://kernelci.org/
'''

padovan · 2025-11-13T11:36:04Z

It is looking great. Thanks for sharing. I am fine with the output being generated and with the yaml config file for hardware.

I'll let the Dashboard team review it in detail and comment on the implementation.

MarceloRobert

Could you update the notifications.md document with this new command too?

MarceloRobert · 2025-11-13T16:26:26Z

backend/kernelCI_app/queries/hardware.py

+                checkouts.git_commit_name AS build__checkout__git_commit_name,
+                checkouts.git_commit_hash AS build__checkout__git_commit_hash,
+                checkouts.tree_name AS build__checkout__tree_name,
+                checkouts.origin AS build__checkout__origin


The reason we have some renamings in our queries is that django uses this format of foreignkey__field to get fields from other tables, and the renaming was keeping compatibility for some functions. In this new query, if you know where you are going to use this data, you don't really need to rename them with this double underscore format; it's not a big problem though, just not needed I think

MarceloRobert · 2025-11-13T16:29:19Z

backend/kernelCI_app/queries/hardware.py

+                tests.misc ->> 'runtime' AS runtime,
+                tests.environment_misc ->> 'job_id' AS job_id,


thought you can select these values at query level, I would prefer if you got them from the already-fetched fields at the Python level, which might have less impact in the database

MarceloRobert · 2025-11-13T16:44:44Z

backend/kernelCI_app/queries/hardware.py

        return dict_fetchall(cursor)


+# select the detailed data depends on hardware_id


nit: If you want to add a comment explaining the function, prefer using a docstring at the beginning of the function

MarceloRobert · 2025-11-13T16:48:10Z

backend/kernelCI_app/management/commands/templates/hardware_report.txt.j2

+{{- "{:>5}".format(test_status_group["inconclusive"]) }} ⚠️
+-------------
+{% for data in hardware_data %}
+#kernelci test {{ data["id"] }}


Depends on what you want, but would it make sense to show the test path here, instead of the id?

MarceloRobert · 2025-11-13T16:54:03Z

backend/kernelCI_app/management/commands/notifications.py

+    hardwares_data_raw = get_hardware_summary_data(
+        keys=list(hardware_key_set),
+        start_date=start_date,
+        end_date=end_date,
+    )


I know that usually there will be results, but just for precaution I would add a

if not hardwares_data_raw: print("No data for hardware summary") return

so that the command can return gracefully

MarceloRobert · 2025-11-13T17:01:50Z

backend/kernelCI_app/management/commands/notifications.py

+        # extract recipient
+        for (hardware_id, origin), report_configs in hardware_prop_map.items():
+            for hardware_report in report_configs:
+                recipients = process_submission_options(
+                    default_recipients=hardware_report.get("default_recipients", []),
+                    specific_recipients=hardware_report.get("recipients", []),
+                    options=hardware_report.get("options", []),
+                )


shouldn't this block be getting only the recipient for the specific hardware in this loop? it seems like it is getting the recipients for all reports and using only the last one.

Also, since you already have (hardware_id, origin), you could do a report_configs = hardware_prop_map[(hardware_id, origin)] and follow that

MarceloRobert · 2025-11-13T17:14:33Z

backend/kernelCI_app/management/commands/notifications.py

+    start_date = now - timedelta(days=30)
+    end_date = now


should the user be able to change this value? 30 days seem to be long, maybe the user just wants a recent summary

ximiali force-pushed the notifications-hardware-summary-report branch 2 times, most recently from 629b5a1 to 269cd86 Compare November 12, 2025 08:44

MarceloRobert assigned ximiali Nov 12, 2025

ximiali force-pushed the notifications-hardware-summary-report branch from 269cd86 to 2f1cc1a Compare November 13, 2025 03:00

feat(notifications): add hardware_summary action

603424a

Add new hardware_summary action in the notifications management command. This action generates hardware reports based on the subscription file. Signed-off-by: Yushan Li <[email protected]>

ximiali force-pushed the notifications-hardware-summary-report branch from 2f1cc1a to 603424a Compare November 13, 2025 08:04

MarceloRobert requested changes Nov 13, 2025

View reviewed changes

MarceloRobert added the notifications label Nov 13, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(notifications): add hardware_summary action #1610

feat(notifications): add hardware_summary action #1610

ximiali commented Nov 12, 2025

Uh oh!

MarceloRobert commented Nov 12, 2025

Uh oh!

padovan commented Nov 12, 2025

Uh oh!

ximiali commented Nov 13, 2025

Uh oh!

padovan commented Nov 13, 2025

Uh oh!

MarceloRobert left a comment

Uh oh!

MarceloRobert Nov 13, 2025

Uh oh!

MarceloRobert Nov 13, 2025

Uh oh!

MarceloRobert Nov 13, 2025

Uh oh!

MarceloRobert Nov 13, 2025

Uh oh!

MarceloRobert Nov 13, 2025

Uh oh!

MarceloRobert Nov 13, 2025

Uh oh!

MarceloRobert Nov 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		tests.misc ->> 'runtime' AS runtime,
		tests.environment_misc ->> 'job_id' AS job_id,

		return dict_fetchall(cursor)


		# select the detailed data depends on hardware_id

feat(notifications): add hardware_summary action #1610

Are you sure you want to change the base?

feat(notifications): add hardware_summary action #1610

Conversation

ximiali commented Nov 12, 2025

Description

Changes

How to Test

Related Issue

Uh oh!

MarceloRobert commented Nov 12, 2025

Uh oh!

padovan commented Nov 12, 2025

Uh oh!

ximiali commented Nov 13, 2025

Uh oh!

padovan commented Nov 13, 2025

Uh oh!

MarceloRobert left a comment

Choose a reason for hiding this comment

Uh oh!

MarceloRobert Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

MarceloRobert Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

MarceloRobert Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

MarceloRobert Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

MarceloRobert Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

MarceloRobert Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

MarceloRobert Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants