Skip to content

Implement results tree-report command #213

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Implement results tree-report command #213

wants to merge 2 commits into from
+171 −1

Conversation

JenySadadia
Copy link

Get tree report.

Implement function to request /tree-report endpoint
89440ee 
Add `dashboard_fetch_tree_report` to get
tree report using dashboard API.

Signed-off-by: Jeny Sadadia <[email protected]>
@JenySadadia JenySadadia force-pushed the tree-report-cmd branch 2 times, most recently from 30de9d0 to f24f752 Compare August 19, 2025 09:58
@JenySadadia JenySadadia marked this pull request as ready for review August 19, 2025 09:59
@JenySadadia
Copy link
Author

Sample command output without --json option:

$ kci-dev results tree-report --giturl  https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git --branch master --min-age 312 --max-age 340 --group-size 2
Tree: https://d.kernelci.org/tree/next/master/b7d4e259682caccb51a25283655f2c8f02e32d23?o=maestro
Commit: b7d4e259682caccb51a25283655f2c8f02e32d23
Origin: maestro
Checkout start time: 2025-08-06T04:12:56.776000Z

Summary: (pass/fail/inconclusive)
builds:	58/1/1
boots:	89/0/57
tests:	15794/5777/5613

Possible regressions:
Platform: dell-latitude-5400-8665U-sarien
- Config: x86_64_defconfig+lab-setup+x86-board+kselftest
  - Test path: kselftest.cpufreq.hibernate
  · https://d.kernelci.org/test/maestro:6892e1a479d5d217e5451ae7 status:FAIL
  · https://d.kernelci.org/test/maestro:68918ae179d5d217e54314e0 status:PASS

  - Test path: kselftest.cpufreq.hibernate.cpufreq_main_sh
  · https://d.kernelci.org/test/maestro:6892e3b679d5d217e54533e7 status:FAIL
  · https://d.kernelci.org/test/maestro:68918eda79d5d217e5433d1d status:PASS

  - Test path: kselftest.cpufreq.suspend
  · https://d.kernelci.org/test/maestro:6892e1ac79d5d217e5451b2b status:FAIL
  · https://d.kernelci.org/test/maestro:68918ae879d5d217e5431524 status:PASS

  - Test path: kselftest.cpufreq.suspend.cpufreq_main_sh
  · https://d.kernelci.org/test/maestro:6892e49e79d5d217e5454ec9 status:FAIL
  · https://d.kernelci.org/test/maestro:68918ed579d5d217e5433ce9 status:PASS

Fixed regressions:
Platform: beaglebone-black
- Config: multi_v7_defconfig+kselftest
  - Test path: kselftest.seccomp.seccomp_seccomp_benchmark
  · https://d.kernelci.org/test/maestro:6892f25779d5d217e5459341 status:PASS
  · https://d.kernelci.org/test/maestro:6891bf8879d5d217e5438bff status:FAIL

Platform: mt8195-cherry-tomato-r2
- Config: defconfig+lab-setup+arm64-chromebook+CONFIG_MODULE_COMPRESS=n+CONFIG_MODULE_COMPRESS_NONE=y
  - Test path: fluster.debian.v4l2.gstreamer_vp8.validate-fluster-results
  · https://d.kernelci.org/test/maestro:6892df6f79d5d217e54506f5 status:PASS
  · https://d.kernelci.org/test/maestro:68918e8a79d5d217e5433add status:FAIL

Platform: dell-latitude-5400-4305U-sarien
- Config: x86_64_defconfig+lab-setup+x86-board+kselftest
  - Test path: kselftest.cpufreq.hibernate
  · https://d.kernelci.org/test/maestro:6892e1a479d5d217e5451ae4 status:PASS
  · https://d.kernelci.org/test/maestro:68918ae079d5d217e54314dd status:FAIL

  - Test path: kselftest.cpufreq.hibernate.cpufreq_main_sh
  · https://d.kernelci.org/test/maestro:6892f14679d5d217e5459180 status:PASS
  · https://d.kernelci.org/test/maestro:6891aae979d5d217e5438288 status:FAIL

Unstable tests:
Platform: bcm2711-rpi-4-b
- Config: defconfig+lab-setup+kselftest
  - Test path: boot
  · https://d.kernelci.org/test/maestro:6892e0c579d5d217e5451677 status:INCONCLUSIVE (None)
  · https://d.kernelci.org/test/maestro:6892e0ba79d5d217e545162c status:PASS

Platform: beaglebone-black
- Config: multi_v7_defconfig
  - Test path: boot
  · https://d.kernelci.org/test/maestro:6892e3d879d5d217e54535fb status:PASS
  · https://d.kernelci.org/test/maestro:6892e3d679d5d217e54535b6 status:INCONCLUSIVE (MISS)

- Config: multi_v7_defconfig+kselftest
  - Test path: kselftest.landlock
  · https://d.kernelci.org/test/maestro:6892deb279d5d217e544ffc9 status:INCONCLUSIVE (None)
  · https://d.kernelci.org/test/maestro:6891893f79d5d217e5430eb5 status:INCONCLUSIVE (ERROR)

Platform: cd8180-orion-o6
- Config: defconfig+lab-setup+kselftest
  - Test path: kselftest.signal
  · https://d.kernelci.org/test/maestro:6892e0e779d5d217e5451774 status:INCONCLUSIVE (MISS)
  · https://d.kernelci.org/test/maestro:6891905979d5d217e5434afe status:INCONCLUSIVE (ERROR)

Platform: imx8mp-evk
- Config: defconfig+lab-setup+kselftest
  - Test path: kselftest.alsa
  · https://d.kernelci.org/test/maestro:6892e0c879d5d217e5451687 status:INCONCLUSIVE (None)
  · https://d.kernelci.org/test/maestro:6891903279d5d217e54349b6 status:INCONCLUSIVE (MISS)

  - Test path: kselftest.arm64
  · https://d.kernelci.org/test/maestro:6892e0cc79d5d217e54516af status:PASS
  · https://d.kernelci.org/test/maestro:6891903779d5d217e54349fe status:INCONCLUSIVE (MISS)

  - Test path: kselftest.device_error_logs
  · https://d.kernelci.org/test/maestro:6892e0d579d5d217e54516f2 status:FAIL
  · https://d.kernelci.org/test/maestro:6891904279d5d217e5434a50 status:INCONCLUSIVE (MISS)

  - Test path: kselftest.dt
  · https://d.kernelci.org/test/maestro:6892e0db79d5d217e5451722 status:FAIL
  · https://d.kernelci.org/test/maestro:6891904979d5d217e5434a83 status:INCONCLUSIVE (MISS)

- Config: defconfig+arm64-chromebook+kselftest
  - Test path: kselftest.kvm
  · https://d.kernelci.org/test/maestro:6892e70c79d5d217e54566b4 status:FAIL
  · https://d.kernelci.org/test/maestro:68918fed79d5d217e5434654 status:INCONCLUSIVE (MISS)

  - Test path: kselftest.pkvm
  · https://d.kernelci.org/test/maestro:6892e70e79d5d217e54566c3 status:FAIL
  · https://d.kernelci.org/test/maestro:68918fef79d5d217e5434663 status:INCONCLUSIVE (MISS)

@JenySadadia
Copy link
Author

Sample output with --json and multiple --path options:

$ kci-dev results tree-report --giturl  https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git --branch master --min-age 312 --max-age 340 --group-size 2 --path kselftest.cpufreq.suspend --path boot --json
{
 "dashboard_url": "https://d.kernelci.org/tree/next/master/b7d4e259682caccb51a25283655f2c8f02e32d23?o=maestro",
 "git_url": "https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git",
 "git_branch": "master",
 "commit_hash": "b7d4e259682caccb51a25283655f2c8f02e32d23",
 "origin": "maestro",
 "checkout_start_time": "2025-08-06T04:12:56.776000Z",
 "build_status_summary": {
  "PASS": 58,
  "ERROR": 0,
  "FAIL": 1,
  "SKIP": 0,
  "MISS": 1,
  "DONE": 0,
  "NULL": 0
 },
 "boot_status_summary": {
  "pass_count": 89,
  "error_count": 4,
  "fail_count": 0,
  "skip_count": 0,
  "miss_count": 50,
  "done_count": 0,
  "null_count": 3
 },
 "test_status_summary": {
  "pass_count": 15794,
  "error_count": 37,
  "fail_count": 5777,
  "skip_count": 5449,
  "miss_count": 54,
  "done_count": 0,
  "null_count": 73
 },
 "possible_regressions": {},
 "fixed_regressions": {},
 "unstable_tests": {
  "bcm2711-rpi-4-b": {
   "defconfig+lab-setup+kselftest": {
    "boot": [
     {
      "id": "maestro:6892e0c579d5d217e5451677",
      "start_time": "2025-08-06T04:57:41.785000Z",
      "status": null
     },
     {
      "id": "maestro:6892e0ba79d5d217e545162c",
      "start_time": "2025-08-06T04:57:30.875000Z",
      "status": "PASS"
     }
    ]
   }
  },
  "beaglebone-black": {
   "multi_v7_defconfig": {
    "boot": [
     {
      "id": "maestro:6892e3d879d5d217e54535fb",
      "start_time": "2025-08-06T05:10:48.208000Z",
      "status": "PASS"
     },
     {
      "id": "maestro:6892e3d679d5d217e54535b6",
      "start_time": "2025-08-06T05:10:46.224000Z",
      "status": "MISS"
     }
    ]
   }
  },
  "meson-gxl-s905x-libretech-cc": {
   "defconfig+lab-setup+kselftest": {
    "boot": [
     {
      "id": "maestro:6892e0c079d5d217e5451656",
      "start_time": "2025-08-06T04:57:36.701000Z",
      "status": "MISS"
     },
     {
      "id": "maestro:6892e0be79d5d217e5451645",
      "start_time": "2025-08-06T04:57:34.628000Z",
      "status": "PASS"
     }
    ]
   }
  },
  "rk3399-roc-pc": {
   "defconfig+lab-setup+kselftest": {
    "boot": [
     {
      "id": "maestro:6892e0c279d5d217e545165c",
      "start_time": "2025-08-06T04:57:38.302000Z",
      "status": "MISS"
     },
     {
      "id": "maestro:6892e0bf79d5d217e545164d",
      "start_time": "2025-08-06T04:57:35.454000Z",
      "status": "PASS"
     }
    ]
   }
  },
  "rk3588-rock-5b": {
   "defconfig+lab-setup+kselftest": {
    "boot": [
     {
      "id": "maestro:6892e0c279d5d217e5451660",
      "start_time": "2025-08-06T04:57:38.665000Z",
      "status": "MISS"
     },
     {
      "id": "maestro:6892e0b579d5d217e54515fb",
      "start_time": "2025-08-06T04:57:25.469000Z",
      "status": "PASS"
     }
    ]
   }
  },
  "sun50i-a64-pine64-plus": {
   "defconfig+lab-setup+kselftest": {
    "boot": [
     {
      "id": "maestro:6892e0bf79d5d217e5451650",
      "start_time": "2025-08-06T04:57:35.878000Z",
      "status": "PASS"
     },
     {
      "id": "maestro:6892e0ba79d5d217e5451629",
      "start_time": "2025-08-06T04:57:30.511000Z",
      "status": "MISS"
     }
    ]
   }
  },
  "sun50i-h6-pine-h64": {
   "defconfig+lab-setup+kselftest": {
    "boot": [
     {
      "id": "maestro:6892e0c479d5d217e545166e",
      "start_time": "2025-08-06T04:57:40.037000Z",
      "status": "MISS"
     },
     {
      "id": "maestro:6892e0b679d5d217e54515fe",
      "start_time": "2025-08-06T04:57:26.464000Z",
      "status": "PASS"
     }
    ]
   }
  }
 }
}

Implement results tree-report command
9bfd141 
Fetch tree report using the dashboard API endpoint
and display it.
The report will consist of builds, boots, tests
summary along with regression and unstable tests
information.

Signed-off-by: Jeny Sadadia <[email protected]>
@padovan
Copy link
Contributor

padovan commented Aug 19, 2025

Thanks for this, Jeny!

  1. I'd try to imitate the email report for the output:
 last run: https://d.kernelci.org/test/maestro:6890eeae236da8c8a0d7faea
 history:  > ❌  > ⚠️  > ✅  > ✅  > ❌

  1. what are min-age and max-age. Why would we need that as parameter?

  2. --group-size rename to --history-size?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@padovan padovan Awaiting requested review from padovan

@aliceinwire aliceinwire Awaiting requested review from aliceinwire

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@JenySadadia @padovan