Skip to content

Commit e84e88a

Browse files
committed
Added initial configuration to measure K2 performance
1 parent bf9ad2a commit e84e88a

15 files changed

+1480
-0
lines changed

.gitignore

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
.idea/
2+
.gradle/
3+
build/
4+
.kotlin/
5+
local.properties
6+
/reports

README.md

+113
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
# Description
2+
3+
This projects allows you to run performance tests on your Gradle project to compare
4+
performance difference between [Kotlin K1 and K2 compilers](https://blog.jetbrains.com/kotlin/2023/02/k2-kotlin-2-0/).
5+
6+
Performance tests will build your project with kotlin `2.0.0-RC1` and kotlin `1.9.23` releases by running `assmeble` task.
7+
The `assemble` task is used by default to build your project.
8+
Benchmarks will be run three scenarios: "clean build",
9+
"incremental build with non-ABI changes" and "incremental build with ABI changes".
10+
After benchmark will finish open [benchmarkResult.ipynb](benchmarkResult.ipynb) Kotlin notebook to compare Gradle
11+
and Kotlin time in the produced result.
12+
13+
## Getting Started
14+
15+
## Project requirements
16+
17+
Project should allow changing the Kotlin version via `kotlin_version` Gradle property.
18+
19+
If a project is using version catalog,
20+
you can update `settings.gradle.kts` files in it to override the kotlin version:
21+
```kotlin
22+
versionCatalogs {
23+
create("libs") {
24+
val kotlinVersion = System.getenv("kotlin_version")
25+
if (kotlinVersion != null) {
26+
version("kotlin", kotlinVersion)
27+
}
28+
}
29+
}
30+
31+
```
32+
33+
## Tested project
34+
35+
There are two options to configure your project for performance tests:
36+
- set the path to your project via `project.path` in [gradle.properties](gradle.properties).
37+
- set `project.git.url` and `project.git.commit.sha` in [gradle.properties](gradle.properties) to clone a project from git repository.
38+
39+
In addition, you can specify directory where the project will be stored with `project.git.directory` property,
40+
otherwise Git project name will be used.
41+
42+
## Update build scenarios for your project
43+
44+
Additional scenarios configuration is required to run performance tests on your project.
45+
Please set in [gradle.properties](gradle.properties) the following properties:
46+
- set list of files split by comma for incremental build with non-ABI changes in `scenario.non.abi.changes` property
47+
- set a single file path for incremental build with ABI change in `scenario.abi.changes` property
48+
- set the default task to build your project in `scenario.task` property, otherwise `assemble` task will be used.
49+
50+
## Custom build scenarios
51+
52+
If you are interested in running performance build on custom scenarios, you can configure them via
53+
`PerformanceTask.scenarios` task input,
54+
or set [Gradle profiler](https://github.com/gradle/gradle-profiler) scenario file via `PerformanceTask.scenarioFile` task input.
55+
56+
For example, you can do follow update in [build.gradle.kts](build.gradle.kts) file to use custom scenarios:
57+
58+
```kotlin
59+
60+
import org.jetbrains.kotlin.k2.blogpost.PerformanceTask
61+
62+
val customScenarios = listOf(Scenario(
63+
name = "new scenario",
64+
cleanTasks = listOf("clean_task_if_need"),
65+
kotlinVersion = kotlinVersion,
66+
tasks = listOf("task_to_execute"),
67+
projectDir = project.projectDir,
68+
nonAbiChanges = listOf("path_to_file"),
69+
warmUpRounds = 5,
70+
executionRounds = 5,
71+
))
72+
73+
val benchmark_2_0 = PerformanceTask.registerPerformanceTask(project, "benchmark_2_0", "2.0.0-RC1") {
74+
scenarios.set(customScenarios)
75+
}
76+
77+
val benchmark_1_9 = PerformanceTask.registerPerformanceTask(project, "benchmark_1_9", "1.9.23") {
78+
scenarios.set(customScenarios)
79+
}
80+
81+
tasks.register("runBenchmarks") {
82+
dependsOn(benchmark_2_0, benchmark_1_9)
83+
}
84+
85+
```
86+
87+
## Run performance test
88+
Please set up `JAVA_HOME` and `ANDROID_HOME`(if need) variables before run tests
89+
90+
Execute
91+
```bash
92+
./gradlew runBenchmarks
93+
```
94+
95+
## Visualise results
96+
97+
To analyse benchmark results,
98+
you need to have installed [Kotlin notebook](https://blog.jetbrains.com/kotlin/2023/07/introducing-kotlin-notebook/)
99+
plugin in IntelliJ IDEA.
100+
101+
Please open [benchmarkResult.ipynb](benchmarkResult.ipynb) and run it with `Run All` action to compare produced results.
102+
103+
## Troubleshooting
104+
105+
It is possible that a new warning appears in the build with kotlin `2.0.0`.
106+
Please either fix them or disable `allWarningsAsErrors` compiler option if you use it.
107+
108+
If [dependencies verification](https://docs.gradle.org/8.2.1/userguide/dependency_verification.html#sub:enabling-verification) is enabled,
109+
please check that both kotlin `1.9` and kotlin `2.0` dependencies are included.
110+
111+
The `JSON` build report output type is available since kotlin `2.0.0-RC1` and kotlin `1.9.23`.
112+
If a `kotlinDsl` plugin is applied in the `buildSrc` subproject,
113+
we recommend to also apply `kotlin(“jvm”)` plugin to avoid any issues with unknown output type.

0 commit comments

Comments
 (0)