Compiler Plugin API: Any materials on implementation details?

[GavinRay97]

Hiya,

I saw a demo of the DF Compiler Plugin at the KotlinConf 25 keynote:
https://www.youtube.com/watch?v=F5NaqGF9oT4&t=543s

This demo was the highlight of the keynote for me!

(See my comment here)

The ability to compute value types ad-hoc from user code opens some VERY interesting doors.

My immediate thought it that it would allow you to write type-safe GraphQL clients, for example.
Something like the TypeScript GQL clients such as genql and graphql-zeus which do this:

👇CLICK TO SHOW IMAGE👇

It seems like this should be feasible, since it's similar in theory to the DF functionality.

I could imagine use something like:

val result = client.query {
    countries {
        +capital
        continent {
            +name
        }
    }
}

Where the computed type would be something like:

// Generated at compile time
data class CountryProjection_capital_continent(
  val capital: String
  val continent: ContinentProjection_name
)

data class ContinentProjection_name(
  val name: String
)

val result: CountryProjection_capital_continent = client.query { ... }

I'd be interested in taking a crack at implementing a plugin of this sort.
But I found the docs a bit sparse on the compiler plugin implementation code:

https://github.com/JetBrains/kotlin/tree/master/plugins/kotlin-dataframe

[koperagen]

Hi :) Disclaimer: keep in mind that extension point used is highly experimental, intended for specific use case of DataFrame library and not guaranteed to continue working in the future. We do not recommend to use it

You can read about extension point and overall plugin pipeline here:
https://youtrack.jetbrains.com/issue/KT-65859/Calls-refinement-extension-point
Plus kdocs for this extension point in Kotlin repository.

Otherwise, take a small example like dataFrameOf("a" to columnOf(1)) and run testPlayground in debugger to see how process goes
https://github.com/JetBrains/kotlin/blob/master/plugins/kotlin-dataframe/tests-gen/org/jetbrains/kotlin/fir/dataframe/DataFrameBlackBoxCodegenTestGenerated.java

Classes that do main job regarding updating type based on call arguments and resolution:
https://github.com/JetBrains/kotlin/blob/master/plugins/kotlin-dataframe/kotlin-dataframe.k2/src/org/jetbrains/kotlinx/dataframe/plugin/extensions/FunctionCallTransformer.kt

https://github.com/JetBrains/kotlin/blob/master/plugins/kotlin-dataframe/kotlin-dataframe.k2/src/org/jetbrains/kotlinx/dataframe/plugin/extensions/ReturnTypeBasedReceiverInjector.kt

[koperagen]

But what i could definitely recommend is looking at Kotlin Notebook and its Jupyter Kernel API for any prototypes. Have you tried Kotlin Notebook? Basically, DataFrame compiler plugin evolves what we had there way earlier and much cheaper:
Whenever cell is executed, callback to generate code is called for variables with DataFrame type, additional classes are generated, and user-declared variable is shadowed with more precise type.

So, for instance, this works even without compiler plugin in Kotlin Notebook thanks to this multi-stage / layered compilation

val df = dataFrameOf("col" to listOf(123))

df.col

Here is an example of using such API right from the notebook

import kotlin.reflect.KProperty

class MyWrapper(val s: String)

USE {
    onLoaded {
        addTypeConverter(object : FieldHandler {
            override val execution: FieldHandlerExecution<*> = FieldHandlerFactory.createUpdateExecution<Any> { instance, property ->
                execute("MyWrapper(${property.name})").name
            }

            override fun accepts(value: Any?, property: KProperty<*>): Boolean {
                return value is String && value.startsWith("wrap")
            }
        })
    }
}

It results in a variable with updated type:

Here's documentation of Jupyter API https://github.com/Kotlin/kotlin-jupyter/blob/7837e956dcac1a557b94f4fadee0332518ef49dc/docs/libraries.md#integration-using-the-kotlin-api

[GavinRay97]

Thank you so much! This is quite a lot more than I was expecting to be helped with, to be honest.
A great starting point here =)

[GavinRay97]

Have you tried Kotlin Notebook?

I have! I used to use Kotlin worksheets/scratch .kts files before Notebooks were around.

(I actually had an interview session with someone from Jetbrains Kotlin team where we talked quite a bit about this workflow, and how Notebooks were the new feature meant to replace them).

Notebooks/Interactive code sheets are the best way to develop any application.