Docs: Kotlin Serialization Guide (#933)

Documentation from the docs folder is incorporated into a common story that spans six chapters and proceeds from the basic serialization to the experimental alternative and custom formats. The example files are automatically generated from the text with the Knit tool (guide/example folder) and are verified with the automatically generated tests (guide/test folder).

The entry point to the new is docs/serialization-guide.md that contains the overall table of contents and links to the specific chapters.

The old docs that are being incorporated:
* cbor_byte_string.md -> formats.md
* custom_serializers.md -> serializers.md
* examples.md -> basic-serialization.md
* json_transformations.md -> json.md
* old12.md (removed completely as obsolete)
* runtime_usage.md -> various chapters

Additional changes include:
* Top-level README.md file is cleaned up from legacy information and also contains a modern knitted example.
* More consistent error messages in various exceptions.
* Minor corrections to KDocs and APIs.
* Better style.css for the documentation that is generated by Dokka.
* Improved update_docs.sh to put temporary files inside the build folder.

Co-authored-by: Vsevolod Tolstopyatov <[email protected]>

Author: elizarov

README.md

@@ -1,117 +1,100 @@
 # Kotlin multiplatform / multi-format reflectionless serialization
 
-[![JetBrains incubator project](https://jb.gg/badges/incubator.svg)](https://confluence.jetbrains.com/display/ALL/JetBrains+on+GitHub)
+[![official JetBrains project](https://jb.gg/badges/official.svg)](https://confluence.jetbrains.com/display/ALL/JetBrains+on+GitHub)
 [![GitHub license](https://img.shields.io/badge/license-Apache%20License%202.0-blue.svg?style=flat)](http://www.apache.org/licenses/LICENSE-2.0)
 [![TeamCity build](https://img.shields.io/teamcity/http/teamcity.jetbrains.com/s/KotlinTools_KotlinxSerialization_Ko.svg)](https://teamcity.jetbrains.com/viewType.html?buildTypeId=KotlinTools_KotlinxSerialization_Ko&guest=1)
-[![Download](https://api.bintray.com/packages/kotlin/kotlinx/kotlinx.serialization.runtime/images/download.svg) ](https://bintray.com/kotlin/kotlinx/kotlinx.serialization.runtime/_latestVersion)
+[![Download](https://api.bintray.com/packages/kotlin/kotlinx/kotlinx.serialization.runtime/images/download.svg?version=0.20.0) ](https://bintray.com/kotlin/kotlinx/kotlinx.serialization.runtime/0.20.0)
 
 Kotlin serialization consists of a compiler plugin, that generates visitor code for serializable classes,
- runtime library with core serialization API and JSON format, and support libraries with ProtoBuf, CBOR and properties formats.
+ runtime libraries with core serialization API and JSON format, and support libraries with ProtoBuf, CBOR and properties formats.
 
 * Supports Kotlin classes marked as `@Serializable` and standard collections.
-* Provides JSON, [CBOR](formats/README.md#CBOR), and [Protobuf](formats/README.md#ProtoBuf) formats.
+* Provides JSON (as a part of the core library), [Protobuf](formats/README.md#ProtoBuf), [CBOR](formats/README.md#CBOR), [Hocon](formats/README.md#HOCON) and [Properties](formats/README.md#properties) formats.
 * Complete multiplatform support: JVM, JS and Native.
 
 ## Table of contents
 
-* [Quick example](#quick-example)
-* [Runtime overview](#runtime-overview)
-* [Current status](#current-project-status)
-* [Library installing](#setup)
-    + [Gradle](#gradle)
-    + [Android/JVM](#androidjvm)
-    + [Multiplatform (common, JS, Native)](#multiplatform-common-js-native)
-    + [Maven/JVM](#mavenjvm)
-    + [Incompatible changes from older versions](#incompatible-changes)
-* [Troubleshooting IntelliJ IDEA](#troubleshooting-intellij-idea)
-* [Usage](docs/runtime_usage.md)
-* [More examples of supported Kotlin classes](docs/examples.md)
-* [Writing JSON transformations](docs/json_transformations.md)
-* [Writing custom serializers](docs/custom_serializers.md)
-* [Multiplatform polymorphic serialization](docs/polymorphism.md)
-* [Add-on formats](formats/README.md)
-* [Building library and compiler plugin from source](docs/building.md)
-* [Instructions for old versions under Kotlin 1.2 and migration guide](docs/old12.md)
-
-
-## Quick example
+<!--- TOC -->
+
+* [Introduction and references](#introduction-and-references)
+* [Setup](#setup)
+  * [Gradle](#gradle)
+    * [Using the `plugins` block](#using-the-plugins-block)
+    * [Using `apply plugin` (the old way)](#using-apply-plugin-the-old-way)
+    * [Dependency on the runtime library](#dependency-on-the-runtime-library)
+  * [Android/JVM](#android/jvm)
+  * [Multiplatform (common, JS, Native)](#multiplatform-common-js-native)
+  * [Maven/JVM](#maven/jvm)
+
+<!--- END -->
+
+## Introduction and references
+
+Here is a small example.
 
 ```kotlin
 import kotlinx.serialization.*
 import kotlinx.serialization.json.*
 
-@Serializable
-data class Data(val a: Int, val b: String = "42")
+@Serializable 
+data class Project(val name: String, val language: String)
 
 fun main() {
-    // Json also has .Default configuration which provides more reasonable settings,
-    // but is subject to change in future versions
-    val json = Json(JsonConfiguration.Stable)
-    // serializing objects
-    val jsonData = json.stringify(Data.serializer(), Data(42))
-    // serializing lists
-    val jsonList = json.stringify(Data.serializer().list, listOf(Data(42)))
-    println(jsonData) // {"a": 42, "b": "42"}
-    println(jsonList) // [{"a": 42, "b": "42"}]
-
-    // parsing data back
-    val obj = json.parse(Data.serializer(), """{"a":42}""") // b is optional since it has default value
-    println(obj) // Data(a=42, b="42")
+    // Serializing objects
+    val data = Project("kotlinx.serialization", "Kotlin")
+    val string = Json.encodeToString(data)  
+    println(string) // {"name":"kotlinx.serialization","language":"Kotlin"} 
+    // Deserializing back into objects
+    val obj = Json.decodeFromString<Project>(string)
+    println(obj) // Project(name=kotlinx.serialization, langauge=Kotlin)
 }
-```
-
-## Runtime overview
+``` 
 
-This project contains the runtime library. Runtime library provides:
+> You can get the full code [here](guide/example/example-readme-01.kt).
 
-* Ready-to-use JSON serialization format.
-* Core primitives for plugin-generated code (`Encoder`, `Decoder`, `SerialDescriptor`).
-* Basic skeleton implementations of core primitives for custom serialization formats. 
-* Built-ins and collections serializers.
+<!--- TEST_NAME ReadmeTest -->
 
-You can open example projects for [JS](examples/example-js) and [JVM](examples/example-jvm).
+<!--- TEST 
+{"name":"kotlinx.serialization","language":"Kotlin"}
+Project(name=kotlinx.serialization, language=Kotlin)
+-->
 
-To learn more about JSON usage and other formats, see [usage](docs/runtime_usage.md).
-More examples of various kinds of Kotlin classes that can be serialized can be found [here](docs/examples.md).
-
-## Current project status
-
-Starting from Kotlin 1.3-RC2, serialization plugin is shipped with the rest of Kotlin compiler distribution, and the IDEA plugin is bundled into the Kotlin plugin.
-
-Runtime library is under reconstruction to match the corresponding [KEEP](https://github.com/Kotlin/KEEP/blob/serialization/proposals/extensions/serialization.md), so some features described there can be not implemented yet. While library is stable and has successfully been used in various scenarios, there is no API compatibility guarantees between versions, that's why it is called experimental.
-This document describes setup for Kotlin 1.3 and higher. To watch instructions regarding 1.2, follow [this document](docs/old12.md).
+**Read the [Kotlin Serialization Guide](docs/serialization-guide.md) for all details.**
 
 ## Setup
 
-Using Kotlin Serialization requires Kotlin compiler `1.3.30` or higher.
-Make sure that you have corresponding Kotlin plugin installed in the IDE.
-Since serialization is now bundled into Kotlin plugin, no additional plugins for IDE are required (but make sure you have deleted old additional plugin for 1.2, if you had one).
+Kotlin serialization plugin is shipped with the Kotlin compiler distribution, and the IDEA plugin is bundled into the Kotlin plugin.
+
+Using Kotlin Serialization requires Kotlin compiler `1.4.0` or higher.
+Make sure you have the corresponding Kotlin plugin installed in the IDE, no additional plugins for IDE are required.
 Example projects on JVM are available for [Gradle](examples/example-jvm/build.gradle) and [Maven](examples/example-jvm/pom.xml).
 
 ### Gradle
 
 #### Using the `plugins` block
 
-You can setup the serialization plugin with the Kotlin plugin using [Gradle plugins DSL](https://docs.gradle.org/current/userguide/plugins.html#sec:plugins_block):
+You can set up the serialization plugin with the Kotlin plugin using 
+[Gradle plugins DSL](https://docs.gradle.org/current/userguide/plugins.html#sec:plugins_block):
 
 Kotlin DSL:
 
 ```kotlin
 plugins {
-    kotlin("multiplatform") // or kotlin("jvm") or any other kotlin plugin
-    kotlin("plugin.serialization") version "1.3.70"
+    kotlin("jvm") version "1.4.0" // or kotlin("multiplatform") or any other kotlin plugin
+    kotlin("plugin.serialization") version "1.4.0"
 }
-```
+```       
+
 Groovy DSL:
 
 ```gradle
 plugins {
-    id 'org.jetbrains.kotlin.multiplatform' version '1.3.70' // or any other kotlin plugin
-    id 'org.jetbrains.kotlin.plugin.serialization' version '1.3.70'
+    id 'org.jetbrains.kotlin.multiplatform' version '1.4.0'
+    id 'org.jetbrains.kotlin.plugin.serialization' version '1.4.0'
 }
 ```
 
-Note: plugin marker for serialization has been published in Kotlin 1.3.50. If you need to use the earlier Kotlin version, see [KT-27612](https://youtrack.jetbrains.com/issue/KT-27612) for workaround with [plugin resolution rules](https://docs.gradle.org/current/userguide/plugins.html#sec:plugin_resolution_rules).
+> Kotlin versions before are not supported by the stable release of Kotlin serialization
 
 #### Using `apply plugin` (the old way)
 
@@ -124,7 +107,7 @@ buildscript {
     repositories { jcenter() }
 
     dependencies {
-        val kotlinVersion = "1.3.70"
+        val kotlinVersion = "1.4.0"
         classpath(kotlin("gradle-plugin", version = kotlinVersion))
         classpath(kotlin("serialization", version = kotlinVersion))
     }
@@ -135,7 +118,7 @@ Groovy DSL:
 
 ```gradle
 buildscript {
-    ext.kotlin_version = '1.3.70'
+    ext.kotlin_version = '1.4.0'
     repositories { jcenter() }
 
     dependencies {
@@ -165,7 +148,7 @@ repositories {
 
 dependencies {
     implementation(kotlin("stdlib", KotlinCompilerVersion.VERSION)) // or "stdlib-jdk8"
-    implementation("org.jetbrains.kotlinx:kotlinx-serialization-runtime:0.20.0") // JVM dependency
+    implementation("org.jetbrains.kotlinx:kotlinx-serialization-core:1.0.0-RC") // JVM dependency
 }
 ```
 
@@ -178,7 +161,7 @@ repositories {
 
 dependencies {
     implementation "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version" // or "kotlin-stdlib-jdk8"
-    implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime:0.20.0" // JVM dependency
+    implementation "org.jetbrains.kotlinx:kotlinx-serialization-core:1.0.0-RC" // JVM dependency
 }
 ```
 
@@ -245,17 +228,14 @@ sourceSets {
 }
 ```
 
-JavaScript example is located at [`example-js`](examples/example-js) folder.
-Multiplatform example is located at [`example-multiplatform`](examples/example-multiplatform) folder.
-
 ### Maven/JVM
 
 Ensure the proper version of Kotlin and serialization version:
 
 ```xml
 <properties>
-    <kotlin.version>1.3.70</kotlin.version>
-    <serialization.version>0.20.0</serialization.version>
+    <kotlin.version>1.4.0</kotlin.version>
+    <serialization.version>1.0.0-RC</serialization.version>
 </properties>
 ```
 
@@ -301,29 +281,7 @@ Add dependency on serialization runtime library:
 ```xml
 <dependency>
     <groupId>org.jetbrains.kotlinx</groupId>
-    <artifactId>kotlinx-serialization-runtime</artifactId>
+    <artifactId>kotlinx-serialization-core</artifactId>
     <version>${serialization.version}</version>
 </dependency>
 ```
-
-### Incompatible changes
-
-Library versions `0.20.0` and higher require Kotlin 1.3.70 and higher and incompatible with previous versions.
-
-Library version `0.14.0` require Kotlin 1.3.60/61 and incompatible with other versions.
-
-All versions of library before `0.13.0` are using Gradle metadata v0.4 and therefore it is recommended to use Gradle 4.8-5.1 to build.
-
-Library versions `0.11.0` and higher require Kotlin 1.3.30 and higher and incompatible with previous versions.
-
-All versions of library before `0.10.0` are using Gradle metadata v0.3 and therefore require Gradle 4.7 for build.
-
-Maven plugin coordinates before Kotlin 1.3.20 were `kotlinx-maven-serialization-plugin`.
-
-
-## Troubleshooting IntelliJ IDEA
-
-Serialization support should work out of the box, if you have `1.3.x` Kotlin plugin installed and have imported the project from Maven or Gradle with serialization enabled in their build scripts. 
-If you have Kotlin `1.3.10` or lower, you have to delegate build to Gradle (`Settings - Build, Execution, Deployment - Build Tools - Gradle - Runner -` tick `Delegate IDE build/run actions to gradle`). 
-Starting from `1.3.11`, no delegation is required.
-In case of problems, force project re-import from Gradle.

build.gradle

@@ -19,8 +19,8 @@ buildscript {
 
     ext.experimentalsInTestEnabled = ["-progressive", "-Xuse-experimental=kotlin.Experimental",
                                       "-Xuse-experimental=kotlin.ExperimentalMultiplatform",
-                                      "-Xuse-experimental=kotlinx.serialization.UnsafeSerializationApi",
-                                      "-Xuse-experimental=kotlinx.serialization.InternalSerializationApi"
+                                      "-Xuse-experimental=kotlinx.serialization.InternalSerializationApi",
+                                      "-Xuse-experimental=kotlin.ExperimentalUnsignedTypes"
     ]
 
     /*
@@ -71,6 +71,7 @@ buildscript {
         classpath "org.jetbrains.kotlin:kotlin-serialization:$kotlin_version"
         classpath "org.jetbrains.dokka:dokka-gradle-plugin:$dokka_version"
         classpath "org.jetbrains.kotlinx:binary-compatibility-validator:$validator_version"
+        classpath "org.jetbrains.kotlinx:kotlinx-knit:$knit_version"
 
         classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.8'
 
@@ -86,9 +87,19 @@ ext.compilerVersion = org.jetbrains.kotlin.config.KotlinCompilerVersion.VERSION
 apply plugin: 'binary-compatibility-validator'
 
 apiValidation {
-    ignoredProjects += ["benchmark", "kotlinx-serialization"]
+    ignoredProjects += ["benchmark", "guide", "kotlinx-serialization"]
 }
 
+apply plugin: 'base'
+apply plugin: 'kotlinx-knit'
+
+knit {
+    siteRoot = "https://kotlin.github.io/kotlinx.serialization"
+}
+
+// Build API docs for all modules with dokka before running Knit
+knitPrepare.dependsOn "dokka"
+
 allprojects {
     group 'org.jetbrains.kotlinx'
 
@@ -144,8 +155,10 @@ subprojects {
 
     apply from: rootProject.file('gradle/teamcity.gradle')
     // Configure publishing for some artifacts
-    if (project.name.contains("benchmark")) return
-    apply from: rootProject.file('gradle/publishing.gradle')
+    if (project.name != "benchmark" && project.name != "guide") {
+        apply from: rootProject.file('gradle/publishing.gradle')
+    }
+
 }
 
 apply from: rootProject.file('gradle/compilerVersion.gradle')