KAFKA-16932: Add documentation for the native docker image (apache#16338)

This PR contains the the following documentation changes for the native docker image:

in the docker/README.md: How to build, release and promote the native docker image.
in the tests/README.md: How to run system tests by bringing up kafka in the native mode.
added docker/native/README.md
added html changes for the kafka-site
added native docker image support in the docker compose files examples.

Testing:
Tested all the docker compose files with both the docker images - jvm and native
Tested the html changes locally with the kafka-site

Reviewers: Manikumar Reddy <[email protected]>, Vedarth Sharma <[email protected]>

Author: kagarwal06

docker/README.md

@@ -4,8 +4,10 @@ Docker Images
 Introduction
 ------------
 
-This directory contains scripts to build, test, push and promote docker image for kafka. It also contains scripts to build, test the JVM-based Docker Official Image, and generate a PR template for the same to be raised under the Docker official images repo. 
-All of the steps can be either performed locally or by using Github Actions.
+This directory contains:
+- Scripts to build, test, push and promote docker images for apache kafka - both `jvm` based and graalvm based `native` kafka
+- Scripts to build, test the JVM-based Docker Official Image, and generate a PR template for the same to be raised under the Docker official images repo.
+  All of the steps can be either performed locally or by using Github Actions.
 
 Github Actions
 --------------
@@ -27,41 +29,61 @@ Building image and running tests using github actions
 
 - kafka-url - This is the url to download kafka tarball from. For example kafka tarball url from (https://archive.apache.org/dist/kafka). For building RC image this will be an RC tarball url.
 
-- image-type - This is the type of image that we intend to build. This will be dropdown menu type selection in the workflow. `jvm` image type is for official docker image (to be hosted on apache/kafka) as described in [KIP-975](https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka)
+- image-type - This is the type of image that we intend to build. This will be dropdown menu type selection in the workflow.
+  - `jvm` image type is for official docker image (to be hosted on apache/kafka) as described in [KIP-975](https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka)
+  - `native` image type is for graalvm based `native` kafka docker image (to be hosted on apache/kafka-native) as described in [KIP-974](https://cwiki.apache.org/confluence/display/KAFKA/KIP-974%3A+Docker+Image+for+GraalVM+based+Native+Kafka+Broker#KIP974:DockerImageforGraalVMbasedNativeKafkaBroker-ImageNaming)
 
-- Example:-
+- Example(jvm):-
 To build and test a jvm image type ensuring kafka to be containerised should be https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz (it is recommended to use scala 2.13 binary tarball), following inputs in github actions workflow are recommended.
 ```
 image_type: jvm
 kafka_url: https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz
 ```
-
+- Example(native):-
+  To build and test a native image type ensuring kafka to be containerised should be https://archive.apache.org/dist/kafka/3.8.0/kafka_2.13-3.8.0.tgz (it is recommended to use scala 2.13 binary tarball), following inputs in github actions workflow are recommended.
+```
+image_type: native
+kafka_url: https://archive.apache.org/dist/kafka/3.8.0/kafka_2.13-3.8.0.tgz
+```
 Creating a Release Candidate using github actions
 -------------------------------------------------
 
 - This is the recommended way to push an RC docker image.
 - Go to `Build and Push Release Candidate Docker Image` Github Actions Workflow.
 - Choose the `image_type` and provide `kafka_url` that needs to be containerised in the `rc_docker_image` that will be pushed to github.
-- Example:-
+- Example(jvm):-
 If you want to push a jvm image which contains kafka from https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz to dockerhub under the namespace apache, repo name as kafka and image tag as 3.6.0-rc1 then following values need to be added in Github Actions Workflow:-
 ```
 image_type: jvm
 kafka_url: https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz
 rc_docker_image: apache/kafka:3.6.0-rc0
 ```
+- Example(native):-
+  If you want to push a native image which contains kafka from https://archive.apache.org/dist/kafka/3.8.0/kafka_2.13-3.8.0.tgz to dockerhub under the namespace apache, repo name as kafka-native and image tag as 3.8.0-rc0 then following values need to be added in Github Actions Workflow:-
+```
+image_type: native
+kafka_url: https://archive.apache.org/dist/kafka/3.8.0/kafka_2.13-3.8.0.tgz
+rc_docker_image: apache/kafka-native:3.8.0-rc0
+```
 
 Promoting a Release Candidate using github actions
 --------------------------------------------------
 
 - This is the recommended way to promote an RC docker image.
 - Go to `Promote Release Candidate Docker Image` Github Actions Workflow.
 - Choose the RC docker image (`rc_docker_image`) that you want to promote and where it needs to be pushed to (`promoted_docker_image`), i.e. the final docker image release.
-- Example:-
+- Example(jvm):-
 If you want to promote apache/kafka:3.6.0-rc0 RC docker image to apache/kafka:3.6.0 then following parameters can be provided to the workflow.
 ```
 rc_docker_image: apache/kafka:3.6.0-rc0
 promoted_docker_image: apache/kafka:3.6.0
 ```
+- Example(native):-
+  If you want to promote apache/kafka-native:3.8.0-rc0 RC docker image to apache/kafka-native:3.8.0 then following parameters can be provided to the workflow.
+```
+rc_docker_image: apache/kafka-native:3.8.0-rc0
+promoted_docker_image: apache/kafka-native:3.8.0
+```
 
 Cron job for checking CVEs in supported docker images
 -----------------------------------------------------
@@ -100,22 +122,32 @@ Building image and running tests locally
 - Sanity tests for the docker image are present in test/docker_sanity_test.py.
 - By default image will be built and tested, but if you only want to build the image, pass `--build` (or `-b`) flag and if you only want to test the given image pass `--test` (or `-t`) flag.
 - An html test report will be generated after the tests are executed containing the results.
-- Example :- To build and test an image named test under kafka namespace with 3.6.0 tag and jvm image type ensuring kafka to be containerised should be https://downloads.apache.org/kafka/3.6.0/kafka_2.13-3.6.0.tgz (it is recommended to use scala 2.13 binary tarball), following command can be used
+- Example(jvm) :- To build and test an image named test under kafka namespace with 3.6.0 tag and jvm image type ensuring kafka to be containerised should be https://downloads.apache.org/kafka/3.6.0/kafka_2.13-3.6.0.tgz (it is recommended to use scala 2.13 binary tarball), following command can be used
 ```
 python docker_build_test.py kafka/test --image-tag=3.6.0 --image-type=jvm --kafka-url=https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz
 ```
+- Example(native) :- To build and test an image named test under kafka namespace with 3.8.0 tag and native image type ensuring kafka to be containerised should be https://downloads.apache.org/kafka/3.8.0/kafka_2.13-3.8.0.tgz (it is recommended to use scala 2.13 binary tarball), following command can be used
+```
+python docker_build_test.py kafka/test --image-tag=3.8.0 --image-type=native --kafka-url=https://archive.apache.org/dist/kafka/3.8.0/kafka_2.13-3.8.0.tgz
+```
 
 Creating a Release Candidate
 ----------------------------
 
 - `docker_release.py` script builds a multi-architecture image and pushes it to provided docker registry.
 - Ensure you are logged in to the docker registry before triggering the script.
 - kafka binary tarball url along with image name (in the format `<registry>/<namespace>/<image_name>:<image_tag>`) and type is needed to build the image. For detailed usage description check `python docker_release.py --help`.
-- Example:- To push an image named test under kafka dockerhub namespace with 3.6.0 tag and jvm image type ensuring kafka to be containerised should be https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz (it is recommended to use scala 2.13 binary tarball), following command can be used. (Make sure you have push access to the docker repo)
+- Example(jvm):- To push an image named test under kafka dockerhub namespace with 3.6.0 tag and jvm image type ensuring kafka to be containerised should be https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz (it is recommended to use scala 2.13 binary tarball), following command can be used. (Make sure you have push access to the docker repo)
 ```
 # kafka/test is an example repo. Please replace with the docker hub repo you have push access to.
 
-python docker_release.py kafka/test:3.6.0 --kafka-url https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz
+python docker_release.py kafka/test:3.6.0 --kafka-url --image-type=jvm https://archive.apache.org/dist/kafka/3.6.0/kafka_2.13-3.6.0.tgz
+```
+- Example(native):- To push an image named test under kafka-native dockerhub namespace with 3.8.0 tag and native image type ensuring kafka to be containerised should be https://archive.apache.org/dist/kafka/3.8.0/kafka_2.13-3.8.0.tgz (it is recommended to use scala 2.13 binary tarball), following command can be used. (Make sure you have push access to the docker repo)
+```
+# kafka-native/test is an example repo. Please replace with the docker hub repo you have push access to.
+
+python docker_release.py kafka-native/test:3.8.0 --kafka-url --image-type=native https://archive.apache.org/dist/kafka/3.8.0/kafka_2.13-3.8.0.tgz
 ```
 
 - Please note that we use docker buildx for preparing the multi-architecture image and pushing it to docker registry. It's possible to encounter build failures because of buildx. Please retry the command in case some buildx related error occurs.
@@ -124,12 +156,18 @@ Promoting a Release Candidate
 -----------------------------
 
 - It's not recommended to promote the docker image locally, as we have github actions doing it in a convenient way, but if needed following command can be used to promote a Release Candidate image.
-- Example:- If you want to promote RC image apache/kafka:3.6.0-rc0 to apache/kafka:3.6.0, following command can be used
+- Example(jvm):- If you want to promote RC image apache/kafka:3.6.0-rc0 to apache/kafka:3.6.0, following command can be used
 
 ```
 # Ensure docker buildx is enabled in your system and you have access to apache/kafka
 docker buildx imagetools create --tag apache/kafka:3.6.0 apache/kafka:3.6.0-rc0
 ```
+- Example(native):- If you want to promote RC image apache/kafka-native:3.8.0-rc0 to apache/kafka-native:3.8.0, following command can be used
+
+```
+# Ensure docker buildx is enabled in your system and you have access to apache/kafka-native
+docker buildx imagetools create --tag apache/kafka-native:3.8.0 apache/kafka-native:3.8.0-rc0
+```
 
 Using the image in a docker container
 -------------------------------------
@@ -172,5 +210,6 @@ python generate_kafka_pr_template.py --image-type=jvm
 ```
 
 - kafka-version - This is the version to create the Docker official images static Dockerfile and assets for, as well as the version to build and test the Docker official image for.
-- image-type - This is the type of image that we intend to build. This will be dropdown menu type selection in the workflow. `jvm` image type is for official docker image (to be hosted on apache/kafka) as described in [KIP-975](https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka). As of now [KIP-1028](https://cwiki.apache.org/confluence/display/KAFKA/KIP-1028%3A+Docker+Official+Image+for+Apache+Kafka) only aims to release JVM based Docker Official Images.
+- image-type - This is the type of image that we intend to build. This will be dropdown menu type selection in the workflow. `jvm` image type is for official docker image (to be hosted on apache/kafka) as described in [KIP-975](https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka). 
+  - **NOTE:** As of now [KIP-1028](https://cwiki.apache.org/confluence/display/KAFKA/KIP-1028%3A+Docker+Official+Image+for+Apache+Kafka) only aims to release JVM based Docker Official Images and not GraalVM based native Apache Kafka docker image.