refactor(docs): group docs in categories (numaproj#362)

Signed-off-by: Derek Wang <[email protected]>
dseapy · Nov 17, 2022 · f817057 · f817057
1 parent 3588bf3
commit f817057
Show file tree

Hide file tree

Showing 39 changed files with 57 additions and 49 deletions.
diff --git a/docs/debugging.md → docs/development/debugging.md b/docs/debugging.md → docs/development/debugging.md
diff --git a/docs/development.md → docs/development/development.md b/docs/development.md → docs/development/development.md
diff --git a/docs/releasing.md → docs/development/releasing.md b/docs/releasing.md → docs/development/releasing.md
diff --git a/docs/static-code-analysis.md → docs/development/static-code-analysis.md b/docs/static-code-analysis.md → docs/development/static-code-analysis.md
diff --git a/docs/controller-configmap.md → docs/operations/controller-configmap.md b/docs/controller-configmap.md → docs/operations/controller-configmap.md
@@ -22,7 +22,7 @@ data:
 
 ### ISB Service Configuration
 
-One of the important configuration items in the ConfigMap is about [ISB Service](./inter-step-buffer-service.md). We currently use 3rd party technologies such as `JetStream` to implement ISB Services, if those applications have new releases, to make them available in Numaflow, the new versions need to be added in the ConfigMap.
+One of the important configuration items in the ConfigMap is about [ISB Service](../user-guide/inter-step-buffer-service.md). We currently use 3rd party technologies such as `JetStream` to implement ISB Services, if those applications have new releases, to make them available in Numaflow, the new versions need to be added in the ConfigMap.
 
 For example, there's a new `Nats JetStream` version `x.y.x` available, a new version configuration like below needs to be added before it can be referenced in the `InterStepBufferService` spec.
 

diff --git a/docs/installation.md → docs/operations/installation.md b/docs/installation.md → docs/operations/installation.md
diff --git a/docs/metrics/metrics.md → docs/operations/metrics/metrics.md b/docs/metrics/metrics.md → docs/operations/metrics/metrics.md
diff --git a/docs/numaflow-controller-config.yaml → ...perations/numaflow-controller-config.yaml b/docs/numaflow-controller-config.yaml → ...perations/numaflow-controller-config.yaml
diff --git a/docs/releases.md → docs/operations/releases.md b/docs/releases.md → docs/operations/releases.md
diff --git a/docs/quick-start.md b/docs/quick-start.md
@@ -16,6 +16,7 @@ You will also need `kubectl` to manage the cluster.
 ## Installation
 
 Run the following command lines to install Numaflow and start the Inter-Step Buffer Service that handles communication between vertices.
+
 ```shell
 kubectl create ns numaflow-system
 kubectl apply -n numaflow-system -f https://raw.githubusercontent.com/numaproj/numaflow/stable/config/install.yaml
@@ -25,6 +26,7 @@ kubectl apply -f https://raw.githubusercontent.com/numaproj/numaflow/stable/exam
 ## A Simple Pipeline
 
 Create a `simple pipeline`, which contains a source vertex to generate messages, a processing vertex that echos the messages, and a sink vertex that logs the messages.
+
 ```shell
 kubectl apply -f https://raw.githubusercontent.com/numaproj/numaflow/stable/examples/1-simple-pipeline.yaml
 kubectl get pipeline # or "pl" as a short name
@@ -56,20 +58,23 @@ kubectl logs -f simple-pipeline-out-0-xxxx
 # Port forward the UI to https://localhost:8443/
 kubectl -n numaflow-system port-forward deployment/numaflow-server 8443:8443
 ```
+
 ![Numaflow UI](assets/numaflow-ui-simple-pipeline.png)
 
 The pipeline can be deleted by
+
 ```shell
 kubectl delete -f https://raw.githubusercontent.com/numaproj/numaflow/stable/examples/1-simple-pipeline.yaml
 ```
 
 ## An Advanced Pipeline
 
-In this example, there are five vertices in a pipeline. An [HTTP](./sources/http.md) source vertex which serves an HTTP endpoint to receive numbers as source data, a [UDF](./user-defined-functions.md) vertex to tag the ingested numbers with the key `even` or `odd`, three [Log](./sinks/log.md) sinks, one to print the `even` numbers, one to print the `odd` numbers, and the other one to print both the even and odd numbers.
+In this example, there are five vertices in a pipeline. An [HTTP](./user-guide/sources/http.md) source vertex which serves an HTTP endpoint to receive numbers as source data, a [UDF](./user-guide/user-defined-functions.md) vertex to tag the ingested numbers with the key `even` or `odd`, three [Log](./user-guide/sinks/log.md) sinks, one to print the `even` numbers, one to print the `odd` numbers, and the other one to print both the even and odd numbers.
 
 ![Pipeline Diagram](assets/even-odd.png)
 
 Create the `even-odd` pipeline.
+
 ```shell
 kubectl apply -f https://raw.githubusercontent.com/numaproj/numaflow/stable/test/e2e/testdata/even-odd.yaml
 
@@ -91,9 +96,9 @@ kubectl port-forward even-odd-in-0-xxxx 8444:8443
 
 # Post data to the HTTP endpoint
 curl -kq -X POST -d "101" https://localhost:8444/vertices/in
-curl -kq -X POST -d "102" https://localhost:8444/vertices/in 
+curl -kq -X POST -d "102" https://localhost:8444/vertices/in
 curl -kq -X POST -d "103" https://localhost:8444/vertices/in
-curl -kq -X POST -d "104" https://localhost:8444/vertices/in 
+curl -kq -X POST -d "104" https://localhost:8444/vertices/in
 
 # Watch the log for the even vertex
 kubectl logs -f even-odd-even-sink-0-nf2ql
@@ -109,9 +114,10 @@ kubectl logs -f even-odd-odd-sink-0-a6p0n
 View the UI for the advanced pipeline at https://localhost:8443/
 ![Numaflow UI](assets/numaflow-ui-advanced-pipeline.png)
 
-The source code of the `even-odd` [User Defined Function](./user-defined-functions.md) can be found [here](https://github.com/numaproj/numaflow-go/tree/main/pkg/function/examples/evenodd). You also can replace the [Log](./sinks/log.md) Sink with some other sinks like [Kafka](./sinks/kafka.md) to forward the data to Kafka topics.
+The source code of the `even-odd` [User Defined Function](./user-guide/user-defined-functions.md) can be found [here](https://github.com/numaproj/numaflow-go/tree/main/pkg/function/examples/evenodd). You also can replace the [Log](./user-guide/sinks/log.md) Sink with some other sinks like [Kafka](./user-guide/sinks/kafka.md) to forward the data to Kafka topics.
 
 The pipeline can be deleted by
+
 ```shell
 kubectl delete -f https://raw.githubusercontent.com/numaproj/numaflow/stable/test/e2e/testdata/even-odd.yaml
 ```
@@ -120,4 +126,4 @@ kubectl delete -f https://raw.githubusercontent.com/numaproj/numaflow/stable/tes
 
 Try more examples in the [`examples`](https://github.com/numaproj/numaflow/tree/main/examples) directory.
 
-After exploring how Numaflow pipeline run, you can check what data [Sources](./sources/generator.md) and [Sinks](./sinks/kafka.md) Numaflow supports out of the box, or learn how to write [User Defined Functions](./user-defined-functions.md).
+After exploring how Numaflow pipeline run, you can check what data [Sources](./user-guide/sources/generator.md) and [Sinks](./user-guide/sinks/kafka.md) Numaflow supports out of the box, or learn how to write [User Defined Functions](./user-guide/user-defined-functions.md).
diff --git a/docs/specs/autoscaling.md → docs/specifications/autoscaling.md b/docs/specs/autoscaling.md → docs/specifications/autoscaling.md
@@ -62,4 +62,4 @@ When the calculated desired replicas is greater than current replicas:
 
 ## Autoscaling Tuning
 
-Numaflow autoscaling can be tuned by updating some parameters, find the details at the [doc](../autoscaling.md).
+Numaflow autoscaling can be tuned by updating some parameters, find the details at the [doc](../user-guide/autoscaling.md).
diff --git a/docs/specs/overview.md → docs/specifications/overview.md b/docs/specs/overview.md → docs/specifications/overview.md
diff --git a/docs/autoscaling.md → docs/user-guide/autoscaling.md b/docs/autoscaling.md → docs/user-guide/autoscaling.md
diff --git a/docs/builtin-functions/README.md → docs/user-guide/builtin-functions/README.md b/docs/builtin-functions/README.md → docs/user-guide/builtin-functions/README.md
diff --git a/docs/builtin-functions/cat.md → docs/user-guide/builtin-functions/cat.md b/docs/builtin-functions/cat.md → docs/user-guide/builtin-functions/cat.md
diff --git a/docs/builtin-functions/filter.md → docs/user-guide/builtin-functions/filter.md b/docs/builtin-functions/filter.md → docs/user-guide/builtin-functions/filter.md
diff --git a/docs/conditional-forwarding.md → docs/user-guide/conditional-forwarding.md b/docs/conditional-forwarding.md → docs/user-guide/conditional-forwarding.md
diff --git a/docs/container-resources.md → docs/user-guide/container-resources.md b/docs/container-resources.md → docs/user-guide/container-resources.md
diff --git a/docs/environment-variables.md → docs/user-guide/environment-variables.md b/docs/environment-variables.md → docs/user-guide/environment-variables.md
@@ -1,6 +1,6 @@
 # Environment Variables
 
-For the `numa` container of vertex pods, environment variable `NUMAFLOW_DEBUG` can be set to `true` for [debugging](./debugging.md).
+For the `numa` container of vertex pods, environment variable `NUMAFLOW_DEBUG` can be set to `true` for [debugging](../development/debugging.md).
 
 In [`udf`](./user-defined-functions.md) and [`udsink`](./sinks/user-defined-sinks.md) containers, there are some preset environment variables that can be used directly.
 

diff --git a/docs/init-containers.md → docs/user-guide/init-containers.md b/docs/init-containers.md → docs/user-guide/init-containers.md
diff --git a/docs/inter-step-buffer-service.md → docs/user-guide/inter-step-buffer-service.md b/docs/inter-step-buffer-service.md → docs/user-guide/inter-step-buffer-service.md
@@ -25,7 +25,9 @@ spec:
   # Optional, if not specified, defaults to "default"
   interStepBufferServiceName: different-name
 ```
+
 To query `Inter-Step Buffer Service` objects with `kubectl`:
+
 ```sh
 kubectl get isbsvc
 ```
@@ -137,7 +139,7 @@ Once a JetStream ISB Service is created, toggling the `encryption` field will ca
 
 ### Other Configuration
 
-Check [here](APIs.md#numaflow.numaproj.io/v1alpha1.JetStreamBufferService) for the full spec of `spec.jetstream`.
+Check [here](../APIs.md#numaflow.numaproj.io/v1alpha1.JetStreamBufferService) for the full spec of `spec.jetstream`.
 
 ## Redis
 
@@ -147,6 +149,7 @@ Check [here](APIs.md#numaflow.numaproj.io/v1alpha1.JetStreamBufferService) for t
 We also support external redis.
 
 #### External Redis
+
 If you have a managed Redis, say in AWS, etc., we can make that Redis your ISB. All you need to do is provide the external Redis endpoint name.
 
 ```yaml
@@ -163,15 +166,14 @@ spec:
 
 ### Cluster Mode
 
-We support [cluster mode](https://redis.io/docs/reference/cluster-spec/), only if the Redis is an external managed Redis. 
-You will have to enter the `url` twice to indicate that the mode is cluster. This is because we use `Universal Client` which 
+We support [cluster mode](https://redis.io/docs/reference/cluster-spec/), only if the Redis is an external managed Redis.
+You will have to enter the `url` twice to indicate that the mode is cluster. This is because we use `Universal Client` which
 requires more than one address to indicate the Redis is in cluster mode.
 
 ```yaml
 url: "numaflow-redis-cluster-0.numaflow-redis-cluster-headless:6379,numaflow-redis-cluster-1.numaflow-redis-cluster-headless:6379"
 ```
 
-
 ### Version
 
 Property `spec.redis.native.version` is required for a `native` Redis `InterStepBufferService`. Supported versions can be found from the ConfigMap `numaflow-controller-config` in the control plane namespace.
@@ -243,4 +245,4 @@ Here is the [reference](https://github.com/redis/redis/blob/unstable/redis.conf)
 
 ### Other Configuration
 
-Check [here](APIs.md#numaflow.numaproj.io/v1alpha1.NativeRedis) for the full spec of `spec.redis.native`.
+Check [here](../APIs.md#numaflow.numaproj.io/v1alpha1.NativeRedis) for the full spec of `spec.redis.native`.
diff --git a/docs/inter-step-buffer.md → docs/user-guide/inter-step-buffer.md b/docs/inter-step-buffer.md → docs/user-guide/inter-step-buffer.md
diff --git a/docs/pipeline-customization.md → docs/user-guide/pipeline-customization.md b/docs/pipeline-customization.md → docs/user-guide/pipeline-customization.md
diff --git a/docs/pipeline-tuning.md → docs/user-guide/pipeline-tuning.md b/docs/pipeline-tuning.md → docs/user-guide/pipeline-tuning.md
diff --git a/docs/pipeline.md → docs/user-guide/pipeline.md b/docs/pipeline.md → docs/user-guide/pipeline.md
diff --git a/docs/sidecar-containers.md → docs/user-guide/sidecar-containers.md b/docs/sidecar-containers.md → docs/user-guide/sidecar-containers.md
diff --git a/docs/sinks/blackhole.md → docs/user-guide/sinks/blackhole.md b/docs/sinks/blackhole.md → docs/user-guide/sinks/blackhole.md
diff --git a/docs/sinks/kafka.md → docs/user-guide/sinks/kafka.md b/docs/sinks/kafka.md → docs/user-guide/sinks/kafka.md
diff --git a/docs/sinks/log.md → docs/user-guide/sinks/log.md b/docs/sinks/log.md → docs/user-guide/sinks/log.md
diff --git a/docs/sinks/user-defined-sinks.md → docs/user-guide/sinks/user-defined-sinks.md b/docs/sinks/user-defined-sinks.md → docs/user-guide/sinks/user-defined-sinks.md
diff --git a/docs/sources/generator.md → docs/user-guide/sources/generator.md b/docs/sources/generator.md → docs/user-guide/sources/generator.md
diff --git a/docs/sources/http.md → docs/user-guide/sources/http.md b/docs/sources/http.md → docs/user-guide/sources/http.md
diff --git a/docs/sources/kafka.md → docs/user-guide/sources/kafka.md b/docs/sources/kafka.md → docs/user-guide/sources/kafka.md
diff --git a/docs/user-defined-functions.md → docs/user-guide/user-defined-functions.md b/docs/user-defined-functions.md → docs/user-guide/user-defined-functions.md
diff --git a/docs/vertex.md → docs/user-guide/vertex.md b/docs/vertex.md → docs/user-guide/vertex.md
diff --git a/docs/volumes.md → docs/user-guide/volumes.md b/docs/volumes.md → docs/user-guide/volumes.md
diff --git a/docs/watermarks.md → docs/user-guide/watermarks.md b/docs/watermarks.md → docs/user-guide/watermarks.md
diff --git a/examples/0-isbsvc-jetstream.yaml b/examples/0-isbsvc-jetstream.yaml
@@ -1,6 +1,6 @@
 #
 # This manifest is intended for demonstration purpose, it's not suitable for production.
-# Check https://numaflow.numaproj.io/inter-step-buffer-service/ to figure out reliable configuration for production.
+# Check https://numaflow.numaproj.io/user-guide/inter-step-buffer-service/ to figure out reliable configuration for production.
 #
 apiVersion: numaflow.numaproj.io/v1alpha1
 kind: InterStepBufferService

diff --git a/mkdocs.yml b/mkdocs.yml
@@ -40,48 +40,48 @@ nav:
       - quick-start.md
   - User Guide:
       - Core Concepts:
-          - pipeline.md
-          - vertex.md
-          - inter-step-buffer.md
-          - inter-step-buffer-service.md
+          - user-guide/pipeline.md
+          - user-guide/vertex.md
+          - user-guide/inter-step-buffer.md
+          - user-guide/inter-step-buffer-service.md
       - Sources:
-          - sources/generator.md
-          - sources/http.md
-          - sources/kafka.md
+          - user-guide/sources/generator.md
+          - user-guide/sources/http.md
+          - user-guide/sources/kafka.md
       - Sinks:
-          - sinks/kafka.md
-          - sinks/log.md
-          - sinks/blackhole.md
-          - User Defined Sinks: "sinks/user-defined-sinks.md"
+          - user-guide/sinks/kafka.md
+          - user-guide/sinks/log.md
+          - user-guide/sinks/blackhole.md
+          - User Defined Sinks: "user-guide/sinks/user-defined-sinks.md"
       - User Defined Functions:
-          - Overview: "user-defined-functions.md"
+          - Overview: "user-guide/user-defined-functions.md"
           - Built-in UDFs:
-              - Overview: "builtin-functions/README.md"
-              - Cat: "builtin-functions/cat.md"
-              - Filter: "builtin-functions/filter.md"
-      - pipeline-tuning.md
-      - conditional-forwarding.md
-      - autoscaling.md
-      - container-resources.md
-      - volumes.md
-      - environment-variables.md
-      - init-containers.md
-      - sidecar-containers.md
-      - pipeline-customization.md
-      - watermarks.md
+              - Overview: "user-guide/builtin-functions/README.md"
+              - Cat: "user-guide/builtin-functions/cat.md"
+              - Filter: "user-guide/builtin-functions/filter.md"
+      - user-guide/pipeline-tuning.md
+      - user-guide/conditional-forwarding.md
+      - user-guide/autoscaling.md
+      - user-guide/container-resources.md
+      - user-guide/volumes.md
+      - user-guide/environment-variables.md
+      - user-guide/init-containers.md
+      - user-guide/sidecar-containers.md
+      - user-guide/pipeline-customization.md
+      - user-guide/watermarks.md
       - APIs.md
   - Operator Manual:
-      - Releases ⧉: "releases.md"
-      - installation.md
+      - Releases ⧉: "operations/releases.md"
+      - operations/installation.md
       - Configuration:
-          - Controller Configuration: "controller-configmap.md"
-          - metrics/metrics.md
+          - Controller Configuration: "operations/controller-configmap.md"
+          - operations/metrics/metrics.md
   - Developer Guide:
-      - development.md
+      - development/development.md
       - Specifications:
-          - Overview: "specs/overview.md"
-          - "specs/autoscaling.md"
-      - debugging.md
-      - static-code-analysis.md
-      - releasing.md
+          - Overview: "specifications/overview.md"
+          - "specifications/autoscaling.md"
+      - development/debugging.md
+      - development/static-code-analysis.md
+      - development/releasing.md
   - Numaproj: https://numaproj.io
Original file line number	Diff line number	Diff line change
Expand Up		@@ -62,4 +62,4 @@ When the calculated desired replicas is greater than current replicas:

		## Autoscaling Tuning

		Numaflow autoscaling can be tuned by updating some parameters, find the details at the [doc](../autoscaling.md).
		Numaflow autoscaling can be tuned by updating some parameters, find the details at the [doc](../user-guide/autoscaling.md).