Skip to content

Commit 27074de

Browse files
nicholasnetnicholasnet
authored
Added GitHub template, expanded view for messages, and switched the protocol for message tailing. (#3)
1 parent 9426c31 commit 27074de

32 files changed

+1362
-561
lines changed

‎.github/ISSUE_TEMPLATE/PULL_REQUEST_TEMPLATE.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
<!-- ignore-task-list-start -->
2+
- [ ] **Breaking change?** (if so, please describe the impact and migration path for existing application instances)
3+
4+
5+
<!-- ignore-task-list-end -->
6+
**What changes did you make?** (Give an overview)
7+
8+
**Is there anything you'd like reviewers to focus on?**
9+
10+
11+
**How Has This Been Tested?** (put an "x" (case-sensitive!) next to an item)
12+
<!-- ignore-task-list-start -->
13+
- [ ] No need to
14+
- [ ] Manually (please, describe, if necessary)
15+
- [ ] Unit checks
16+
- [ ] Integration checks
17+
- [ ] Covered by existing automation
18+
<!-- ignore-task-list-end -->
19+
20+
**Checklist** (put an "x" (case-sensitive!) next to all the items, otherwise the build will fail)
21+
- [ ] I have performed a self-review of my own code
22+
- [ ] I have commented my code, particularly in hard-to-understand areas
23+
- [ ] I have made corresponding changes to the documentation (e.g. **ENVIRONMENT VARIABLES**)
24+
- [ ] I have added tests that prove my fix is effective or that my feature works
25+
- [ ] New and existing unit tests pass locally with my changes
26+
- [ ] Any dependent changes have been merged
27+
28+
Check out [Contributing](https://github.com/ideasbucketlabs/tansen/blob/main/documentation/CONTRIBUTING.md)

‎.github/ISSUE_TEMPLATE/bug_report.md

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
---
2+
name: "\U0001F41E Bug report"
3+
about: Create a bug report
4+
title: ''
5+
labels: status/triage, type/bug
6+
assignees: ''
7+
8+
---
9+
10+
<!--
11+
12+
We will close the issue without further explanation if you don't follow this template and don't provide the information requested within this template.
13+
14+
Don't forget to check for existing issues/discussions regarding your proposal. We might already have it.
15+
https://github.com/ideasbucketlabs/tansen/issues
16+
https://github.com/ideasbucketlabs/tansen/discussions
17+
18+
-->
19+
20+
<!--
21+
Please follow the naming conventions for bugs:
22+
<Feature/Area/Scope> : <Compact, but specific problem summary>
23+
Avoid generic titles, like “Topics: incorrect layout of message sorting drop-down list”. Better use something like: “Topics: Message sorting drop-down list overlaps the "Submit" button”.
24+
25+
-->
26+
27+
**Describe the bug** (Actual behavior)
28+
<!--(A clear and concise description of what the bug is.Use a list, if there is more than one problem)-->
29+
30+
**Expected behavior**
31+
<!--(A clear and concise description of what you expected to happen.)-->
32+
33+
**Set up**
34+
<!--
35+
WE MIGHT CLOSE THE ISSUE without further explanation IF YOU DON'T PROVIDE THIS INFORMATION.
36+
37+
How do you run the app? Please provide as much info as possible:
38+
1. App version (docker image version or check commit hash in the top left corner in UI)
39+
2. Helm chart version, if you use one
40+
3. Any IAAC configs
41+
-->
42+
43+
44+
**Steps to Reproduce**
45+
<!-- We'd like you to provide an example setup (via docker-compose, helm, etc.)
46+
to reproduce the problem, especially with a complex setups. -->
47+
48+
1.
49+
50+
**Screenshots**
51+
<!--
52+
(If applicable, please add screenshots to help explain your problem)
53+
-->
54+
55+
56+
**Additional context**
57+
<!--
58+
Add any other context about the problem here. E.g.:
59+
1. Are there any alternative scenarios (different data/methods/configuration/setup) you have tried?
60+
Were they successful or same issue occurred? Please provide steps as well.
61+
2. Related issues (if there are any).
62+
3. Logs (if available)
63+
4. Is there any serious impact or behaviour on the end-user because of this issue, that can be overlooked?
64+
-->

‎.github/ISSUE_TEMPLATE/feature_request.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
---
2+
name: "\U0001F680 Feature request"
3+
about: Propose a new feature
4+
title: ''
5+
labels: status/triage, type/feature
6+
assignees: ''
7+
8+
---
9+
10+
<!--
11+
12+
Don't forget to check for existing issues/discussions regarding your proposal. We might already have it.
13+
https://github.com/ideasbucketlabs/tansen/issues
14+
https://github.com/ideasbucketlabs/tansen/discussions
15+
16+
-->
17+
18+
### Which version of the app are you running?
19+
<!-- Please provide docker image version or check commit hash in the top left corner in UI) -->
20+
21+
### Is your proposal related to a problem?
22+
23+
<!--
24+
Provide a clear and concise description of what the problem is.
25+
For example, "I'm always frustrated when..."
26+
-->
27+
28+
### Describe the solution you'd like
29+
30+
<!--
31+
Provide a clear and concise description of what you want to happen.
32+
-->
33+
34+
### Describe alternatives you've considered
35+
36+
<!--
37+
Let us know about other solutions you've tried or researched.
38+
-->
39+
40+
### Additional context
41+
42+
<!--
43+
Is there anything else you can add about the proposal?
44+
You might want to link to related issues here, if you haven't already.
45+
-->
46+

‎.github/workflows/main.yaml

Whitespace-only changes.

‎.github/workflows/pr-checks.yaml

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
name: "PR Checklist checked"
2+
on:
3+
pull_request_target:
4+
types: [opened, edited, synchronize, reopened]
5+
6+
jobs:
7+
task-check:
8+
runs-on: ubuntu-latest
9+
steps:
10+
- uses: kentaro-m/[email protected]
11+
with:
12+
repo-token: "${{ secrets.GITHUB_TOKEN }}"
13+
- uses: dekinderfiets/[email protected]
14+
with:
15+
repo-token: "${{ secrets.GITHUB_TOKEN }}"

‎README.md

Lines changed: 21 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
1-
<p align="center">
1+
<p align="center" style="height: 100px">
22
<picture>
33
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/ideasbucketlabs/tansen/main/documentation/images/logo-dark.svg">
44
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/ideasbucketlabs/tansen/main/documentation/images/logo-light.svg">
5-
<img alt="Tansen" src="https://raw.githubusercontent.com/ideasbucketlabs/tansen/main/documentation/images/logo-light.svg" width="144" height="58" style="max-width: 100%;">
5+
<img alt="Tansen" src="https://raw.githubusercontent.com/ideasbucketlabs/tansen/main/documentation/images/logo-light.svg" width="194" height="108" style="max-width: 100%;">
66
</picture>
77
</p>
88

@@ -20,10 +20,10 @@
2020
## Features
2121

2222
* Multi-Cluster Management — see all your clusters in one place
23-
* Light and Dark theme based on your device preference
23+
* Light and Dark theme — based on your device preference
2424
* View Kafka Brokers — view topic and partition assignments, controller status
2525
* View Kafka Topics — view partition count, replication status, and custom configuration
26-
* Topic schema Management for key and message - view, edit and delete your schema
26+
* Topic schema Management for key and message — view, edit and delete your schema
2727
* View Consumer Groups — view per-partition parked offsets, combined and per-partition lag
2828
* Browse Messages — browse messages with JSON, plain text, Protobuf, and Avro encoding
2929
* Topic Configuration — create and configure new topics and edit existing one
@@ -32,9 +32,24 @@
3232

3333
## Getting started ##
3434

35-
To run Tansen for Apache Kafka, you can use either a pre-built Docker image or build it (or a jar file) yourself. For more information regarding configuration please visit [configuration page](https://github.com/ideasbucketlabs/tansen/blob/main/documentation/configuration.md).
35+
To run Tansen for Apache Kafka, you can use either a pre-built Docker image or build it (or a jar file) yourself. For running with Docker you can use this command.
36+
```shell
37+
docker run -d -p 8080:8080
38+
-e 'TANSEN.KAFKA-CLUSTERS.0.BOOTSTRAP_SERVERS=broker:9092' \
39+
-e 'TANSEN.KAFKA-CLUSTERS.0.NAME=local' \
40+
ideasbucket/tansen:tag
41+
```
3642

37-
---
43+
Please refer to our [configuration page](https://github.com/ideasbucketlabs/tansen/blob/main/documentation/configuration.md) for various configuration options that Tansen provides.
44+
45+
46+
### Liveliness and readiness probes
47+
48+
Liveliness and readiness endpoint is at `/insight/health`.
49+
50+
Info endpoint (build info) is located at `/insight/info`.
51+
52+
___
3853

3954
## Contributing
4055
If you're interested in contributing to Tansen, please read our [contributing docs](https://github.com/ideasbucketlabs/tansen/blob/main/documentation/CONTRIBUTING.md) **before submitting a pull request**.

‎backend/main/java/com/ideasbucket/tansen/entity/MessageSelectionCriteria.java

Lines changed: 7 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -7,17 +7,23 @@
77
package com.ideasbucket.tansen.entity;
88

99
import com.fasterxml.jackson.annotation.*;
10-
import jakarta.validation.constraints.AssertFalse;
10+
import jakarta.validation.constraints.Min;
11+
import jakarta.validation.constraints.NotNull;
12+
1113
import java.time.Instant;
1214

1315
@JsonInclude(JsonInclude.Include.NON_NULL)
1416
@JsonPropertyOrder({ "offset", "partition", "timestamp" })
1517
public final class MessageSelectionCriteria {
1618

1719
@JsonProperty("offset")
20+
@NotNull(message = "Offset cannot be null")
21+
@Min(value = 0, message = "Offset cannot be less than 0.")
1822
private final Long offset;
1923

2024
@JsonProperty("partition")
25+
@NotNull(message = "Partition cannot be null")
26+
@Min(value = 0, message = "Partition cannot be less than 0.")
2127
private final Integer partition;
2228

2329
@JsonProperty("timestamp")
@@ -46,18 +52,6 @@ public Instant getTimestamp() {
4652
return timestamp;
4753
}
4854

49-
@AssertFalse(message = "Invalid offset.")
50-
@JsonIgnore
51-
private boolean hasValidOffset() {
52-
return this.offset != null && this.offset < 0;
53-
}
54-
55-
@AssertFalse(message = "Invalid partition.")
56-
@JsonIgnore
57-
private boolean hasValidPartition() {
58-
return this.partition != null && this.partition < -1;
59-
}
60-
6155
@JsonIgnore
6256
public String getCase() {
6357
if (this.partition == null) {

‎backend/main/kotlin/com/ideasbucket/tansen/controller/api/MessagesController.kt

Lines changed: 11 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ import com.ideasbucket.tansen.util.JsonConverter
2121
import io.confluent.kafka.serializers.KafkaAvroDeserializer
2222
import io.confluent.kafka.serializers.json.KafkaJsonSchemaDeserializer
2323
import io.confluent.kafka.serializers.protobuf.KafkaProtobufDeserializer
24+
import jakarta.validation.Valid
2425
import kotlinx.coroutines.flow.Flow
2526
import kotlinx.coroutines.reactive.asFlow
2627
import org.apache.avro.generic.GenericData
@@ -31,6 +32,7 @@ import org.apache.kafka.common.serialization.StringDeserializer
3132
import org.slf4j.LoggerFactory
3233
import org.springframework.http.HttpMethod
3334
import org.springframework.http.MediaType
35+
import org.springframework.validation.annotation.Validated
3436
import org.springframework.web.bind.annotation.GetMapping
3537
import org.springframework.web.bind.annotation.PathVariable
3638
import org.springframework.web.bind.annotation.RequestMapping
@@ -45,7 +47,8 @@ import java.time.Duration
4547
import java.util.*
4648

4749
@RestController
48-
@RequestMapping("api/{clusterId:[a-zA-Z0-9][a-zA-Z0-9\\_\\-]+}/messages")
50+
@RequestMapping("api/{clusterId:[a-zA-Z0-9][a-zA-Z0-9\\_\\-]+}/messages/{keySerde:auto|string}/{valueSerde:auto|string}")
51+
@Validated
4952
class MessagesController(
5053
private val clusterService: ClusterService,
5154
private val schemaExecutor: SchemaExecutor,
@@ -59,7 +62,9 @@ class MessagesController(
5962
suspend fun getMessagesByTopic(
6063
@PathVariable clusterId: String,
6164
@PathVariable topic: String,
62-
@RequestParam(name = "parameters", required = false) criteria: MessageSelectionCriteria?
65+
@PathVariable keySerde: String,
66+
@PathVariable valueSerde: String,
67+
@Valid @RequestParam(name = "parameters", required = false) criteria: MessageSelectionCriteria?
6368
): Flow<ObjectNode> {
6469
val topicInformation =
6570
topicService.getTopic(clusterId, topic)
@@ -76,8 +81,8 @@ class MessagesController(
7681
throw UnknownTopicOrPartitionException("This server does not host this topic-partition.")
7782
}
7883

79-
val valueSchemaFormat = getSchemaFormat(clusterId, "$topic-value")
80-
val keySchemaFormat = getSchemaFormat(clusterId, "$topic-key")
84+
val valueSchemaFormat = if (valueSerde == "string") null else getSchemaFormat(clusterId, "$topic-value")
85+
val keySchemaFormat = if (keySerde == "string") null else getSchemaFormat(clusterId, "$topic-key")
8186
val uniqueId = UUID.randomUUID().toString()
8287

8388
val properties = Properties()
@@ -94,37 +99,19 @@ class MessagesController(
9499
properties["schema.registry.url"] = it
95100
}
96101

97-
// properties[ConsumerConfig.MAX_POLL_RECORDS_CONFIG] = 20
98-
99-
if ((criteria !== null) && (criteria.case == "timestamp")) {
102+
if (criteria !== null) {
100103
properties[ConsumerConfig.AUTO_OFFSET_RESET_CONFIG] = "earliest"
101104
}
102105

103106
val receiverOptions =
104-
if ((criteria != null) && (criteria.case == "offset") && (criteria.partition == -1)) {
105-
ReceiverOptions.create<Any, Any>(properties)
106-
.commitInterval(Duration.ZERO)
107-
.commitBatchSize(0)
108-
.addAssignListener { partitions ->
109-
partitions.forEach { it.seek(criteria.offset) }
110-
}
111-
.subscription(setOf(topic))
112-
} else if ((criteria != null) && (criteria.case == "offset")) {
107+
if ((criteria != null) && (criteria.case == "offset")) {
113108
ReceiverOptions.create<Any, Any>(properties)
114109
.commitInterval(Duration.ZERO)
115110
.commitBatchSize(0)
116111
.addAssignListener { partitions ->
117112
partitions.forEach { it.seek(criteria.offset) }
118113
}
119114
.assignment(setOf(TopicPartition(topic, criteria.partition)))
120-
} else if ((criteria != null) && (criteria.case == "timestamp") && (criteria.partition == -1)) {
121-
ReceiverOptions.create<Any, Any>(properties)
122-
.commitInterval(Duration.ZERO)
123-
.commitBatchSize(0)
124-
.addAssignListener { partitions ->
125-
partitions.forEach { it.seekToTimestamp(criteria.timestamp.epochSecond) }
126-
}
127-
.subscription(setOf(topic))
128115
} else if ((criteria != null) && (criteria.case == "timestamp")) {
129116
ReceiverOptions.create<Any, Any>(properties)
130117
.commitInterval(Duration.ZERO)

‎backend/main/kotlin/com/ideasbucket/tansen/controller/api/SubjectController.kt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
package com.ideasbucket.tansen.controller.api
88

99
import com.fasterxml.jackson.databind.JsonNode
10+
import com.ideasbucket.tansen.entity.MessageSelectionCriteria
1011
import com.ideasbucket.tansen.entity.Response
1112
import com.ideasbucket.tansen.entity.SaveSchemaRequest
1213
import com.ideasbucket.tansen.entity.TopicSchema

‎backend/main/resources/templates/errorTemplate.html

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,8 @@
33
<meta charset="utf-8">
44
<meta http-equiv="X-UA-Compatible" content="IE=edge">
55
<meta name="viewport" content="width=device-width,initial-scale=1.0">
6-
<link rel="preconnect" href="https://fonts.bunny.net">
7-
<link href="https://fonts.bunny.net/css?family=source-sans-pro:200,200i,400,400i,600,700,700i|source-serif-pro:200,200i,400,400i,700,700i"
8-
rel="stylesheet"/>
6+
<link rel="preconnect" href="https://fonts.bunny.net" crossorigin>
7+
<link href="https://fonts.bunny.net/css2?family=Source+Sans+Pro:ital,wght@0,200;0,400;0,600;1,200;1,400;1,600&family=Source+Serif+Pro:ital,wght@0,200;0,400;0,600;0,700;1,200;1,400;1,600;1,700&display=swap" rel="stylesheet">
98
<link rel="icon" href="/favicon.ico"/>
109
<link rel="apple-touch-icon" sizes="180x180" href="/assets/apple-touch-icon.png"
1110
th:href="@{/assets/apple-touch-icon.png}">

‎build.gradle.kts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
import com.github.gradle.node.npm.task.NpmTask
22

33
plugins {
4-
id("org.springframework.boot") version "3.0.5"
4+
id("org.springframework.boot") version "3.0.6"
55
id("io.spring.dependency-management") version "1.1.0"
66
kotlin("jvm") version "1.7.22"
77
kotlin("plugin.spring") version "1.7.22"

‎documentation/CONTRIBUTING.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,10 @@ For frontend
2020
```sh
2121
cd frontend && npm run lint
2222
```
23+
> **Note**
24+
> If you are creating Pull Requests please make that you run both commands `frontend` and `backend`
25+
26+
2327
## Running tests
2428

2529
You can run the test suite using the following commands:
@@ -56,10 +60,10 @@ tansen.kafka-clusters.0.name=local;
5660
tansen.kafka-clusters.0.schema_registry_url=http://localhost:8081/
5761
SPRING_PROFILES_ACTIVE=test
5862
```
59-
If you are using `IntelliJ` you can use following screenshot as reference.
63+
If you are using `IntelliJ` you can use following screenshot as a reference.
6064
<p>
6165
<picture>
62-
<img alt="Tansen Environment configuration" src="https://raw.githubusercontent.com/ideasbucketlabs/tansen/main/documentation/images/environment-settings.png" width="144" height="58" style="max-width: 100%;">
66+
<img alt="Tansen Environment configuration in IntelliJ" src="https://raw.githubusercontent.com/ideasbucketlabs/tansen/main/documentation/images/environment-settings.png" width="900" height="548" style="max-width: 100%;">
6367
</picture>
6468
</p>
6569

0 commit comments

Comments
 (0)