Skip to content

Commit 3c6b91b

Browse files
lfranckelfrancke
committed
Merge branch 'main' into feat/hadoop-rework
# Conflicts: # hadoop/Dockerfile
2 parents ce03769 + 1e5f085 commit 3c6b91b

File tree

179 files changed

+9358
-2986
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

179 files changed

+9358
-2986
lines changed

‎.gitattributes

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
nix/** linguist-generated

‎.github/ISSUE_TEMPLATE/add-product.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
---
2+
name: Add new Product
3+
about: >-
4+
This template contains instructions specific to adding a new product.
5+
title: >-
6+
tracking: Add new product <PRODUCT>
7+
labels: []
8+
# Currently, projects cannot be assigned via front-matter.
9+
projects: ['stackabletech/10']
10+
assignees: ''
11+
---
12+
13+
```[tasklist]
14+
### Tasks
15+
- [ ] Create a new top-level folder for the product. The name of the folder must
16+
use the lowercase product name.
17+
- [ ] Create a README.md file outlining special considerations required to
18+
update or run the product. See existing README files as a guide of
19+
reference.
20+
- [ ] Add a `versions.py` file to the folder. Add all required key-value pairs.
21+
- [ ] Add a new "Update Product" issue template in `.github/ISSUE_TEMPLATE/`
22+
folder. See existing ones as a guide of reference.
23+
- [ ] Add a new `dev_<PRODUCT>.yml` GitHub Action workflow in the
24+
`.github/workflows` folder. Use existing local action whenever possible
25+
or consider creating a new one when there is no fitting action available.
26+
- [ ] Run `.scripts/update_readme_badges.sh` to generate the new status badge.
27+
```
28+
29+
_Please consider updating this template if these instructions are wrong, or
30+
could be made clearer._

‎.github/ISSUE_TEMPLATE/early-pre-release.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
---
2+
name: Early Pre-Release Container Image Updates
3+
about: This template can be used to track the container image updates leading up to the next Stackable release
4+
title: "chore: Update Container Images for Stackable Release XX.(X)X"
5+
labels: ['epic']
6+
assignees: ''
7+
---
8+
9+
<!--
10+
DO NOT REMOVE THIS COMMENT. It is intended for people who might copy/paste from the previous release issue.
11+
This was created by an issue template: https://github.com/stackabletech/docker-images/issues/new/choose.
12+
-->
13+
14+
<!-- Update this with the parent tracking issue for the release -->
15+
Part of stackabletech/issues#xxx.
16+
17+
## Container Image Updates for Stackable Release XX.(X)X
18+
19+
> [!NOTE]
20+
> Update the product versions based on what has been decided upon in the _Product Spreadsheet[^1]_.
21+
22+
[^1]: Currently this is a private spreadsheet
23+
24+
Replace the items in the task lists below with the subsequent tracking issue.
25+
26+
<!--
27+
Find templates for bases/products:
28+
29+
find .github/ISSUE_TEMPLATE/update-*.md -printf "%f\n" \
30+
| sort \
31+
| xargs -I {} echo "- [ ] https://github.com/stackabletech/docker-images/issues/new?template={}"
32+
-->
33+
34+
<!-- todo: consider removing the ubi*-rust-builder from the release process. -->
35+
```[tasklist]
36+
### Product Container Images
37+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-base-java.md
38+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-base-stackable.md
39+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-base-ubi-rust-builders.md
40+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-base-vector.md
41+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-product-airflow.md
42+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-product-druid.md
43+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-product-hbase-phoenix-omid.md
44+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-product-hdfs.md
45+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-product-hive.md
46+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-product-kafka.md
47+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-product-nifi.md
48+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-product-opa.md
49+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-product-spark.md
50+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-product-superset.md
51+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-product-trino.md
52+
- [ ] https://github.com/stackabletech/docker-images/issues/new?template=update-product-zookeeper.md
53+
```
54+
55+
```[tasklist]
56+
### Additional items which don't have a tracking issue
57+
- [ ] hello-world
58+
- [ ] krb5
59+
- [ ] tools
60+
- [ ] testing-tools
61+
- [ ] statsd_exporter
62+
```

‎.github/ISSUE_TEMPLATE/pre-release.md

Lines changed: 0 additions & 76 deletions
This file was deleted.

‎.github/ISSUE_TEMPLATE/update-base-java.md

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
---
2+
name: Update Java Bases
3+
about: >-
4+
This template contains instructions specific to updating this product and/or
5+
container image(s).
6+
title: >-
7+
chore(java-bases): Update container images ahead of Stackable Release XX.(X)X
8+
labels: []
9+
# Currently, projects cannot be assigned via front-matter.
10+
projects: ['stackabletech/10']
11+
assignees: ''
12+
---
13+
14+
Part of #xxx.
15+
16+
<!--
17+
This gives hints to the person doing the work.
18+
Add/Change/Remove anything that isn't applicable anymore
19+
-->
20+
- Add: `x.x.x`
21+
- Remove: `y.y.y`
22+
23+
> [!TIP]
24+
> Please add the `scheduled-for/20XX-XX` label, and add to the [Stackable Engineering][1] project.
25+
>
26+
> [1]: https://github.com/orgs/stackabletech/projects/10
27+
28+
<!-- markdownlint-disable-next-line MD028 -->
29+
> [!IMPORTANT]
30+
>
31+
> - `java-base` uses [vector](https://github.com/stackabletech/docker-images/blob/main/vector/Dockerfile) and is used as a base for java products.
32+
> - `java-devel` uses [stackable-base](https://github.com/stackabletech/docker-images/blob/main/stackable-base/Dockerfile) and is used to build java products.
33+
34+
Typically product updates will determine which version of Java is required, but
35+
we should also make new versions of Java available for use.
36+
37+
> [!TIP]
38+
> You can search for available java versions at [rpmfind.net], and search the
39+
> term `openjdk-headless`.
40+
> _It isn't perfect, as it will depend on what is available via microdnf._
41+
42+
```[tasklist]
43+
### Update tasks
44+
- [ ] Add any new versions of java to both `java-base/versions.py` and `java-devel/versions.py`
45+
- [ ] Remove versions when there are no long any references (eg: `grep java- **/versions.py | grep "1.8.0"`)
46+
```
47+
48+
```[tasklist]
49+
### Related Pull Requests
50+
- [ ] _Link to the docker-images PR (product update)_
51+
```
52+
53+
This list should be completed by the assignee(s), once respective PRs have been merged. Once all items have been checked, the issue can be moved into _Development: Done_.
54+
55+
```[tasklist]
56+
### Acceptance
57+
- [ ] Can build a product image that uses the new version(s)
58+
- [ ] Both `java-base` and `java-devel` have the same Java versions in `versions.py`
59+
- [ ] Kuttl smoke test passes locally for a product using the new Java version
60+
```
61+
62+
<details>
63+
<summary>Testing instructions</summary>
64+
65+
```shell
66+
# See the latest version at https://pypi.org/project/image-tools-stackabletech/
67+
pip install image-tools-stackabletech==0.0.13
68+
69+
# Test a product image can build, eg: ZooKeeper
70+
bake --product zookeeper=x.y.z # where x.y.z is a valid product version using the newly added Java version
71+
72+
kind load docker-image docker.stackable.tech/stackable/zookeeper:x.y.z-stackable0.0.0-dev
73+
74+
# Change directory into one of the operator repositories (eg: zookeeper-operator) and update the
75+
# product version in tests/test-definition.yaml
76+
./scripts/run-tests --test-suite smoke-latest # or similar
77+
```
78+
79+
</details>
80+
81+
_Please consider updating this template if these instructions are wrong, or
82+
could be made clearer._
83+
84+
[rpmfind.net]: https://rpmfind.net/linux/RPM/Development_Java.html

‎.github/ISSUE_TEMPLATE/update-base-stackable.md

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
---
2+
name: Update Stackable Base
3+
about: >-
4+
This template contains instructions specific to updating this product and/or
5+
container image(s).
6+
title: >-
7+
chore(stackable-base): Update container images ahead of Stackable Release XX.(X)X
8+
labels: []
9+
# Currently, projects cannot be assigned via front-matter.
10+
projects: ['stackabletech/10']
11+
assignees: ''
12+
---
13+
14+
Part of #xxx.
15+
16+
<!--
17+
This gives hints to the person doing the work.
18+
Add/Change/Remove anything that isn't applicable anymore
19+
-->
20+
- Add: `x.x.x`
21+
- Remove: `y.y.y`
22+
23+
> [!TIP]
24+
> Please add the `scheduled-for/20XX-XX` label, and add to the [Stackable Engineering][1] project.
25+
>
26+
> [1]: https://github.com/orgs/stackabletech/projects/10
27+
28+
```[tasklist]
29+
### Update tasks
30+
- [ ] Update UBI version hash in the Dockerfile (`FROM`)
31+
- [ ] Update `RUST_DEFAULT_TOOLCHAIN_VERSION`
32+
- [ ] Update `CARGO_CYCLONEDX_CRATE_VERSION`
33+
- [ ] Update `CARGO_AUDITABLE_CRATE_VERSION`
34+
- [ ] Update `PROTOC_VERSION`
35+
- [ ] Update `CONFIG_UTILS_VERSION`
36+
```
37+
38+
```[tasklist]
39+
### Related Pull Requests
40+
- [ ] _Link to the docker-images PR (product update)_
41+
```
42+
43+
This list should be completed by the assignee(s), once respective PRs have been merged. Once all items have been checked, the issue can be moved into _Development: Done_.
44+
45+
```[tasklist]
46+
### Acceptance
47+
- [ ] Can build the image locally
48+
- [ ] Can build the vector image
49+
```
50+
51+
<details>
52+
<summary>Testing instructions</summary>
53+
54+
```shell
55+
# See the latest version at https://pypi.org/project/image-tools-stackabletech/
56+
pip install image-tools-stackabletech==0.0.13
57+
58+
bake --product vector=x.y.z # where x.y.z is a valid version
59+
```
60+
61+
</details>
62+
63+
_Please consider updating this template if these instructions are wrong, or
64+
could be made clearer._

‎.github/ISSUE_TEMPLATE/update-base-ubi-rust-builders.md

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
---
2+
name: Update UBI Rust Builders
3+
about: >-
4+
This template contains instructions specific to updating this product and/or
5+
container image(s).
6+
title: >-
7+
chore(ubi-rust-builders): Update container images ahead of Stackable Release XX.(X)X
8+
labels: []
9+
# Currently, projects cannot be assigned via front-matter.
10+
projects: ['stackabletech/10']
11+
assignees: ''
12+
---
13+
14+
<!--
15+
I don't think we should tie rust bumps to releases - they should be done as
16+
developers need newer versions , which could be multiple times in a release.
17+
If there are no bumps in a release, we can still rely on SecObserve and Renovate
18+
to alert us to security vulnerabilities.
19+
-->
20+
Part of #xxx.
21+
22+
<!--
23+
This gives hints to the person doing the work.
24+
Add/Change/Remove anything that isn't applicable anymore
25+
-->
26+
- Add: `x.x.x`
27+
- Remove: `y.y.y`
28+
29+
> [!TIP]
30+
> Please add the `scheduled-for/20XX-XX` label, and add to the [Stackable Engineering][1] project.
31+
>
32+
> [1]: https://github.com/orgs/stackabletech/projects/10
33+
34+
```[tasklist]
35+
### Update tasks
36+
- [ ] Update UBI version hash in the Dockerfile (`FROM`)
37+
- [ ] Update `RUST_DEFAULT_TOOLCHAIN_VERSION`
38+
- [ ] Update `CARGO_CYCLONEDX_CRATE_VERSION`
39+
- [ ] Update `CARGO_AUDITABLE_CRATE_VERSION`
40+
- [ ] Update `PROTOC_VERSION`
41+
```
42+
43+
```[tasklist]
44+
### Related Pull Requests
45+
- [ ] _Link to the docker-images PR (product update)_
46+
- [ ] _Bump rust toolchain in operator-rs_
47+
- [ ] _Bump rust toolchain in operator-templating_
48+
```
49+
50+
This list should be completed by the assignee(s), once respective PRs have been merged. Once all items have been checked, the issue can be moved into _Development: Done_.
51+
52+
```[tasklist]
53+
### Acceptance
54+
- Done for [ubi8-rust-builder/Dockerfile](https://github.com/stackabletech/docker-images/blob/main/ubi8-rust-builder/Dockerfile)
55+
- Done for [ubi9-rust-builder/Dockerfile](https://github.com/stackabletech/docker-images/blob/main/ubi9-rust-builder/Dockerfile)
56+
- [ ] Can build the image locally
57+
- [ ] Can build an operator image
58+
```
59+
60+
<details>
61+
<summary>Testing instructions</summary>
62+
63+
```shell
64+
docker build -t oci.stackable.tech/sdp/ubi9-rust-builder . -f ubi9-rust-builder/Dockerfile
65+
66+
# Change directory into the an operator repository and ensure the image can build
67+
docker build . -f docker/Dockerfile
68+
```
69+
70+
</details>
71+
72+
_Please consider updating this template if these instructions are wrong, or
73+
could be made clearer._

0 commit comments

Comments
 (0)