Skip to content

Commit

Permalink
Minor updates
Browse files Browse the repository at this point in the history
  • Loading branch information
pfisterer committed Mar 21, 2022
1 parent 20fc7bc commit d7a4fe3
Show file tree
Hide file tree
Showing 5 changed files with 54 additions and 24 deletions.
30 changes: 29 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,20 +51,48 @@ The following table lists the configurable parameters of the Hadoop chart and th
| `persistence.dataNode.size` | Size of the volume | `200Gi` |


---

## Customized Hadoop Base Docker Image

This image is modified from [comcast/kube-yarn](https://github.com/Comcast/kube-yarn/tree/add-hadoop-image-versions) and [mgit-at/helm-hadoop-3](https://github.com/mgit-at/helm-hadoop-3). Currently, native libraries are not been included.

### Build and Push the Docker Image

```bash
# Set version
HADOOP_VERSION=3.3.2

# Build
docker buildx build --push --platform "linux/arm64,linux/amd64" -t farberg/apache-hadoop:latest -t farberg/apache-hadoop:$HADOOP_VERSION .
```

### Testing with minikube

If you are running locally with minikube and want to try your images without pushing them to a registry, build the images on the minikube VM first:

```bash
eval $(minikube docker-env)
# use the build command from above
```

---

## Development

Help is always appreciated. Please create pull requests.

### Open Issues

- Include native libraries
- List of ports needs to be updated (cf. https://www.oreilly.com/library/view/big-data-analytics/9781788628846/5c5821cc-4a3d-498a-a3eb-23256cd79c8b.xhtml)

### Upload a new version of the chart

```bash
helm lint
helm package .
mv apache-hadoop-helm-*.tgz docs/
mv hadoop*.tgz docs/
helm repo index docs/ --url https://pfisterer.github.io/apache-hadoop-helm/
git add docs/
git commit -a -m "Updated helm repository"
Expand Down
Binary file added docs/hadoop-1.2.0.tgz
Binary file not shown.
24 changes: 24 additions & 0 deletions docs/index.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
apiVersion: v1
entries:
hadoop:
- apiVersion: v1
appVersion: 3.3.2
created: "2022-03-21T16:15:24.698221+01:00"
description: The Apache Hadoop software library is a framework that allows for
the distributed processing of large data sets across clusters of computers using
simple programming models.
digest: 5e655a09f2e502dfaf6350333698e6969c796ff4fefa15cc6e1b0b4f82f9e05e
home: https://hadoop.apache.org/
icon: http://hadoop.apache.org/images/hadoop-logo.jpg
maintainers:
- email: [email protected]
name: pfisterer
url: http://github.com/pfisterer
name: hadoop
sources:
- https://github.com/pfisterer/apache-hadoop-helm
- https://github.com/apache/hadoop
urls:
- https://pfisterer.github.io/apache-hadoop-helm/hadoop-1.2.0.tgz
version: 1.2.0
generated: "2022-03-21T16:15:24.697105+01:00"
2 changes: 1 addition & 1 deletion image/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM openjdk:8-jre
FROM openjdk:8-jdk

ENV HADOOP_VERSION 3.3.2
ENV HADOOP_HOME=/opt/hadoop
Expand Down
22 changes: 0 additions & 22 deletions image/README.md

This file was deleted.

0 comments on commit d7a4fe3

Please sign in to comment.