Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dapr deployed with ArgoCD mtls renew-certificate -k --restart #1484

Open
jwhitcraft opened this issue Feb 7, 2025 · 2 comments
Open

Dapr deployed with ArgoCD mtls renew-certificate -k --restart #1484

jwhitcraft opened this issue Feb 7, 2025 · 2 comments
Labels
kind/bug Something isn't working

Comments

@jwhitcraft
Copy link

Expected Behavior

We have moved to using GitOps to manage our deployments of system services, which means that ArgoCD is not creating a helm release. as it just does a helm template | kubectl apply -f .

Thus we should be able to use the dapr cli to run mtl renew-certificate and still have it restart everything we need.

Actual Behavior

It fails:

⌛  Starting certificate rotation
ℹ️  generating fresh certificates
ℹ️  Updating certifcates in your Kubernetes cluster
ℹ️  Dapr control plane version 1.14.4 detected in namespace dapr-system
❌  certificate rotation failed: could not find release name "dapr" in your helm releases

Steps to Reproduce the Problem

Create the yaml with helm template dapr dapr/dapr --set=global.ha.enabled=true | kubectl apply -f - and then try try and run the cli command to renew the cert.

Release Note

RELEASE NOTE:

@jwhitcraft jwhitcraft added the kind/bug Something isn't working label Feb 7, 2025
@antontroshin
Copy link
Contributor

Hello @jwhitcraft,
Installing Dapr on k8s using helm template is not ideal and not something that is officially supported in the docs, I would suggest using helm upgrade --install as described on this page https://docs.dapr.io/operations/hosting/kubernetes/kubernetes-deploy/ under "Helm" tab.

According to ArgoCD docs (here) it is impossible to use it with helm install.

The issue renewing cert after using helm template + kubectl apply is that they do not create a Helm chart "release" as opposed to helm upgrade --install would.
Helm releases can be listed with the helm list -A command or kubectl get secrets -A --field-selector type=helm.sh/release.v1
For dapr mtls renew-certificate -k to work, the Helm chart "release" must be in the k8s cluster, meaning the chart must be installed using helm upgrade --install command.

Another issue that you will experience with dapr via ArgoCD, is sync, since the helm template will not produce certificates in the rendered manifests, the Secret manifest named dapr-trust-bundle will be empty at the beginning, the dapr-sentry would populate it with certificates after install, but for ArgoCD the empty manifest will be as "source" and if sync is enabled, the dapr-trust-bundle will be overwritten with empty secret.

At this time, my suggestion would be:

  • Do not sync Dapr with ArgoCD.
  • Manage Dapr lifecycle versions separately with helm upgrade --install

This topic comes up once in a while, definitely something to take a look into in the future.
Ideas, suggestions, or any feedback is always welcome.

@jwhitcraft
Copy link
Author

@antontroshin that works well when you only have a few clusters. We currently are over 50+ clusters with dapr installed and maintaining it outside of Agro can take someone almost a full day to complete maintenance work on.

Ideally the way to solve this would be to support certificates created by cert-manger and just have the sentry app watch for secret changes/updates from cert-manager (see dapr/dapr#8407).

While i get this problem isn't a CLI problem for dapr, but just were we noticed it at.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants