Skip to content

configuration-reference.md

Latest commit

 

History

History
1043 lines (751 loc) · 26.2 KB

configuration-reference.md

File metadata and controls

1043 lines (751 loc) · 26.2 KB

NebariApp Configuration Reference

This document provides a comprehensive reference for all available configuration options in the NebariApp Custom Resource Definition (CRD).

Table of Contents

Overview

The NebariApp resource represents an application onboarding intent, specifying how an application should be:

  • Exposed (routing via HTTPRoute)
  • Secured (TLS/HTTPS via Gateway)
  • Protected (OIDC authentication)

Basic Structure

apiVersion: reconcilers.nebari.dev/v1
kind: NebariApp
metadata:
  name: <app-name>
  namespace: <namespace>
spec:
  hostname: <hostname>
  service:
    name: <service-name>
    port: <port>
  # ... additional configuration

Spec Fields

hostname

Type: string (required)

The fully qualified domain name (FQDN) where the application should be accessible. This will be used to generate the HTTPRoute resource.

Validation:

  • Minimum length: 1
  • Pattern: Must be a valid DNS hostname (lowercase letters, numbers, hyphens, and dots)
  • Examples: myapp.nebari.local, api.example.com

Example:

spec:
  hostname: myapp.nebari.local

service

Type: object (required)

Defines the backend Kubernetes Service that should receive traffic.

service.name

Type: string (required)

The name of the Kubernetes Service in the same namespace as the NebariApp.

Validation:

  • Minimum length: 1

service.port

Type: integer (required)

The port number on the Service to route traffic to.

Validation:

  • Minimum: 1
  • Maximum: 65535

service.namespace

Type: string (optional)

The namespace of the Service, if different from the NebariApp's namespace. This allows referencing services in other namespaces for centralized service architectures. The operator has cluster-scoped permissions to read Services across all namespaces.

Default: The NebariApp's namespace

Validation:

  • Minimum length: 1

Example:

spec:
  service:
    name: shared-api
    port: 8080
    namespace: shared-services

routing

Type: object (optional)

Configures routing behavior including path-based rules and TLS termination.

routing.routes

Type: array (optional)

Defines path-based routing rules for the application. If not specified, the Gateway API automatically adds a default path match of "/" (PathPrefix), which routes all traffic to the hostname to the service. When specified, only traffic matching these path prefixes will be routed.

Important: When no routes are specified, the operator creates an HTTPRoute with an empty matches array, and the Gateway API implementation (Envoy Gateway) automatically adds the default "/" path match.

routing.routes[].pathPrefix

Type: string (required)

The path prefix to match for routing. Traffic matching this prefix will be routed to the service.

Validation:

  • Must start with /
  • Examples: /app-1, /api/v1
routing.routes[].pathType

Type: string (optional)

Specifies how the path should be matched.

Valid values:

  • PathPrefix (default): Match requests with the specified path prefix
  • Exact: Match requests with the exact path

Default: PathPrefix

Example:

spec:
  routing:
    routes:
      - pathPrefix: /api/v1
        pathType: PathPrefix
      - pathPrefix: /admin
        pathType: Exact

routing.publicRoutes

Type: array of RouteMatch (optional)

Specifies paths that should bypass OIDC authentication. When auth is enabled and these are specified, these paths will be routed via a separate HTTPRoute that is not protected by the SecurityPolicy. All other paths remain protected.

This is useful for health checks, public APIs, or login endpoints that must be accessible without authentication.

Each entry uses the same RouteMatch format as routing.routes:

  • pathPrefix (required): The path to match. Must start with /.
  • pathType (optional): Exact (default) or PathPrefix. Defaults to Exact for public routes (safer for auth bypass), unlike routing.routes which defaults to PathPrefix.

Note: Public routes are only created when auth.enabled: true. If auth is disabled, all routes are already public.

Example:

spec:
  routing:
    routes:
      - pathPrefix: /
    publicRoutes:
      - pathPrefix: /api/v1/health            # Exact match (default)
      - pathPrefix: /api/v1/version            # Exact match (default)
      - pathPrefix: /.well-known
        pathType: PathPrefix                   # Explicit prefix match
  auth:
    enabled: true

routing.annotations

Type: map[string]string (optional)

Additional annotations to merge onto the generated HTTPRoute. Useful for tools like ArgoCD that track resources via annotations (e.g., argocd.argoproj.io/tracking-id). These annotations are merged with any operator-managed annotations; operator annotations always take precedence to avoid breaking internal behavior.

Example:

spec:
  routing:
    routes:
      - pathPrefix: /
    annotations:
      argocd.argoproj.io/tracking-id: my-app:gateway.networking.k8s.io/HTTPRoute:my-ns/my-app

routing.tls

Type: object (optional)

Controls TLS termination behavior for the HTTPRoute.

Important: The operator does not manage TLS certificates or Gateway TLS configuration. cert-manager and envoy-gateway handle certificate provisioning and TLS termination. This setting only controls whether the HTTPRoute should reference HTTPS listeners on the Gateway.

routing.tls.enabled

Type: boolean (optional)

Determines whether TLS termination should be used. This controls which Gateway listener the HTTPRoute references:

  • true (default): HTTPRoute uses sectionName: https to reference the HTTPS listener (port 443)
  • false: HTTPRoute uses sectionName: http to reference the HTTP listener (port 80)

Generated HTTPRoute behavior:

  • When enabled: true (or omitted): spec.parentRefs[0].sectionName: "https"
  • When enabled: false: spec.parentRefs[0].sectionName: "http" and annotation nebari.dev/tls-enabled: "false"

Note: The Gateway's TLS certificates are managed by cert-manager, not by this operator. This setting only affects which listener the HTTPRoute references.

Default: true

Example:

spec:
  routing:
    routes:
      - pathPrefix: /api
    tls:
      enabled: true

auth

Type: object (optional)

Configures authentication/authorization for the application. When enabled, the application will require OIDC authentication via a supporting OIDC Provider (currently Keycloak).

auth.enabled

Type: boolean (optional)

Determines whether authentication should be enforced for this application. When true, users must authenticate via OIDC before accessing the application.

Default: false

auth.provider

Type: string (optional)

Specifies the OIDC authentication provider to use.

Valid values:

  • keycloak: Uses Keycloak for authentication with automatic client provisioning
  • generic-oidc: Uses any OIDC-compliant provider (Google, Azure AD, Okta, Auth0, etc.)

Default: keycloak

auth.redirectURI

Type: string (optional)

Specifies the OAuth2 callback path for the application. The full redirect URL will be https://<hostname><redirectURI>.

For Envoy Gateway-level authentication (default), use /oauth2/callback. For application-level authentication handling, specify your app's callback path (e.g., /auth/callback).

Default: /oauth2/callback

auth.clientSecretRef

Type: string (optional)

References a Kubernetes Secret containing OIDC client credentials. The secret must be in the same namespace as the NebariApp and contain:

  • client-id: The OIDC client ID
  • client-secret: The OIDC client secret

If not specified and provisionClient is enabled, the operator will create a secret named <nebariapp-name>-oidc-client.

auth.scopes

Type: array of strings (optional)

Defines the OIDC scopes to request during authentication.

Common scopes: openid, profile, email, roles, groups

Default: ["openid", "profile", "email"]

auth.groups

Type: array of strings (optional)

Specifies the list of groups that should have access to this application. When specified, only users belonging to these groups will be authorized. Group matching is case-sensitive and depends on the OIDC provider's group claim.

Example:

groups:
  - admin
  - developers
  - data-scientists

auth.provisionClient

Type: boolean (optional)

Determines whether the operator should automatically provision an OIDC client in the provider. When true, the operator will create a client (e.g., in Keycloak) and store the credentials in a Secret.

Supported for: keycloak provider only

Default: true

auth.enforceAtGateway

Type: boolean (optional)

Determines whether the operator should create an Envoy Gateway SecurityPolicy to enforce authentication at the gateway level.

When true (default), the operator creates a SecurityPolicy that handles the OIDC flow at the gateway before requests reach the application.

When false, the operator provisions the OIDC client and stores credentials in a Secret, but does NOT create a SecurityPolicy. The application is expected to handle OAuth natively (e.g., Grafana's built-in generic_oauth provider). The app reads the client credentials from the operator-created Secret.

Default: true

Example (app-native OAuth):

spec:
  auth:
    enabled: true
    provider: keycloak
    provisionClient: true
    enforceAtGateway: false    # App handles OAuth itself

auth.issuerURL

Type: string (required when provider: generic-oidc)

Specifies the OIDC issuer URL for generic-oidc provider. This field is required when using generic-oidc provider and ignored for other providers.

Examples:

  • Google: https://accounts.google.com
  • Azure AD: https://login.microsoftonline.com/<tenant-id>/v2.0
  • Okta: https://<your-domain>.okta.com
  • Auth0: https://<your-domain>.auth0.com

auth.spaClient

Type: object (optional)

Configures a public OIDC client for Single-Page Applications that use browser-based authentication with PKCE (e.g., React apps using keycloak-js). This client is separate from the confidential client used by Envoy Gateway's OIDC filter.

The public client is configured with:

  • publicClient: true (no client secret, safe for browser)
  • Redirect URIs: https://<hostname>/* and https://<hostname>
  • PKCE enforcement with S256 code challenge method

Only supported for provider: keycloak.

auth.spaClient.enabled

Type: boolean (optional)

Whether to provision a public OIDC client for SPA use.

Default: false

auth.spaClient.clientId

Type: string (optional)

Override the generated client ID for the SPA client.

Default: <namespace>-<name>-spa

Example:

spec:
  auth:
    enabled: true
    provider: keycloak
    spaClient:
      enabled: true
      clientId: my-app-spa

auth.keycloakConfig

Type: object (optional)

Keycloak-specific configuration for fine-grained control over realm resources like groups and protocol mappers. Only used when provider: keycloak and provisionClient: true; silently ignored for other providers.

auth.keycloakConfig.groups

Type: array (optional)

Keycloak groups to ensure exist in the realm, with optional user membership assignments.

Each entry has:

  • name (string, required): Group name to create in Keycloak
  • members (array of strings, optional): Keycloak usernames to add to the group. Membership sync is additive-only - users in this list are added, but existing members not in the list are NOT removed. Users that don't exist in Keycloak are logged as warnings but don't cause errors.
auth.keycloakConfig.protocolMappers

Type: array (optional)

Client-level protocol mappers to configure on the OIDC client. These are applied directly to the client (not to shared client scopes), so each NebariApp gets isolated mapper configuration.

When specified, the operator's default mappers (e.g., group-membership) are not auto-created - this configuration takes full control.

Each entry has:

  • name (string, required): Name of the protocol mapper
  • protocolMapper (string, required): Mapper type identifier (e.g., oidc-group-membership-mapper, oidc-usermodel-attribute-mapper)
  • config (map[string]string, optional): Mapper configuration as key-value pairs

Example:

spec:
  auth:
    enabled: true
    provider: keycloak
    keycloakConfig:
      groups:
        - name: data-science-team
          members:
            - alice
            - bob
      protocolMappers:
        - name: group-membership
          protocolMapper: oidc-group-membership-mapper
          config:
            claim.name: groups
            full.path: "false"
            id.token.claim: "true"
            access.token.claim: "true"

Keycloak Authentication Example:

spec:
  hostname: my-app.example.com
  service:
    name: backend
    port: 8080
  auth:
    enabled: true
    provider: keycloak
    provisionClient: true
    scopes:
      - openid
      - profile
      - email
      - groups
    groups:
      - developers
      - data-scientists

Generic OIDC Authentication Example (Google):

spec:
  hostname: my-app.example.com
  service:
    name: backend
    port: 8080
  auth:
    enabled: true
    provider: generic-oidc
    provisionClient: false
    issuerURL: https://accounts.google.com
    clientSecretRef: my-app-google-oauth
    scopes:
      - openid
      - profile
      - email
    redirectURI: /oauth2/callback

gateway

Type: string (optional)

Specifies which shared Gateway to use for routing.

Valid values:

  • public (default): Use the public-facing gateway
  • internal: Use the internal gateway

Default: public

Example:

spec:
  gateway: public

landingPage

Type: object (optional)

Configures how this service appears on the Nebari landing page portal. When enabled, the service will be discoverable through the landing page.

landingPage.enabled

Type: boolean (required)

Whether this service appears on the landing page.

Default: false

landingPage.displayName

Type: string (optional)

Human-readable name shown on the landing page.

Validation:

  • Maximum length: 64 characters

landingPage.description

Type: string (optional)

Supplementary text shown on the service card.

Validation:

  • Maximum length: 256 characters

landingPage.icon

Type: string (optional)

Icon identifier or URL to a custom icon image. Supported built-in icons: jupyter, grafana, prometheus, keycloak, argocd, kubernetes.

landingPage.category

Type: string (optional)

Groups related services together on the landing page. Common categories: Development, Monitoring, Platform, Data Science.

landingPage.priority

Type: integer (optional)

Sort order within a category. Lower number = higher in the list.

Validation:

  • Minimum: 0
  • Maximum: 1000

Default: 100

landingPage.externalUrl

Type: string (optional)

Override the default URL derived from the hostname. Use when the service URL differs from https://<hostname>.

landingPage.healthCheck

Type: object (optional)

Configures health status monitoring for this service.

landingPage.healthCheck.enabled

Type: boolean (required)

Whether health checks are performed.

Default: false

landingPage.healthCheck.path

Type: string (optional)

HTTP path to check for health status. Common paths: /health, /healthz, /api/health.

Default: /health

landingPage.healthCheck.port

Type: integer (optional)

Port for health checks. If not specified, defaults to spec.service.port.

Validation:

  • Minimum: 1
  • Maximum: 65535
landingPage.healthCheck.intervalSeconds

Type: integer (optional)

How often to perform health checks (in seconds).

Validation:

  • Minimum: 10
  • Maximum: 300

Default: 30

landingPage.healthCheck.timeoutSeconds

Type: integer (optional)

Request timeout for health checks (in seconds).

Validation:

  • Minimum: 1
  • Maximum: 30

Default: 5

Example:

spec:
  landingPage:
    enabled: true
    displayName: My Data App
    description: Internal data science toolkit
    icon: jupyter
    category: Data Science
    priority: 10
    healthCheck:
      enabled: true
      path: /api/health
      intervalSeconds: 60

Status Fields

The status section is managed by the operator and reflects the observed state of the NebariApp.

conditions

Type: array

Represents the current state of the NebariApp resource.

Standard condition types:

  • RoutingReady: HTTPRoute has been created and is functioning
  • TLSReady: TLS termination is functioning (Gateway's TLS listeners are accessible)
  • AuthReady: Authentication policy is configured (if auth is enabled)
  • Ready: All components are ready (aggregate condition)

Common reasons:

  • Reconciling: Reconciliation is in progress
  • Available: The resource is functioning correctly
  • Failed: Reconciliation failed
  • NamespaceNotOptedIn: The namespace doesn't have the required label
  • ServiceNotFound: The referenced service doesn't exist
  • SecretNotFound: The referenced secret doesn't exist
  • GatewayNotFound: The target gateway doesn't exist
  • CertificateNotReady: TLS certificate is not yet ready (for TLSReady condition)
  • GatewayListenerConflict: Multiple NebariApps share hostname with per-app TLS (for TLSReady condition)

hostname

Type: string

The actual hostname where the application is accessible. This mirrors spec.hostname for easy reference.

gatewayRef

Type: object

Identifies the Gateway resource that routes traffic to this application.

Fields:

  • name: Name of the Gateway
  • namespace: Namespace of the Gateway

clientSecretRef

Type: object

Identifies the Secret containing OIDC client credentials (when auth is enabled).

Fields:

  • name: Name of the Secret
  • namespace: Namespace of the Secret (optional)

Complete Examples

Minimal Configuration

The simplest possible NebariApp that exposes a service with default settings:

apiVersion: reconcilers.nebari.dev/v1
kind: NebariApp
metadata:
  name: simple-app
  namespace: default
spec:
  hostname: simple-app.nebari.local
  service:
    name: my-service
    port: 8080

This will:

  • Route all traffic from simple-app.nebari.local to my-service:8080
  • Use the default wildcard TLS certificate
  • Use the public gateway
  • No authentication required

Path-Based Routing

Multiple path prefixes routing to the same service:

apiVersion: reconcilers.nebari.dev/v1
kind: NebariApp
metadata:
  name: api-app
  namespace: default
spec:
  hostname: api.nebari.local
  service:
    name: api-service
    port: 3000
  routing:
    routes:
      - pathPrefix: /api/v1
        pathType: PathPrefix
      - pathPrefix: /api/v2
        pathType: PathPrefix
      - pathPrefix: /health
        pathType: Exact
    tls:
      enabled: true

Disable TLS Termination

Use HTTP only (no TLS termination at the Gateway):

apiVersion: reconcilers.nebari.dev/v1
kind: NebariApp
metadata:
  name: http-only-app
  namespace: default
spec:
  hostname: http-app.nebari.local
  service:
    name: http-service
    port: 8080
  routing:
    tls:
      enabled: false

Protected Application with OIDC

Application requiring authentication via Keycloak:

apiVersion: reconcilers.nebari.dev/v1
kind: NebariApp
metadata:
  name: protected-app
  namespace: default
spec:
  hostname: protected.nebari.local
  service:
    name: protected-service
    port: 8080
  auth:
    enabled: true
    provider: keycloak
    provisionClient: true
    scopes:
      - openid
      - profile
      - email
      - roles

Protected Application with Public Routes

Application requiring authentication, but with health and version endpoints publicly accessible:

apiVersion: reconcilers.nebari.dev/v1
kind: NebariApp
metadata:
  name: api-app
  namespace: default
spec:
  hostname: api.nebari.local
  service:
    name: api-service
    port: 3000
  routing:
    routes:
      - pathPrefix: /
    publicRoutes:
      - pathPrefix: /api/v1/health       # Exact match (default)
      - pathPrefix: /api/v1/version      # Exact match (default)
    tls:
      enabled: true
  auth:
    enabled: true
    provider: keycloak
    provisionClient: true

This will:

  • Route all traffic from api.nebari.local to api-service:3000
  • Require OIDC authentication for all paths
  • Except /api/v1/health and /api/v1/version, which are publicly accessible
  • The operator creates two HTTPRoutes: a protected one (targeted by SecurityPolicy) and a public one (no SecurityPolicy)

Full Configuration Example

A comprehensive example using all available options:

apiVersion: reconcilers.nebari.dev/v1
kind: NebariApp
metadata:
  name: full-featured-app
  namespace: production
spec:
  hostname: app.example.com
  service:
    name: backend-service
    port: 8080
    namespace: production
  routing:
    routes:
      - pathPrefix: /
        pathType: PathPrefix
      - pathPrefix: /api/v1
        pathType: PathPrefix
    publicRoutes:
      - pathPrefix: /api/v1/health
        pathType: Exact
      - pathPrefix: /api/v1/ready
        pathType: Exact
      - pathPrefix: /api/v1/webhooks
        pathType: PathPrefix
    tls:
      enabled: true
    annotations:
      argocd.argoproj.io/tracking-id: full-featured-app:gateway.networking.k8s.io/HTTPRoute:production/full-featured-app
  auth:
    enabled: true
    provider: keycloak
    provisionClient: true
    enforceAtGateway: true
    redirectURI: /oauth2/callback
    scopes:
      - openid
      - profile
      - email
      - groups
    groups:
      - admin
      - data-science-team
    spaClient:
      enabled: true
      clientId: my-app-spa
    keycloakConfig:
      groups:
        - name: data-science-team
          members:
            - alice
            - bob
      protocolMappers:
        - name: group-membership
          protocolMapper: oidc-group-membership-mapper
          config:
            claim.name: groups
            full.path: "false"
            id.token.claim: "true"
            access.token.claim: "true"
  gateway: public
  landingPage:
    enabled: true
    displayName: My Application
    description: Full-featured production application
    icon: jupyter
    category: Data Science
    priority: 10
    healthCheck:
      enabled: true
      path: /api/v1/health
      port: 8080
      intervalSeconds: 30
      timeoutSeconds: 5

Internal Gateway Example

Application accessible only through the internal gateway:

apiVersion: reconcilers.nebari.dev/v1
kind: NebariApp
metadata:
  name: internal-app
  namespace: default
spec:
  hostname: internal.nebari.local
  service:
    name: internal-service
    port: 8080
  gateway: internal
  routing:
    tls:
      enabled: true

Additional Notes

Namespace Requirements

The namespace where you deploy a NebariApp may need to be opted-in with specific labels. Check with your cluster administrator for namespace requirements.

Service Requirements

  • The referenced Kubernetes Service must exist in the same namespace as the NebariApp, unless service.namespace is specified to reference a service in a different namespace
  • The Service must be listening on the specified port
  • The Service should be ready to handle traffic before creating the NebariApp

Gateway Requirements

  • The specified gateway (public or internal) must exist in the cluster
  • The gateway must be properly configured with listeners for HTTP/HTTPS traffic

Certificate Requirements

When using TLS (routing.tls.enabled: true or omitted):

  • The Gateway must have a valid TLS certificate configured
  • For development: Use self-signed certificates via cert-manager
  • For production: Use Let's Encrypt or your organization's CA
  • The operator does not manage certificates - cert-manager handles this

Authentication Requirements

When auth.enabled: true:

  • An OIDC provider must be available (Keycloak or generic-oidc)
  • For Keycloak with provisionClient: true: Operator needs admin credentials
  • For generic-oidc or provisionClient: false: Client secret must exist

Multiple Apps Sharing a Hostname

Important: When deploying multiple NebariApps that share the same hostname (e.g., frontend and API at different paths), you must use the shared wildcard TLS listener to avoid Gateway listener conflicts.

Problem: With per-app TLS enabled (the default), each NebariApp tries to create its own HTTPS listener for the hostname. Gateway API requires that port + protocol + hostname combinations be unique, causing a GatewayListenerConflict error.

Solution: Set routing.tls.enabled: false on all apps sharing the hostname:

# Frontend app
apiVersion: reconcilers.nebari.dev/v1
kind: NebariApp
metadata:
  name: myapp-frontend
spec:
  hostname: myapp.example.com  # Shared hostname
  service:
    name: frontend-service
    port: 80
  routing:
    routes:
      - pathPrefix: /
    tls:
      enabled: false  # Use shared wildcard listener
---
# API app
apiVersion: reconcilers.nebari.dev/v1
kind: NebariApp
metadata:
  name: myapp-api
spec:
  hostname: myapp.example.com  # Same hostname
  service:
    name: api-service
    port: 8080
  routing:
    routes:
      - pathPrefix: /api/
    tls:
      enabled: false  # Use shared wildcard listener

Note: Setting tls.enabled: false does NOT disable HTTPS. It tells the HTTPRoute to use the Gateway's shared HTTPS listener (with wildcard certificate) instead of creating a per-app listener. Traffic is still encrypted via TLS.

For more details, see the troubleshooting guide.

  • Client secret must contain client-id and client-secret keys