Skip to content

Latest commit

 

History

History
1340 lines (1118 loc) · 30 KB

hyscale-spec-reference.md

File metadata and controls

1340 lines (1118 loc) · 30 KB

Hyscale Spec File Reference

Version 0.6.1.1

Table of contents

Overview

  • The Hyscale Service Spec file is meant for user to declaratively describe the service/application and its related configuration.

  • Hyscale Spec File is a valid YAML file.

  • Abstracts Kubernetes concepts/knowledge with a simplified DSL.

  • Developer friendly and can be added to version control.

  • Spec file should end with “.hspec” so that service specs can be easily identified.

  • Filename should be named exactly as per service name <service-name>.hspec

  • Multiple spec files can be present in a directory.

  • Support environment profiles (eg: dev, stage and prod etc.) Env props and customizations are in profile file.

  • Profile files may mirror some of the fields given in the service spec. Only few fields allowed as indicated in this reference. Any fields specified in profile may get merged or override the ones given in spec, this behaviour is specified below for each field. Future item

  • Supports camel case for keys. Along with camel case, Uppercase for keys can be accepted in the future. In case of Uppercase, multi-word keys are separated by underscore (_) delimiter.

  • Works for any service be it bespoke, off-the-shelf, stateless, stateful, etc.

  • Following Infra targets are assumed :

    • kubernetes cluster from ~/.kube/config
    • registry authentication credentials from ~/.docker/config.json

Reference Spec File with all options

name: <service-name>                # should be same as in filename
image:
    name:  <image-name>
    registry: <target-registry-url>
    tag: <tag>
    dockerfile:
        path: <build-context-path>               # default is ./
        dockerfilePath: <dockerfile-path>        # default is <path>/Dockerfile
        args:
            <key1>:<value1> 
    buildSpec:
       stackImage: [<pull-registry-url>/]<stack-image-name>[:<stack-image-tag>]
       artifacts:
           - name: <artifact-name>
             source: <relative source path>      # eg: /login/target/{{ buildNumber }}/login.war
             destination: <destination-in-container>

       configCommandsScript: <config-commands-script>    # output of this script is baked inside image
       configCommands: |-
         <config-command1>
         [<config-command2>]
         .
         .
         [<config-commandN>]

       runCommandsScript: <run-commands-script>          # runs on container start
                                          
                                                 # CMD in dockerfile
       runCommands: |-
         # Executes while container start
        [<run-command1>]
        [<run-command2>]
         .
         .
        [<run-commandN>]

startCommand: <start-command with args>          # command+args in k8s yaml, overrides ENTRYPOINT+CMD
replicas: <replica-count>                        # one of integer or struct
                                                 # represents the number of instances of this service
                                                 # you can directly declare a integer like `replicas: 2` (default is 1)
                                                 # or use struct for autoscaling setting 

memory: [<min-memory>-]<max-memory>              # Supported units are
                                                 # 1. Ki|Mi|Gi|Ti|Pi|Ei as power of two equivalents
                                                 # 2. n|u|m|k|M|G|T|P|E as plain integers
cpu: [<min-cpu>-]<max-cpu>			 # Supported units are `m` or `none`

external: <true/false>				 # default is false
ports:
    - port: <port-number1>[/<port-type>]         # default type is tcp
      healthCheck:
          httpPath: <http-path>                  # default is false

   [- port: <port-number2>/<port-type>]

volumes:                        
    - name: <volume-name1>
      path: <volume-mount-path>
     [size: <size>]             		 # default size is 1G
                                                 # Supported units are 
                                                 # 1. Ki|Mi|Gi|Ti|Pi|Ei as power of two equivalents
                                                 # 2. n|u|m|k|M|G|T|P|E as plain integers
     storageClass: <storageClass>

propsVolumePath: <volume-path-of-configmap> 
props: 
    <key1>: [<file/endpoint/string>(]<value1>[)]     # default type is string
   [<key2>: <value2>]
   .
   [<keyN>: <valueN>]

secrets:		 	# Secrets accept both list of keys & key value pairs 
    - <secret1>			# Incase of list, secret should be created prior as <appname>-<servicename>
    - <secret2>			# - <key1> : <value1>
                                # - <key2> : <value2>
    - <secretN>			# - <keyN> : <valueN>

secretsVolumePath: <volume-path-of-secrets>
agents:
    - name: <sidecarName>
      image: [<pull-registry-url>/]<sidecar-image-name>[:<sidecar-image-tag>]
      props: 
          <key1>: [<file/endpoint/string>(]<value1>[)]
         [<key2>: [<file/endpoint/string>(]<value2>[)]
      secrets:                        # Secrets accept both list of keys & key value pairs
        - <secret1>                 # Incase of list, secret should be created prior as <appname>-<servicename>
        - <secret2>                 # - <key1> : <value1>
                                    # - <key2> : <value2>
        - <secretN>                 # - <keyN> : <valueN>
      secretsVolumePath: <volume-path-of-secrets>
      volumes: 
          - mountPath: <volume-mount-path1>
            attach: <volume-name1>             # use same name as service vol for sharing
           [readOnly: <true/false>]        # readOnly is valid for shared volume
         [- mountPath: <volume-mount-path2>
            attach: <volume-name2>]
           [readOnly: <true/false>]
      propsVolumePath: <volume-path-of-configmap>

Here is the Service Spec Schema

Example

name: hrms-frontend
image:
   name: gcr.io/hyscale-hrms/hrms-frontend:1.0
   dockerfile: {} # all default values
replicas: 1
volumes:
  - name: logs
    path: /usr/local/tomcat/logs
    size: 1Gi
  - name: data
    path: /data
    size: 2Gi
 
secrets:
  - keystore_password
 
props:
    max_conn: file(./config/config.xml)
    country: india
    region: hyderabad
    mysql_ost: endpoint(mysqldb)
    max_threads: 15
    server_xml: file(./config/tomcat/server.xml)
external: true
ports:
  - port: 8080/tcp
    healthCheck:
  	httpPath: /hrms

agents:
  - name: logging-agent
    image: gcr.io/test/logstash:2.2
    props:
        FLUEND_CONF: file(./config/log/fluentd.conf)
    volumes:
      - mountPath: /usr/local/tomcat/logs
        attach: logs

Field Reference

Note: A Boolean represents a true/false value. Booleans are formatted as English words (“true”/“false”, “yes”/“no” or “on”/“off”) for readability and may be abbreviated as a single character “y”/“n” or “Y”/“N”.

y|Y|yes|Yes|YES|n|N|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF

all the above expressions are considered to be boolean values not a string . If you want to refer the value as string use quotes like "yes".

Field Type default Explanation
image struct Can’t be overridden

Describes an image to be deployed.

image section contains following:

  1. name and tag
  2. Registry url where the image resides
  3. dockerfile or buildSpec fields
replicas int 1 Optional

Can be overridden

Number of instances of the service i.e represents the number of instances of this service You can directly declare a integer like `replicas: 2` (default is 1) or by autoscaling setting (struct)

memory string Optional

Can be overridden

Specify the range

[<minMemory>-]<maxMemory>

Eg: 512m-1024m or 512m

cpu string Optional

Can be overridden

Specify the cpu range

[<minCpu>-]<maxCpu>

Eg: 60-80 or 50

startCommand string Optional

Can't be overridden

  • startCommand is a command which gets executed at the time of container start
  • it includes both ENTRYPOINT and CMD separated by space delimiter
  • User may refer to the runCommandsScript above in CMD
external string false Optional

  • Exposes the service externally.
  • External IP would be assigned for the service.
ports list Can be overridden

List of ports to be declared along healthcheck and ingressrules if any.

props list Optional

Can be overridden 


<keyName>:[<file/endpoint/string>(]<value>[)]
  • List of key value pairs
  • Value is typed and can be of type: string, file, endpoint.
    • if the value is file typed. utf-8 content of the file will be sent.
    • if the value is endpoint typed. service discovery name of the given service is passed as the value.
  • DEFAULT type is string

Eg: 

 props:
   MAX_NO_OF_CONNECTIONS: STRING(10)
   MYSQL_HOST: ENDPOINT(mysql)
   KEY_FILE: FILE(/tmp/file.txt)
secrets list Optional Can be overridden

List of secret key Names

Note: The secret should be pre-created by the k8s-admin in your namespace as appName-serviceName. The appName & serviceName should be normalized to lowercase & replace dot(.) as hyphen(-) ,space( ) as hyphen(-). Eg: appName=Sample, namespace=Sample dev results in secret of sample-sample-dev.

Eg: 


secrets:
- "MYSQL_ROOT_PASSWORD"
volumes list Optional Can be overridden

List of volumes to be specified in a pod.

agents list Optional Can be overridden

List of sidecars to be attached to the pod.

image

Following are the fields of image section.

Field Type default Explanation
name string Name of the image to be built

Note: retagging should be done for the already provided image

tag string latest Tag would be overridden if tagpolicy is mentioned
registry string docker.io/library registry url along with the namespace
buildSpec struct Optional generate Docker and build with local docker daemon
dockerfile struct Optional build with Dockerfile using local docker daemon
tagPolicy

(future, currently unspecified)

struct sha256 Optional

Supports following tag policies

  • Gitcommit
  • sha256
  • dateTime

Builders:

Hyscale supports following to build your image:

  • hyscaleBuildSpec locally with Docker
  • Dockerfile with local docker

Note: Following additional building mechanisms can be supported in future:
Dockerfile with kaniko in-cluster
jib maven and gradle projects locally
bazel locally
Cloud-native build packs

buildSpec

HyscaleBuildSpec locally with Docker

Option Type default Explanation
stackImage string Stack indicates the base image on top of which artifacts and configuration is layered.
artifacts list Optional Represents the list of artifacts to be present inside the container at defined destinations.
configCommandsScript string Optional path-to-script relative or absolute path to a config-Commands script
configCommands list Optional Array of commands which is invoked during image build. The commands are also part of the image.
runCommandsScript string Optional path-to-script relative or absolute path to a run-Commands script
runCommands list Optional runCommand is a command which gets executed at the time of container start and is baked into the image

stackImage

stack: [<registryUrl>/]<image>[:<tag>]

  • Stack indicates the base image on top of which artifacts and configuration is layered.
  • registryUrl is optional.
  • tag or digest is optional. latest is assumed.

Eg:

   stackImage: tomcat:8.5

artifacts

can be overridden

    artifacts:
      - name: <artifactName1>
      	destination: <destination1InContainer>
      	provider: [ssh/http/local]                  # default local (ssh, http will be implemented in future versions)
        source: <url>

Fields:

Option Type default Description
name string Name of the artifact
destination string Destination path to copy the artifact inside the container
provider string Optional type of provider. Could be one of:
  • local
    • Artifact available locally
    • Relative path to the artifact is expected in the source field
  • ssh
    • Artifact available remotely
    • Accessible through ssh
    • Auth credentials can be mentioned in infrastructure spec
    • source is ssh url
  • http
    • Artifact available remotely
    • Accessible through http
    • Auth credentials can be mentioned in infrastructure spec
    • Source is http url

source string Url to the artifact
  • List of artifacts to be copied inside the container image
  • Copies the artifact mentioned in sourcePath of local folder to destinationPath inside image
  • Artifact can be from one of the following sources:
  • local
  • Remote (like jfrog, jenkins etc..)

configCommandsScript

Optional string type

  • Path to a Script containing config commands.
  • It is mutually exclusive with configCommands field.
  • If configCommands is given and not empty configCommandsScript has no effect.

runCommandsScript

Optional string type

  • Path to a Script containing runConfig commands.
  • It is mutually exclusive with runCommands field.
  • If runCommands is given and not empty runCommandsScript has no effect.

dockerfile

Use local docker to build docker image with the given Dockerfile

   dockerfile:
  	target: <target>
        path: <buildContextPath>            # Optional buildcontext path
        dockerfilePath: <DockerfilePath>    # Optional path to Dockerfile
  	useBuildKit: <true/false>           # use buildKit for building (will be implemented in future versions)
  	buildArgs:
  	    - <buildarg1>
           [- <buildarg2>]
            .
            .
           [- <buildargN>]
Option Type default Explanation
path string ./ Optional buildContext for docker build
dockerfilePath string ./Dockerfile Optional dockerfile path with in the build context
target string Optional target stage to be built
useBuildKit boolean Optional Use buildkit (will be implemented in future versions)
buildArgs list Optional List of build arguments to be passed

Eg:

   dockerfile:
  	path: ./
        dockerfilePath: Dockerfile.build
  	target: finalstage
  	useBuildKit: false
  	buildArgs:
           - foo=value1
  	   - bar=value2

ports

List of port objects.

ports:
  - port: <portNumber1>[/<portType>]
    healthCheck:
       httpPath: <httpPath> # optional if not http type

  [- port: <portNumber2>/<portType>]

Port Object contains:

  • port to be declared in a pod
  • healthcheck if available for the port to be specified along with the port definition

Note: Currently Health Check would be present for only one port if any. If there are multiple healthChecks declared first healthCheck will be picked.

Following are the Fields of Port object:

Option Type default Explanation
port string <portNumber>[/<portType>]
  • portNumber Port Number to be declared in pod 0-65535
  • portType tcp/udp
healthCheck struct http incase healthCheckPath is given, tcp if nothing given Optional 
healthCheck:
    type: <tcp/http/udp> # optional type of health check (will be implemented in future versions)
    httpPath: <httpPath> # optional if not type: http

type

type string Optional

Type of HealthCheck for the specified port

  • tcp
  • http

    http incase httpPath is given, tcp if nothing given

    Note: exec/command healthchecks should be mentioned in a separate section

httpPath
type string Optional
http Path for http health check

Eg: 

healthcheck:
    httpPath: /hmrs

Eg:

    ports:
      - port: 8080/TCP
  	healthCheck:
  	    httpPath: "/hrms" # optional

      - port: 8008/TCP

volumes

List of volume Objects.

  volumes:# optional can be inferred from docker inspection and default 2 GB + default sc
    - name: <volumeName1>
      path: <volumeMountPath>
     [size: <size>]

volume Object contains:

  • name - volume name
  • path - mount Path inside container
  • size - volume size to provision using environment defined storage class.

Note: volumes referring other volumes is _future, currently unspecified_

Following are the Fields in volume object

Option Type default Explanation
name string Name of volume

should be up to maximum length of 253 characters and consist of lower case alphanumeric characters, -, and ., but certain resources have more specific restrictions.

Eg:

name: logsDirectory

path string Mount Path of the volume inside container

Eg: path: /usr/local/tomcat/logs

size string 1Gi Optional

Size of volume to be provisioned

Makes use of environment tied kubernetes storage class to provision volume.

Eg: size: 1Gi

Eg:

volumes:
  - name: tomcat-logs
    path: /usr/local/content/tomcat/current/logs
    size: 1Gi

  - name: data
    path: /usr/local/content/tomcat/current/webapps

replicas

Autoscaling Setting.

Note : Autoscaling will be effective only if cpu is specified

replicas:
  min: <minReplicas>                   # minimum number of instances to start (default is 1)
  max: <maxReplicas>                   # maximum number of instances
  cpuThreshold: <cpuThreshold>         # Average pod cpu utilization threshold as percent that will cause a scale event

agents

List of agent (aka sidecar) objects.

agents:
  - name: <sidecarName>
    image: <sidecarWithVersion>
    props:
    - "<sidecarKey1Name>=<file/endpoint/string>(<sidecarValue1>)"
    [- "<sidecarKey2Name>=<file/endpoint/string>(<sidecarValue2>)"]
    volumes: 
    - mountPath: <sidecarVolumeMountPath1>
      attach: <sidecarVolumeName1>
   [- mountPath: <sidecarVolumeMountPath2>
      attach: <sidecarVolumeName2>]
    secrets:                        # Secrets accept both list of keys & key value pairs 
       - <secret1>                 # Incase of list, secret should be created prior as <appname>-<servicename>
       - <secret2>                 # - <key1> : <value1>
                                   # - <key2> : <value2>
       - <secretN>                 # - <keyN> : <valueN>

    secretsVolumePath: <volume-path-of-secrets>
    propsVolumePath: <volume-path-of-configmap>

agent Object contains:

  • name - name of sidecar attachment
  • image - sidecar image full name
  • props - list of env including secrets
  • volumes - list of volumes
  • secrets - list of secrets

Following are the Fields in agent object

Option Type default Explanation
name string Name of sidecar Attachment

should be up to maximum length of 253 characters and consist of lower case alphanumeric characters, -, and ., but certain resources have more specific restrictions.

Eg:

name: fluentd

image string sidecar with version

Eg:

sidecar: fluentd:5.6

props list 2 Optional

=[(][)]

  • List of key value pairs
  • Value is typed and can be of type: string, file, endpoint
  • DEFAULT type is string

    Eg:

    props:

    - "FLUENTD_ARGS=--no-supervisor -vv"

    - "system.input.conf=file(/system.input.conf)"

    - "output.conf=file(/tmp/output.conf)"

volumes list List of volumes to be declared for a sidecar
secrets list Optional Can be overridden

List of secret key Names

Note: The secret should be pre-created by the k8s-admin in your namespace as appName-serviceName. The appName & serviceName should be normalized to lowercase & replace dot(.) as hyphen(-) ,space( ) as hyphen(-). Eg: appName=Sample, namespace=Sample dev results in secret of sample-sample-dev.

Eg: 


secrets:
- "MYSQL_ROOT_PASSWORD"

Spec Template File

will be implemented in future versions

For Off-the-Shelf (OTS) services as well as for commonly used configurations of known services, spec template files could be made available as a starting point. Once a spec template is downloaded for use, it can be extended to create a service spec (hspec) in order to override commonly specified configurations.

For example, a mysql spec template may include things like 3306 for ports, /var/lib/mysql/ as a volume path, etc. Anyone extending this template to create their hspec may wish to override things like the password secret, etc.

The following rules apply to a spec template:

  1. Spec templates are valid YAML files.
  2. Spec template end with the extension htpl.yaml
  3. A template file must include a name and version attribute. Name is same as in the filename of the template. Name & version would be used in the hspec which extends.
  4. The filename of the spec template should be same as the service name + version. Eg. <service-name>-<version>.htpl.yaml
  5. A template file may include any of the fields that a regular hspec can have, except as per the following.
  6. A template file cannot include:
    1. buildSpec & dockerSpec
    2. external
NOTE: Consideration for future versions of spec templates:
	a. Spec Templates as bundles (htpl.tar)
	b. This enables things like artifacts, dockerfiles, config files, etc. to be included in the htpl bundle.
	c. This will enable support for buildSpec and dockerSpec.
	d. The bundle could be name.htpl.tar which expands and should mandatorily include name.htpl.yaml
	e. Allow remote spec template repositories & corresponding spec

Extending a Spec Template

(htpl) into a Service Spec (hspec)

In a service spec, use the extends field as follows:

extends: <service-template-name>:<template-version> In the hspec, any props, secrets, volumes, ports specified are merged or overridden if having same keyname.

image is overridden if specified in hspec, and the image given in the htpl is used as stack image within the buildSpec.

Future versions may support:
	Specifying a remote URL in the extends field
	`$remove: <key-name>` in the hspec in order to skip that key & corresponding sub-tree from the htpl
	Consider extending multiple htpl files in a single hspec file

Profile Files

In order to deploy a service into different environments (such as QA, Stage, UAT, etc.), it maybe necessary to override certain fields to customize as per that environment. This is supported by the use of profile files.

A profile file may override a spec file by specifying the following:

environment: <profile-name>
overrides: <service-name>

Profile files should be <profile-name>-<service-name>.hprof

For example, dev profile for service spec web.hspec would be dev-web.hprof

The following fields of service spec can be overridden or additionally specified (merged) in a profile file:

**props, secrets, replicas, resource-limits, size of volumes.

(overrides if same keyname)

Reference Profile File with all options

environement:  <profile-name>
overrides: <service-name>

memory: [<min-memory>-]<max-memory>     # overrides the range of memory defined in the service spec
cpu: [<min-cpu>-]<max-cpu>              # overrides the range of cpu defined in the service spec

replicas: <replica-count>               # overrides the no of replicas defined in the service spec

volumes:                        
    - name: <volume-name>
      [size: <size>]             		 # overrides the size of volumes defined in the service spec
      [storageClass: <storageClass>]    

props:                                   # overrides the prop values which are defined in the service spec
    <key1>: [<file/endpoint/string>(]<value1>[)]     
   [<key2>: <value2>]
   .
   [<keyN>: <valueN>]

secrets:		 	        # overrides the secret values which are defined in the service spec 
    - <key1>: <value1>		
    - <key2>: <value2>			
                        
    - <keyN>: <valueN>

Example of a Profile file

dev-hrms-frontend.hprof

environment: dev
overrides: hrms-frontend
  
cpu: 512Mi  
replicas: 2
props:
   max_threads: 20
   server_xml: file(./config/tomcat/server.xml)

volumes:
   - name: logs
     size: 2Gi

stage-hrms-frontend.hprof

environment: stage
overrides: hrms-frontend

cpu: 768Mi
replicas: 3

props:
   max_threads: 10
   server_xml: file(./config/tomcat/server.xml)

volumes:
  - name: logs
    size: 4Gi

Future versions may support:
	Disabling of healthChecks using an additional field in the relevant section such as:
$disable: true