Spring Boot Latency

Introduction¶

It can target random pods with a Spring Boot application and allows configuring the assaults to inject network latency to every nth request. This can be tuned via CM_LEVEL ENV.

Scenario: Inject network latency to Spring Boot Application

Spring Boot Latency

Uses¶

View the uses of the experiment

coming soon

Prerequisites¶

Verify the prerequisites

Ensure that Kubernetes Version > 1.16
Ensure that the Litmus Chaos Operator is running by executing kubectl get pods in operator namespace (typically, litmus).If not, install from here
Ensure that the spring-boot-latency experiment resource is available in the cluster by executing kubectl get chaosexperiments in the desired namespace. If not, install from here

Chaos Monkey Spring Boot dependency should be present in application. It can be enabled by two ways:

Add internal dependency inside the spring boot application

Add Chaos Monkey for Spring Boot as dependency for your project

<dependency>
    <groupId>de.codecentric</groupId>
    <artifactId>chaos-monkey-spring-boot</artifactId>
    <version>2.6.1</version>
</dependency>

Start your Spring Boot App with the chaos-monkey spring profile enabled

java -jar your-app.jar --spring.profiles.active=chaos-monkey --chaos.monkey.enabled=true

Add as external dependency

You can extend your existing application with the chaos-monkey and add it as an external dependency at startup, for this it is necessary to use the PropertiesLauncher of Spring Boot

<dependency>
    <groupId>de.codecentric</groupId>
    <artifactId>chaos-monkey-spring-boot</artifactId>
    <classifier>jar-with-dependencies</classifier>
    <version>2.6.1</version>
</dependency>

Start your Spring Boot application, add Chaos Monkey for Spring Boot JAR and properties

java -cp your-app.jar -Dloader.path=chaos-monkey-spring-boot-2.6.1-jar-with-dependencies.jar org.springframework.boot.loader.PropertiesLauncher --spring.profiles.active=chaos-monkey --spring.config.location=file:./chaos-monkey.properties

Default Validations¶

View the default validations

Spring boot pods are healthy before and after chaos injection

Minimal RBAC configuration example (optional)¶

NOTE

If you are using this experiment as part of a litmus workflow scheduled constructed & executed from chaos-center, then you may be making use of the litmus-admin RBAC, which is pre-installed in the cluster as part of the agent setup.

View the Minimal RBAC permissions

apiVersion: v1
kind: ServiceAccount
metadata:
  name: spring-boot-latency-sa
  namespace: default
  labels:
    name: spring-boot-latency-sa
    app.kubernetes.io/part-of: litmus
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: spring-boot-latency-sa
  namespace: default
  labels:
    name: spring-boot-latency-sa
    app.kubernetes.io/part-of: litmus
rules:
  # Create and monitor the experiment & helper pods
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create","delete","get","list","patch","update", "deletecollection"]
  # Performs CRUD operations on the events inside chaosengine and chaosresult
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create","get","list","patch","update"]
  # Track and get the runner, experiment, and helper pods log 
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get","list","watch"]
  # for creating and managing to execute commands inside target container
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["get","list","create"]
  # for configuring and monitor the experiment job by the chaos-runner pod
  - apiGroups: ["batch"]
    resources: ["jobs"]
    verbs: ["create","list","get","delete","deletecollection"]
  # for creation, status polling and deletion of litmus chaos resources used within a chaos workflow
  - apiGroups: ["litmuschaos.io"]
    resources: ["chaosengines","chaosexperiments","chaosresults"]
    verbs: ["create","list","get","patch","update","delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: spring-boot-latency-sa
  namespace: default
  labels:
    name: spring-boot-latency-sa
    app.kubernetes.io/part-of: litmus
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: spring-boot-latency-sa
subjects:
  - kind: ServiceAccount
    name: spring-boot-latency-sa
    namespace: default

Use this sample RBAC manifest to create a chaosServiceAccount in the desired (app) namespace. This example consists of the minimum necessary role permissions to execute the experiment.

Experiment tunables¶

check the experiment tunables

Mandatory Fields

Variables	Description	Notes
CM_PORT	It contains port of the spring boot application
LATENCY	It contains network latency to be injected(in ms)	default value is 2000

Optional Fields

Variables	Description	Notes
CM_LEVEL	It contains number of requests are to be attacked, n value means nth request will be affected	Defaults value: 1, it lies in [1,10000] range
CM_WATCHED_CUSTOM_SERVICES	It limits watched packages/classes/methods, it contains comma seperated list of fully qualified packages(class and/or method names)	ByDefault it is empty list, which means it target all services
CM_WATCHERS	It contains comma separated list of watchers from the following watchers list [controller, restController, service, repository, component, webClient]	ByDefault it is `restController`
TOTAL_CHAOS_DURATION	The time duration for chaos injection (seconds)	Defaults to 30
SEQUENCE	It defines sequence of chaos execution for multiple target pods	Default value: parallel. Supported: serial, parallel
PODS_AFFECTED_PERC	The Percentage of total pods to target	Defaults to 0% (corresponds to 1 replica)
LIB	The chaos lib used to inject the chaos	Defaults to `litmus`. Supported `litmus` only
RAMP_TIME	Period to wait before and after injection of chaos in sec

Experiment Examples¶

Common Experiment Tunables¶

Refer the common attributes and Spring Boot specific tunable to tune the common tunables for all experiments and spring-boot specific tunables.

Spring Boot Application Port¶

It tunes the spring-boot application port via CM_PORT ENV

Use the following example to tune this:

# kill spring-boot target application
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: spring-boot-chaos
  namespace: default
spec:
  appinfo:
    appns: 'default'
    applabel: 'app=spring-boot'
    appkind: 'deployment'
  # It can be active/stop
  engineState: 'active'
  chaosServiceAccount: spring-boot-latency-sa
  experiments:
    - name: spring-boot-latency
      spec:
        components:
          env:
            # port of the spring boot application
            - name: CM_PORT
              value: '8080'

Network Latency¶

It contains network latency value in ms. It can be tunes via LATENCY ENV

Use the following example to tune this:

# provide the network latency
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: spring-boot-chaos
  namespace: default
spec:
  appinfo:
    appns: 'default'
    applabel: 'app=spring-boot'
    appkind: 'deployment'
  # It can be active/stop
  engineState: 'active'
  chaosServiceAccount: spring-boot-latency-sa
  experiments:
    - name: spring-boot-latency
      spec:
        components:
          env:
            # provide the latency (ms)
            - name: LATENCY
              value: '2000'

            # port of the spring boot application
            - name: CM_PORT
              value: '8080'