Event-driven Kubernetes testing with Testkube and Tracetest

We are pleased to announce that Tracetest now works with , you can now run event-driven trace-based tests in your Kubernetes cluster!

Note: Check out this

What is Testkube?

and testing framework designed for testers and developers who use Kubernetes. It integrates test orchestration and execution into Kubernetes and your CI/CD/GitOps pipeline. You can automate the execution of your tests, regardless of the testing framework, by using Testkube's executors or creating your own.

By adopting Kubernetes constructs and GitOps, you can perform K8s-native testing. You can use Kubernetes CRDs to manage and store tests, allowing you to validate your applications by executing tests from inside your cluster.

Use any CI/CD framework for any testing scenario. By decoupling them from your CI/CD, you will spend less time integrating different testing tools.

Analyze all your test results in a centralized place. After running your tests, you can view the results in an intuitive UI, regardless of which testing framework you used.

Debug test failures with ease. With Testkube, it's easy to see all the results, logs, and artifacts of your tests in one place.

In addition, Testkube allows you to easily store and download files generated by your tests from your Kubernetes cluster. All files generated from your tests are saved.

What is Tracetest?

. It allows you to quickly build integration and end-to-end tests, powered by your distributed traces.

Tracetest uses your existing distributed traces to power trace-based testing with assertions against your trace data at every point of the request transaction. You only need to point Tracetest to your existing trace data source, or send traces to Tracetest directly!

Tracetest makes it possible to:

• Define tests and assertions against every single microservice that a trace goes through.


• Work with your existing distributed tracing solution, allowing you to build tests based on your already instrumented system.


• Define multiple transaction triggers, such as a GET against an API endpoint, a GRPC request, etc.


• Define assertions against both the response and trace data, ensuring both your response and the underlying processes worked correctly, quickly, and without errors.


• Save and run the tests manually or via CI build jobs with the Tracetest CLI.

Tracetest Now Works with Testkube!

Tracetest now works with Testkube, allowing you to unlock Testkube's capacity with Tracetest, and leverage OpenTelemetry instrumentation in your services to run end-to-end and integration testing. It works thanks to the

Try Tracetest with Testkube

To run trace-based tests with Tracetest and Testkube, make sure you have these three things installed before starting.

1. A running Kubernetes cluster, either locally or in the cloud

Install Testkube

Testkube is open-source and easy to install. Start by installing the Testkube CLI by following 

From here, follow the

Install Tracetest

Tracetest is open-source and easy to install. Start by installing the Tracetest CLI by following  for more info.

From here, follow the

Once the server is installed, open the Tracetest Web UI in the browser and follow the instructions for connecting the

If you look closely, you’ll see the OpenTelemetry Sample Configuration from the settings page above matches the collector.config.yaml that was generated by the Tracetest CLI when provisioning the Tracetest server.

# collector.config.yaml



receivers:

  otlp:

    protocols:

      grpc:

      http:



processors:

  batch:

    timeout: 100ms



exporters:

  otlp/1:

    endpoint: tracetest:21321

    tls:

      insecure: true



service:

  pipelines:

    traces/1:

      receivers: [otlp]

      processors: [batch]

      exporters: [otlp/1]

Create a Test in Tracetest

Proceed to add a test spec to assert all database queries return within 500 ms. Click the Test tab and proceed to click the Add Test Spec button.

In the span selector make sure to add this selector:

CODE Kopieren

span[tracetest.span.type="database"]

In the assertion field add:

CODE Kopieren

attr:tracetest.span.duration < 500ms

Save the test spec and publish the test.

This is an example of a trace-based test that asserts against every single part of an HTTP transaction, including all interactions with the database.

However, Tracetest cannot run this test as part of your CI/CD without integrating it with another tool.

Let's introduce how Testkube makes it possible.

Deploy the Tracetest Testkube Executor

Note: As of the latest Teskube release, the Tracetest Testkube executor has been added to the Testkube’s available executors out-of-the-box. If you have an older version of Testkube running, proceed with deploying the Tracetest Testkube executor manually.

Testkube works with the concept of Executors. An Executor is a wrapper around a testing framework, Tracetest in this case, in the form of a Docker container and runs as a Kubernetes job.

To start you need to register and deploy the Tracetest executor in your cluster using the Testkube CLI. Run the command below in your terminal.

CODE Kopieren

kubectl testkube create executor --image kubeshop/testkube-executor-tracetest:latest --types "tracetest/test" --name tracetest-executor --icon-uri icon --content-type string --content-type file-uri



[Output]

Executor created tracetest-executor 🥇

Trigger a Trace-based Test in Tracetest with Testkube

In the Tracetest Web UI, click the ⚙️ button in the top right. Then click Test Definition.

Save this into a file called test.yaml.

CODE Kopieren

# test.yaml



type: Test

spec:

  id: RUkKQ_aVR

  name: Pokeshop - List

  description: Get a Pokemon

  trigger:

    type: http

    httpRequest:

      url: http://demo-pokemon-api.demo/pokemon?take=20&skip=0

      method: GET

      headers:

      - key: Content-Type

        value: application/json

  specs:

  - name: Database queries less than 500 ms

    selector: span[tracetest.span.type="database"]

    assertions:

    - attr:tracetest.span.duration  <  500ms

Execute the following command to create the test object in Testkube. Do not forget to provide the path to your Tracetest definition file using the --file argument, and also the Tracetest server endpoint using the TRACETEST_ENDPOINT --variable.

Remember that your TRACETEST_ENDPOINT should be reachable from Testkube in your cluster. Use your Tracetest service's CLUSTER-IP:PORT. E.g: 10.96.93.106:11633.

CODE Kopieren

kubectl testkube create test --file ./test.yaml --type "tracetest/test" --name pokeshop-tracetest-test --variable TRACETEST_ENDPOINT=http://CLUSTER-IP:PORT



[Output]

Test created testkube / pokeshop-tracetest-test 🥇

Opening the Testkube Dashboard will show the test is created successfully.

If the test passes, it'll look like this in the terminal.

CODE Kopieren

[Output]



Type:              tracetest/test

Name:              pokeshop-tracetest-test

Execution ID:      6418873d9922b3e1003dd5b8

Execution name:    pokeshop-tracetest-test-4

Execution number:  4

Status:            running

Start time:        2023-03-20 16:18:05.60245717 +0000 UTC

End time:          0001-01-01 00:00:00 +0000 UTC

Duration:



  Variables:    1

  - TRACETEST_ENDPOINT = http://10.96.93.106:11633



Getting logs from test job 6418873d9922b3e1003dd5b8

Execution completed

🔬 Executing in directory :

 $ tracetest test run --server-url http://10.96.93.106:11633 --definition /tmp/test-content1901459587 --wait-for-result --output pretty

✔ Pokeshop - List (http://10.96.93.106:11633/test/RUkKQ_aVR/run/3/test)

    ✔ Database queries less than 500 ms



✅ Execution succeeded

Execution completed ✔ Pokeshop - List (http://10.96.93.106:11633/test/RUkKQ_aVR/run/3/test)

    ✔ Database queries less than 500 ms

And, like this in Testkube Dashboard.

. Now with trace-based testing available, full, in-depth assertions against trace data is available, not just a response.

By using Testkube's 

Running Event-driven Trace-based Tests

Event-based testing in Kubernetes is a critical aspect of ensuring the reliability and performance of microservices in Kubernetes. This testing approach involves observing events that are emitted by various components and services in the system to trigger tests against the system’s components under various conditions.

The main benefit of event-based testing is that it provides a more comprehensive testing approach than traditional unit, integration, and functional testing. With event-based testing, testers can simulate real-world scenarios and test the system's response to different types of input, load, and failure, while also verifying the system's ability to recover from such events.

To effectively perform event-based testing in Kubernetes, you’ll use Testkube as an event monitoring and management system that can capture and analyze the events generated by the system.

This sample will trigger a test when a deployment is scaled.

You’ve configured the Tracetest assertions to make sure all database queries finish within 500ms. Now, define a trigger that will run the trace-based test every time the deployment scales to ensure each replica satisfies the defined assertions.

Define a Test Trigger for the Deployment resource to run the trace-based test when a deployment-scale-update event occurs:

CODE Kopieren

# testkube-trigger.yaml



apiVersion: tests.testkube.io/v1

kind: TestTrigger

metadata:

  name: deployment-scale-update-trigger

  namespace: testkube

spec:

  resource: deployment

  resourceSelector:

    labelSelector:

      matchLabels:

        app.kubernetes.io/instance: demo

  event: deployment-scale-update

  action: run

  execution: test

  testSelector:

    name: pokeshop-tracetest-test

    namespace: testkube

Save the file, name it testkube-trigger.yaml and apply it.

CODE Kopieren

kubectl apply -f ./testkube-trigger.yaml

Running Tests in a Test Suite or Transaction

Running singular, isolated, event-driven tests has its own important use cases and values. But, in the wild, you’ll more often rely on chaining multiple tests together into a transaction or test suite.

Both Testkube and Tracetest support such logical constructs.

In Tracetest they’re called .

Tracetest Transactions

Running end-to-end tests is not simple. It requires configuration before the actual test can be run, such as creating a new user or removing all items from a cart. Therefore, it's important to be able to execute multiple steps as part of your transaction. Tracetest introduces the concept of Transactions to achieve this goal.

A transaction is a group of steps executed in a defined order, where each step is a test that can access information exported by previous tests.

The main benefit of using transactions is the ability to chain tests together and use values obtained in one test as input for a subsequent test.

When a test is executed within a transaction, if it generates any outputs, the test outputs will be injected into the transaction context environment. After the outputs are injected, all subsequent tests to be run within the transaction will be able to reference those values with env:VARIABLE_NAME.

Note: Outputs generated by steps don't modify the selected environment. It only modifies the transaction run context object.

Tracetest allows tests to declare outputs. An output is a value that is extracted from a trace by providing a  to get the value from the selected spans.

Run a Tracetest Transaction

Start by adding a new test by clicking Create > Create New Test > HTTP Request > Next > Choose Example (dropdown) > Pokeshop - Add (generates a sample test from the Tracetest demo) > Next > URL is prefilled with http://demo-pokemon-api.demo/pokemon > Create & Run.

The request body will be populated with this JSON.

CODE Kopieren

{"name":"meowth","type":"normal","imageUrl":"https://assets.pokemon.com/assets/cms2/img/pokedex/full/052.png","isFeatured":true}

Navigate to Test > Outputs and click Add Test Output. Select the create pokeshop.pokemon database span.

CODE Kopieren

span[tracetest.span.type="database" name="create pokeshop.pokemon" db.system="postgres" db.name="pokeshop" db.user="ashketchum" db.operation="create" db.sql.table="pokemon"]

The attribute to export the Pokemon’s id as a value is:

CODE Kopieren

attr:db.result | json_path 'id'

Finally give it a name:

CODE Kopieren

add_pokemon_db_result_id

Save the test spec and click publish.

Now you can add a transaction and see how it works together. Click Create > Create New Transaction > Give it a name > Create.

Once created, add the Add and List tests to the transaction list. You’ll see the defined variables on the right, below the execution steps. The List test is passing as it is correctly asserting that the result of the List request contains the exported variable from the Add test.

in Testkube are a way to orchestrate different test steps and entirely different testing frameworks to run in a suite. Your front-end team uses Cypress for browser tests, while the back-end team uses Tracetest. You may also have Postman collections testing various parts of your apps.

With test suites, you can orchestrate different test steps and combine them to run in a sequence. Even if each team runs its tests on its own, they can ultimately be combined and triggered from one location by a test suite.

Learn More About Kubernetes Testing

Combined, Testkube and Tracetest provide a comprehensive testing solution for Kubernetes applications. By utilizing Testkube triggers, you can automatically initiate trace-based tests with Tracetest, ensuring that your services are adhering to defined SLAs.

Tracetest provides detailed distributed trace data, allowing you to gain insight into the behavior of your application at a granular level, and ultimately create assertions against this data to write bullet-proof tests.

Would you like to learn more about Tracetest and what it brings to the table? Check the it today!

To explore more options Testkube gives you, check out the documentation on .

Also, please feel free to join our , or schedule a time to chat 1:1.

Vollständiger Original-Bericht

Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf dev.to.

↗ Original-Artikel auf dev.to lesen