Load Testing a Fintech API with CSV Test Data Import

Introduction

We have organized this write-up into two parts to demonstrate two different features of Ddosify. In the first part, we will perform a load test on a GET endpoint that accepts base and target currency and returns their exchange rate of them. The rand() utility method is used to send different currencies on each request. In the second part, we will test a POST endpoint that performs exchange operations. We will use a CSV file that contains test data stored on our Test App's Database. Then we import this CSV file into Ddosify to replay the same transactions stored on DB, but in high concurrency. In both parts, we will gain insights into the reliability of our exchange API across high traffic.

The Environment

1. As in the previous blog post we will again use the Ddosify test API as a backend service.


2. We will use Ddosify Open Source Load Engine version v0.13.2. If you didn't install it yet, you can follow the readme to find the proper installation method for your operating system.


3. The configuration files used in this blog are available in this repository.

Load testing the exchange rate info API

Almost all fintech APIs have an endpoint that provides the exchange rate between two currencies. Our test backend API has a similar one, GET exchange_rate/<base_currency>/<target_currency>/. In this use case, we would like to learn the performance of this endpoint by providing random <base_currency> and <target_currency> on high IPS (iteration per second). Let's take a look at the below configuration first, then we will talk about the details of each section.

fetch_exchange_rates.json

{

  "iteration_count": 1000,

  "duration": 10,

  "load_type": "incremental",

  "env": {

    "currencies": [

      "AED",

      "ARS",

      "AUD",

      "BGN",

      "BHD",

      "BRL",

      "CAD",

      "CHF",

      "CNY",

      "DKK",

      "DZD",

      "EUR",

      "FKP",

      "INR",

      "JEP",

      "JPY",

      "KES",

      "KWD",

      "KZT",

      "MXN",

      "NZD",

      "RUB",

      "SEK",

      "SGD",

      "TRY",

      "USD"

    ],

    "baseUrl": "https://testserver.ddosify.com/exchange_rate"

  },

  "steps": [

    {

      "id": 1,

      "name": "Random Currency Fetch",

      "url": "{{baseUrl}}/{{rand(currencies)}}/{{rand(currencies)}}/"

    }

  ]

}

First of all, we have a list of currencies in our environment variables. If you wonder, we have fetched the supported currencies from the https://testserver.ddosify.com/currencies/ endpoint. The important part of this configuration is the URL field of the step. We used the rand() utility method on both <base_currency> and <target_currency> fields to inject a random currency. Before starting the test, we will use the --debug flag to inspect request headers, request body, response headers, and response body. Ddosify sends only 1 request in debug mode.

$ ddosify -config fetch_exchange_rates.json -debug



<.. truncated ..>

STEP (1) Random Currency Fetch

-------------------------------------

- Environment Variables

    currencies:    [AED ARS AUD BGN BHD BRL CAD CHF CNY DKK DZD EUR FKP INR JEP JPY KES KWD KZT MXN NZD RUB SEK SGD TRY USD]

    baseUrl:       https://testserver.ddosify.com/exchange_rate



- Test Data



- Request

    Target:     https://testserver.ddosify.com/exchange_rate/CNY/RUB/

    Method:     GET

    Headers:

    Body:



- Response

    StatusCode:    200

    Headers:

        Strict-Transport-Security:     max-age=31536000

        Content-Length:                17

        Connection:                    keep-alive

        Server:                        nginx/1.23.3

        Date:                          Wed, 08 Feb 2023 10:16:16 GMT

        Allow:                         GET, HEAD, OPTIONS

        Referrer-Policy:               same-origin

        Vary:                          Accept

        X-Frame-Options:               DENY

        Cross-Origin-Opener-Policy:    same-origin

        Content-Type:                  application/json

        X-Content-Type-Options:        nosniff

    Body:

        {

            "rate": 10.44834

        }

You may notice that we use the -debug flag on CLI instead of putting the debug:true key in the configuration file as we did in the previous article. Although these two ways are valid ways to run Ddosify in debug mode, the CLI flag has priority over the debug key. In this use case, we choose to enable debugging on CLI for ease of use.

Looks like everything is as expected. We have successfully injected random currencies on the request URL and the response contains the exchange rate (10.44834) along with the 200 HTTP OK status code. We can remove the debug flag and start the test.

$ ddosify -config fetch_exchange_rates.json



⚙️  Initializing...

🔥 Engine fired.



🛑 CTRL+C to gracefully stop.

✔️  Successful Run: 35     100%       ❌ Failed Run: 0        0%       ⏱️  Avg. Duration: 0.08713s

✔️  Successful Run: 101    100%       ❌ Failed Run: 0        0%       ⏱️  Avg. Duration: 0.12753s

✔️  Successful Run: 207    100%       ❌ Failed Run: 0        0%       ⏱️  Avg. Duration: 0.13816s

✔️  Successful Run: 326    100%       ❌ Failed Run: 0        0%       ⏱️  Avg. Duration: 0.13880s

✔️  Successful Run: 455    100%       ❌ Failed Run: 0        0%       ⏱️  Avg. Duration: 0.13609s

✔️  Successful Run: 560    100%       ❌ Failed Run: 0        0%       ⏱️  Avg. Duration: 0.29128s

✔️  Successful Run: 821    100%       ❌ Failed Run: 0        0%       ⏱️  Avg. Duration: 0.43275s

✔️  Successful Run: 998     99%       ❌ Failed Run: 2        1%       ⏱️  Avg. Duration: 0.44565s





RESULT

-------------------------------------

Success Count:    998   (99%)

Failed Count:     2     (1%)



Durations (Avg):

  DNS                  :0.0851s

  Connection           :0.0120s

  Request Write        :0.0000s

  Server Processing    :0.2938s

  Response Read        :0.0000s

  Total                :0.4457s



Status Code (Message) :Count

  200 (OK)    :998



Error Distribution (Count:Reason):

  2     :connection timeout

If we follow the real-time logs, we can easily analyze the Avg. Duration increases over time. The reason is we set the load_type as incremental in the configuration file. That means the iteration count increases every second, so our API receives more and more requests per following second. As a final note, 2 requests have been timed out at the last second of the test, so our test API couldn't totally handle 1000 requests in 10 seconds on incremental traffic.

Incremental Load Type

Importing CSV data to replay network traffic

In our test API there is a POST /exchange endpoint that expects the amount, base currency, and target currency as request payload and it exchanges the given amount from the base currency to the target currency. Let's assume that our imaginary customers have used this endpoint many times and our servers started to slow down. The development team did some performance improvements and we would like to replay the last 1000 transactions to test the performance of the new system. This is a great example to demonstrate how to supply test data to Ddosify Engine.

We exported the last 1K transactions from the database and save them to a CSV file called test_data.csv.

test_data.csv

e746d910-bece-4c37-8e6b-325f89acc4d2,969.2,DZD,KWD

531dae65-2cf4-4194-867e-71c0ea7b6381,485.04,ARS,KZT

c0ec5795-2c37-4c08-a384-82f3444d445e,750.09,FKP,JEP

ba53bbdc-715d-4b72-99a2-867d4167861d,374.03,RUB,JPY

a087955c-3dbc-4164-b5a1-9def635107c7,971.02,SGD,SEK

<.. truncated ..>

The CSV file consists of 4 columns. These are api_key for authenticating the user for the transaction, amount, base currency, and target currency. Now we can use the CSV test data import feature of Ddosify Engine to resend these transactions.

transaction_replay.json

{

  "iteration_count": 1000,

  "duration": 60,

  "load_type": "waved",

  "env": {

    "baseUrl": "https://testserver.ddosify.com/exchange/"

  },

  "data": {

    "transactions": {

      "path": "./test_data.csv",

      "vars": {

        "0": {

          "tag": "api_key"

        },

        "1": {

          "tag": "amount",

          "type": "float"

        },

        "2": {

          "tag": "base_currency"

        },

        "3": {

          "tag": "target_currency"

        }

      },

      "delimiter": ",",

      "allow_quota": false,

      "order": "sequential",

      "skip_first_line": false,

      "skip_empty_line": true

    }

  },

  "steps": [

    {

      "id": 1,

      "name": "Exchange",

      "url": "{{baseUrl}}",

      "method": "POST",

      "headers": {

        "X-API-KEY": "{{data.transactions.api_key}}",

        "Content-Type": "application/json"

      },

      "payload_file": "./exchange_body.json"

    }

  ]

}

transaction_replay.json is our main configuration file. In the data section we provide our CSV file in transactions name scope, so we can use it in our steps as {{data.transactions.tag}} format. There are lots of options we configured in this section, here are the details of them:

1. We used path to point the location of our CSV file.


2. 
   vars field is for matching the column index in the CSV to a variable name to use it in step configuration. For example, we named the first column in the CSV as api_key then we use it in the headers section of the Step, like X-API-KEY": "{{data.transactions.api_key}}. Note that we also assigned the type of the second column as float since we want that Ddosify should treat amount values as a float instead of a string.


3. 
   delimiter is the delimiter character of the CSV. While the default value of it is "," we type it to show you can use a custom delimiter.


4. 
   allow_quota enables the situation if a quote may appear in an unquoted field and a non-doubled quote may appear in a quoted field. Default is false. We put it to show this option. In our case, we don't expect quoted data in our CSV.


5. We set the the order as sequential since we want to fetch the lines in the same order that is located in the CSV. The default is random, in that case, Ddosify fetches lines randomly from the CSV.


6. We set skip_first_line as false because the test_data.csv has no headers in the first line.


7. Although we don't have any empty lines on our CSV file, we configured skip_empty_line to show that there is an option like this.

exchange_body.json

{

  "amount": "{{data.transactions.amount}}",

  "base": "{{data.transactions.base_currency}}",

  "target": "{{data.transactions.target_currency}}"

}

exchange_body.json is our POST payload, we filled the fields with the test data fetched from the CSV file. Let's debug our scenario and see what will happen.

$ ddosify -config transaction_replay.json -debug



<.. truncated ..>



STEP (1) Exchange

-------------------------------------

- Environment Variables

    baseUrl:    https://testserver.ddosify.com/exchange/



- Test Data

    data.transactions.base_currency:      DZD

    data.transactions.target_currency:    KWD

    data.transactions.api_key:            e746d910-bece-4c37-8e6b-325f89acc4d2

    data.transactions.amount:             969.2



- Request

    Target:     https://testserver.ddosify.com/exchange/

    Method:     POST

    Headers:

        Content-Type:    application/json

        X-Api-Key:       e746d910-bece-4c37-8e6b-325f89acc4d2

    Body:

        {

            "amount": 969.2,

            "base": "DZD",

            "target": "KWD"

        }



- Response

    StatusCode:    200

    Headers:

        Allow:                         POST, OPTIONS

        <.. truncated ..>

    Body:

        {

            "amount": 2.1690696000000003,

            "currency": "KWD",

            "success": true

        }

The Test Data section in the Debug result shows that we have successfully fetched the first row of our CSV file. Also, everything looks correct on the request payload and X-Api-Key header value. Since we receive the 200 HTTP OK status code we can start the actual test. Remember our configuration on transaction_replay.json, we set iteration_count to 1000 since we have 1000 lines of data in our CSV. Instead of incremental, we will use waved load type to simulate more realistic network traffic.

If the iteration_count is greater than the data count on the CSV file, then Ddosify Engine loops over the file .

Waved load type

$ ddosify -config transaction_replay.json



<.. truncated ..>



RESULT

-------------------------------------

Success Count:    1000  (100%)

Failed Count:     0     (0%)



Durations (Avg):

  DNS                  :0.0037s

  Connection           :0.0015s

  Request Write        :0.0000s

  Server Processing    :0.0831s

  Response Read        :0.0000s

  Total                :0.0886s



Status Code (Message) :Count

  200 (OK)           :1000

The result shows that our backend API can handle 1000 exchange transactions in 60 seconds in the waved load type. We can test the system under different conditions by changing duration, load_type, and iteration_count parameters.

Conclusion

At first, we showed how to use rand() utility method of Ddosify Engine to use environment variables in advance. Then we demonstrated the usage of Test Data import via CSV file along with a traffic replay scenario.

Ddosify Engine has more capabilities on each release, follow it on GitHub to stay updated.

You can find the files we used in this article at Ddosify Blog Examples repository. If you need assistance, you can join our Discord channel.