Running performance tests

From Cyclos4 Wiki
Jump to: navigation, search

Introduction

This page explains how to run a performance test for Cyclos throught its REST API. The tools / frameworks used are:

  • k6: This is the main tool we use for making the performance tests
  • InfluxDB: Used to store the result metrics collected by k6
  • Grafana: Used to visualize the results stored in InfluxDB
  • Docker: Used to run the containerized version of the tools above

IMPORTANT:
Please before run any test disable the rate limits. Uncomment the following properties in the cyclos.properties an set 0 as value:

cyclos.rateLimit.global = 0
cyclos.rateLimit.ip = 0

Basic configuration

The use of Grafana and InfluxDB is optional. They allow storing all the results gathered in the course of a test and finally perform a detailed analysis through a dashboard defined for Grafana.
Here we will explain what are the minimum requirements to get a running test.
First of all, you need to install Docker because it is required to run the applications. You should follow the instructions here for your operating system. Also ensure the docker service is running.
Tests for k6 are written in JavaScript ES2015/ES6. This guide assumes all tests reside in a folder named src (if you want to use another name then you must apply the changes accordingly)

Simple test case

Here you have a very simple test you can use to check everything is working properly. You can run this kind of "ping" test as many times as you need because it does not make any change to your system, just get the data for login.

import { check, sleep } from "k6";
import http from "k6/http";
import { Counter } from "k6/metrics";

// ---------------------- INIT CODE (runs only once per VU) -------------------
export let errorCount = new Counter("errors");

/**
 * Simple test that perform a GET request to /auth/data-for-login
 * It could be used to tests all involved services are ok
 */
export default function () {
  const url = `${__ENV.API_ROOT}/auth/data-for-login`;

  const res = http.get(url);
  const checkResult = check(res, {
    Status: (r) => r.status === 200,
  });
  errorCount.add(!checkResult ? 1 : 0);
}

Copy the content and create a file named data-for-login.js in the src folder then run the test with (please adjust the API_ROOT variable to point to a running Cyclos instance)

docker run --network host -i --rm -v "$(pwd)/src:/src" loadimpact/k6 run -e API_ROOT=http://localhost:8080/api -u 1 -i 1 /src/data-for-login.js 

The command above runs the k6 image in a docker container with 1 virtual user (-u) and 1 iteration (-i), i.e the test will be executed only once.

If everything is correct then you can move on to the next test to register users (required to simulate several users making payments concurrently).

Register user test

This test depends on an utility script to generate the random usernames, this script must be placed in the src folder in a file named random-string.js

const Alpha = 'abcdefghijklmnopqrstuvwxyz';
const Digits = '0123456789';

export function randomString(n) {
  const result = new Array(n);
  for (let i = 0; i < n; i++) {
    result[i] = Alpha.charAt(Math.floor(Math.random() * Alpha.length));
  }
  return result.join('');
}

export function randomNumber(n) {
  const result = new Array(n);
  for (let i = 0; i < n; i++) {
    result[i] = Digits.charAt(Math.floor(Math.random() * Digits.length));
  }
  return result.join('');
}


Please create a new file named register-user.js in the src folder and paste the folowing:

import { check } from 'k6';
import http from 'k6/http';
import encoding from 'k6/encoding';
import { randomString } from './random-string.js';

export default function () {
  const url = `${__ENV.API_ROOT}/users`;
  const username = randomString(10);
  const body = JSON.stringify({
    group: 'members',
    name: `User ${username}`,
    username: username,
    email: `${username}@email.com`,
    skipActivationEmail: true,
    passwords: [
      {
        type: 'login',
        value: '1234'
      }
    ]
  });
  const params = {
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Basic ${encoding.b64encode('admin:1234')}`
    }
  };
  const res = http.post(url, body, params);
  check(res, {
    Status: r => r.status === 201
  });
};

This test registers a user running as an administrator. Before run, please ensure the following:

  • The group's internal name is members;
  • The administrator is correct (admin / 1234);
  • The internal name of the password used for login is login.

The following will register 10,000 users running with 20 virtual executing in parallel

docker run --network host -i --rm -v "$(pwd)/src:/src" loadimpact/k6 run -e API_ROOT=http://localhost:8080/api -u 20 -i 10000 /src/register-user.js

Payment test

In order to run this test you need first register users in Cyclos. Please run the #Register user test first.
Also the generated usernames must be exported to a file in order to the test can run.
Create a folder data and using the command-line interface to PostgreSQL (psql), please run:

psql cyclos4 -t -A -q -c "select u.username from users u inner join groups g on u.user_group_id = g.id where g.internal_name = 'members'" > data/users.txt

Important: Make sure there's no extra line in the end of the file, or it can cause test errors. The command above exports the usernames to a file users.txt in the data folder.

Please create a new file named make-payment.js in the src folder and paste the folowing:

import { check } from 'k6';
import http from 'k6/http';
import encoding from 'k6/encoding';

const Users = open('/data/users.txt').split('\n');

function user() {
  let user;
  // this is to avoid return null users
  while(!user) {
    user = Users[Math.floor(Math.random() * Users.length)];
  }
  return user;
}

export default function () {
  const url = `${__ENV.API_ROOT}/self/payments`;
  const payer = user();
  let payee = user();
  while (payer === payee) {
    payee = user();
  }

  const body = JSON.stringify({
    subject: payee,
    type: 'units.trade',
    amount: Math.floor(Math.random() * 1200) + 1
  });
  const params = {
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Basic ${encoding.b64encode(`${payer}:1234`)}`
    }
  };
  const res = http.post(url, body, params);
  check(res, {
    Status: r => r.status === 201
  });
};

Before run, please ensure the following

  • there is a payment type already defined for the members with internal name trade;
  • the account's internal name is 'units';
  • the users has enough credit (you could set the Default negative balance limit in the member product).

Then run the test:

docker run --network host -i --rm -v "$(pwd)/src:/src" -v "$(pwd)/data:/data" loadimpact/k6 run -e API_ROOT=http://localhost:8080/api -u 20 -i 20000 /src/make-payment.js

The above will make 20,000 payments in total executing with 20 concurrent (virtual) users (1,000 payment per user).

For all cases k6 will print in the console several metrics it collected, for a sample output please click here

Advanced configuration

You could use Grafana to visualize the results metrics in a dashboard. To use Grafana you have to setup k6 to store the results in InfluxDB using the --out parameter when running a test. For example, to store the results for the payment test you must add the following to the command: --out influxdb=http://localhost:8086/payments_load resulting in

docker run --network host -i --rm -v "$(pwd)/src:/src" -v "$(pwd)/data:/data" loadimpact/k6 run \
--out influxdb=http://localhost:8086/payments_load \
-e API_ROOT=http://localhost:8080/api -u 20 -i 20000 /src/make-payment.js

Run Influxdb

Create default configuration

Execute the following the first time to create a default configuration (ensure the influxdb folder is already created)

docker run --rm influxdb influxd config > influxdb/influxdb.conf

Create a new container

This must ve executed only for the first time. The volume /var/lib/influxdb is to persist the data and to avoid lose it between restarts

docker run --name influxdb -p 8086:8086 -v $(pwd)/influxdb:/var/lib/influxdb -v $(pwd)/influxdb/influxdb.conf:/etc/influxdb/influxdb.conf:ro influxdb -config /etc/influxdb/influxdb.conf

Start / stop the container

To manage the lifecycle of the container created above without creating a new one

docker start influxdb
docker stop influxdb

Run Grafana

To avoid permission errors we need to set the current user id when run grafana.

Ensure the grafana folder is already created and the current user has write permissions on it. Use id -u to know the id of the current user.

Create a new container

This must ve executed only for the first time.

docker run --name grafana --user 1000 -p 3000:3000 -v $(pwd)/grafana:/var/lib/grafana grafana/grafana

Start / stop the container

To manage the lifecycle of the container created above without creating a new one

docker start grafana
docker stop grafana

Use Grafana

To connect to grafana go to http://localhost:3000, enter admin / admin (you can change the password).

Create a new data source

  • Type: InfluxDB
  • Url: http://localhost:8086
  • Access: Could be Browser (a direct connection is made from the browser to the influxdb database)
  • Database:
    • payments_load (k6 results for the user payments)

k6 dashboard

Import the premade dashboard with id 2587 (https://grafana.com/dashboards/2587)

Please read this for more in deep documentation: