API performance testing using Postman Canary, a game-changing platform.

API testing plays a pivotal role in the software testing process, and most of the QA teams hugely rely on Postman for it.

QA teams typically utilize several API performance testing tools to assess the performance of APIs, such as Jmeter, LoadRunner, K6, and many more.

❌ This created friction when QA wanted to test the performance of APIs and APIs request and responses at the same time as they needed to utilize 2 different platforms.

💡 The good news is there is this new feature in Postman Canary that supports API performance testing at the same time. Postman Canary.

Postman Canary is more like a beta release of Postman, and they will release the new features to Postman Canary first before they release it to the official build of Postman.

✅ It did provide API performance testing features where users can have the capability to simulate load and stress tests. These tests help to uncover performance bottlenecks, assess the API’s scalability, and ensure it can handle anticipated or unexpected load conditions effectively.

👇🏻 This article showcases the practical implementation of Postman Canary to conduct straightforward API performance testing.

I will demo the features using this deck of cards api.

Shuffle cards sample request and response

Then, draw a card using the above request endpoint

🚀 To put it simply, what we gonna do is shuffle a deck of cards and draw one card from the deck that we just shuffled.

We are gonna stimulate 100 users (ramp up) to perform the same actions in 5 minutes and stimulate the performance test.

For the first endpoint, we are going to save the deck_id from the response for the second endpoint (draw a card).

We are shuffling the deck and saving the deck id. Then, we draw a card from this deck using the next endpoint.

For the second endpoint,

We are going to draw 1 card from the deck

One last step, run the collection and see if it works.

Run the 2 endpoints as a collection and see if it’s returning a 200 OK response code for both endpoints.

Then, we will be utilizing the performance tab in Postman Canary to perform API Performance Test.

Virtual Users: 100 users
Test Duration: 10 mins
Load Profile — this is to determine fixed 20 users or ramp up(slowly increase) to 20 users in the timeframe. We will choose ramp-up in this case.

Ramp-up duration: how long do you want the virtual user (increase) ramp-up to the desired maximum number of virtual users. Eg: Increase the user from 1 to 100 in 5 minutes ramp-up duration.

Run performance test with 100 users, ramp up in 5 mins.

You will see a running graph after the Run Collection button is clicked.

The running graph shows important metrics for you to evaluate the performance of the API.

1. Total requests sent — total number of requests sent to the endpoint
2. Requests/s- total number of requests sent to the endpoint per second.
3. Average response time — average response time across all requests hitting the endpoints.
4. Error rate — the percentage of requests that have returned a non-2xx response or were unable to hit the endpoints.

Let’s evaluate the results

In general, we can see that the total requests has been sent to each endpoint.

Min (ms): 294 to 293, roughly 3 ms. This means that the minimum response time from this endpoint is about 3ms.

Max (ms): 2536 to 2586, roughly 2500ms. This means that the maximum response time from this endpoint is about 2500ms.

90th(ms): 1404 o 1411, roughly 1400ms. This means that 90% of all the requests sample has a 1400ms response time. The remaining 10% may have a higher response time than 1400ms. This tells us that most users will be able to have a response time of 1400ms to these endpoints.

Error rate:

Another test example where the second endpoint cannot identify the deck_id

We can see a 100% of error rate for 2nd endpoint. This means that this endpoint has 100% failed across a total of 553 requests. (I have removed the deck id, and the endpoint does not know which deck to pick a card from, hence returning a 404 not found error code. )

🚀 This is a very simple example to show you how the error rate works. Bear in mind that this can be more complicated when testing the endpoint with different scenarios.

✅ Related Articles:

1. “From REST to GraphQL” — Wait, GraphQL? SQL? Are they the same?
2. Cloud Computing, let’s talk about it [Part 2]
3. AWS! The terms you must know 🤫
4. ISTQB 🧙🏻‍♀️The software testing Myth, do you need that?
5. Why is software testing needed in the first place?
6. What is Agile Methodology?
7. Java 101 for software testing (Using fruits and Vehicle as examples)
8. What do you need to know to test using Selenium in Java? (that I wished somebody had told me)
9. “Understand what Docker is in 5 minutes: Challenged Accepted”
10. Black Box or White Box Testing? (How many boxes do you need ?)

✅ ☎️👇🏻

Or connect with me on LinkedIn to discuss more!

#performacetesting #apitesting #SDLC #STLC #ISTQB #postman #postmancanary #postman #api