API Testing - TAAS (Testing As A Service)

Techsage API-RAFT (Reusable Automation Framework for Testing) Phases of TaaS

API stands for Application Programming Interface, which specifies how one component should interact with the other. It consists of a set of routines, protocols and tools for building the software applications.
API Testing is performed for the system using the API’s. The API-RAFT covers functional testing of the API’s using API specifications as a test basis. Scenarios are created to check all the API calls and cover happy and unhappy scenarios of the call while covering all parameters and payloads for these calls.

The test scope can also be understood with image below:

RESTful API, Web API, Remote API, Embedded API can be tested with the test framework. The framework supports various message types:

  1. JSON (JavaScript Object Notation) over Http/Https
  2. JSON over TCP/IP
  3. JSON RPC2 over Http
  4. JSON RPC2 over TCP/IP
  5. JSON RPC2 over Web sockets
  6. SOAP
  7. XML (Extensible Markup Language)

Some logical scenarios covered in the testing involves Exploring boundary conditions and ensuring that the test harness varies parameters of the API calls in ways that verify functionality and expose failures.
Verifying the behavior of the API which is considering the external environment conditions such as files, peripheral devices, and so forth.
Verifying the Sequence of API calls and check if the API's produce useful results from successive calls.

Other Tests performed on API's include

  • Return Value based on input condition: The return value from the API's are checked based on the input condition.
  • Empty responses: Verify if the API's does not return anything.
  • Verify if the API triggers some other event or calls another API. The Events output tracked and verified.
  • Verify if the API is updating any data structure.

Stress & stability?

For API's it is very important that the API's are Trustworthy. We will reuse the API testing framework and adapt it to run as Performance test framework.
The following will be tested while doing performance tests:

  • Load Test : The API's will be tested under load. Different specific Data models will be create using the performance specifications. The API's will be tested against the thresholds mentioned in the performance test specifications. CPU time/use, memory status, min-max calls status,failure stats, error percentages, Average response times, Elapsed time vs Average response times etc will be measures for the API's.
  • Reliability: tests will concentrate on verifying whether the API's are reliable over time. API component test will be executed at different times and in different conditions. Tests will cover executing user scenarios where calls will be repeatedly sent and API's will be checked for stability and reliability.
  • Availability: Automated tests will be run as pre-scheduled tests and also on different time intervals.