The most common types of tools that one thinks of when the topic of performance testing is discussed are the load generation tools. Dozens of load generation tools come up when any term related to performance testing is entered into a search engine. However, this category of tool is only one of several. This article will explain each tool category and list some examples of each.
Load/Stress Generation Tools:
Load and Stress Generation tools are used to simulate the multi-user traffic that the system under test is expected to handle. Generally, they interact at the protocol layer, but recently, cloud-based tools have come online to exercise the system at the GUI/Browser level. All of these will help provide insight into causes for slow performance.
Below is a list of Load Generation tools that should be familiar to the Performance Tester:
- Tricentis’ Flood.io
- Performance Center / LoadRunner / StormRunner
Monitoring tools are primarily used to watch for potential errors and notify an administrator if any are detected. Monitoring allows businesses to ensure that the service in question is available to users when needed. Commonly tracked metrics include response times, availability/uptime, memory or CPU usage, disk read/write times, and network bandwidth usage. The type of service (e.g. a website, or a corporate network) will dictate the type of tool used; some common examples of tools, broken into categories, can be found below.
(Live) Application Performance Monitoring:
Real User Monitoring:
A profiler is a performance analysis tool that, most commonly, measures the frequency and duration of function calls. There are other specific types of profilers (e.g. memory profilers) in addition to more comprehensive profilers, capable of gathering extensive performance data regarding code runtimes, database transactions, etc. This category of tools is typically specific to the technology stack being used.
- SmartBear Software’s AQtime
- Microsoft’s CLR Profiler
- Apple Inc.‘s Shark (OSX)
- OProfile (Linux)
- Intel VTune
- Oracle Performance Analyzer
Visual and Statistical Data Analysis Tools:
One often overlooked category of tools used by Performance Testers are those used to support the transaction modeling, and results analysis. During the planning phase, it is common for a Performance Tester to use statistical analysis against data that captures current transaction/usage rates to further understand the behavior a performance test is trying to recreate. Similarly, data visualizations and aggregators are often helpful in assisting a Performance Tester in identifying bottlenecks and failure points. Useful tools include: