Performance Engineering of Web-based Applications: 4 Essential Guidelines

When one puts “high performance”, “scalable” and “web applications” together, what comes to mind are websites of the likes of Facebook; sites with millions of users initiating requests of all kinds every single second to what seems to be a black-belt ninja system which, against all odds and adversaries, manages to respond flawlessly every single time… but ahem.. let’s not get carried away.

I’d envy you if you’ve worked on projects of that scale. But, if you’re like me, the scale of projects you’ve worked on has probably ranged from few users to few hundreds or thousands at most. Well, the good news is that there’s a lot of exciting principles to learn when attempting to engineer a system for performance, and they apply to both small-scale systems and large ones.

The Three Basic Measures of System Performance

The performance of a given system when processing a certain Workload can be measured in terms of three major characteristics:

Resource Utilization – The amount of system resources utilized to process a Workload including CPU, Memory, I/O, and Storage. Normally, the objective is to keep Resource Utilization measures in preferred “normal” ranges when the system is processing the Typical Workload.

Throughput – Simply, it is a measure of how fast your system can process Workload Units, regardless how you define what a Workload Unit is for your system. “How fast” is measured in terms of number of Workload Units per unit time, usually second or millisecond. Your Workload Units might be anything from database transactions to file reads/writes.

Response Time – Probably the best indicative performance measure of all – for a single obvious reason – it directly affects your End Users’ experience. It is through Response Time your End Users perceive the overall performance of your system. That being said,  it usually makes sense to gear your Performance Engineering activities towards improving this measure first.

All performance engineering activities aim to improve upon these measures one way or another.

Sketch Your System’s Workloads

- What is a Workload?

I’ve mentioned the term “Workload” many times already and because this concept is so central to the discussion of Performance Engineering, it has to be defined no later than now. A Workload simply is an amount of Work your system is expected to carry out during its operation, defined in terms of specific measurable units such as Transactions. When I mention the “Typical Workload” (or TWL), I mean the Workload under normal operating conditions with the number of users and resource utilization planned for initially during the inception of the system.

However, what are far more interesting to us are the “Peak Workload” (PWL) and the “Spike Workload” (SWL) respectively. Since the Workload of your system is likely to vary over time, it is normal that there will be peaks and troughs. The Peak Workload represents the highest Workload you’d expect a system to encounter and be able to handle at some point during operation. The Peak Workload occurs during the “rush hour” of system use, and may last for hours or days.

The Spike Workload is an unusual, unexpected, and short-term surge in the Workload imposed on your system. For example, the media may suddenly volunteer to provide coverage  for your shiny website causing an unexpectedly large number of people to rush to check it out, causing a Spike Workload.

A vivid example of the damage caused by spikes is when fark.com or slashdot decides that some website has a good piece of news worthy of its readers’ attention. As soon as a link to the featured website is posted, bazillions of users overflow the poor unsuspecting website, all in a very short duration. HTTP Error 500.

The all important question here is how your system should react when it encounters such kinds of Workloads. The answer: your system should be designed  to handle the Typical Workload with ease. It should be able to handle the Peak Workload adequately as well. And since the Spike Workload is by definition unexpected, your system should respond to such condition by 1) alerting system administrators, and more importantly: 2) degrading gracefully.

Graceful degradation could be considered a part of a greater concept called Homeostasis, the ability of a system to regulate itself to maintain a stable condition. Graceful degradation has to be taken into consideration during development, especially when the system has to be highly available.

Performance Engineering: A Set of Guidelines

Now the following guidelines are just plain common sense. These guidelines should best be followed during the development lifecycle to identify which areas of the system need to be designed with performance in mind. However, you might be assigned to tune an underperforming system, and in such case, you should also start by these guidelines — but armed instead with usage data gathered from a real-life production environment.

1. Set Your Priorities Using the 80/20 Rule

First, you have to identify which functions (or Use Cases) in your system need to be engineered for performance. It makes sense to start by focusing on the 20% of Use Cases that are done by end users 80% of the time and create measurable Workload. If you have 1000 users who would have to perform a “Search” use case 80% of the time for example, that makes the “Search” use case a good candidate for your focus.

Also, if users of your system are divided into groups (by role for example), then you have to do the previous analysis for every user group.

2. Set Your Performance Targets

Having identified Use Cases that are good representatives of  the Workload, you need to set their associated performance targets in terms of our three essential measures: Response Time, Throughput, and Resource Utilization. These performance targets have to be set with customer stakeholders.

You start your discussion with your customer by the performance targets of the business itself.  If the business requires that a user be able to complete 600 transactions per day, there you go. By the end of the discussion you should have enough information that can be easily translated into system performance targets in terms of Response Time, Throughput, and Resource Utilization. Sometimes that translation is easy, sometimes you’ll have to make many assumptions. While its OK and even necessary at times to make assumptions, they have to be documented and justified.

You also have to consider with your customer when the system is going be most heavily used daily, monthly, or yearly. For example, in the banking sector, usually the end of every month is a pretty busy time. Also, how many concurrent users should be supported by your system for every user group and therefore, as a whole?

This discussion will ultimately lead you to what are the expected Typical and Peak Workloads for your system. But that’s not the whole story. What about future growth? If the business depending on your system grows, so would your user base. Growth is even trickier with public websites, because you cannot know the extent and duration of this unbounded growth.

Following is a simple example of sketching Workloads for a hypothetical Item Ordering System, just to demonstrate what I’ve just discussed:

[Typical Workload]

3 Administrators concurrent, workload-representative Use Cases(Monitor Orders 2000/day)

300-500 Users concurrent, workload-representative Use Cases( Search 5,000/day, Browse Item 60,000/day, Order Item 1000/day)

Response Time: 2 Seconds maximum

[Peak Workload]

4 Administrators concurrent, Workload-representative Use Cases(Monitor Orders 3000/day)

1000-1200 Users concurrent, Workload-representative Use Cases( Search 10,000/day, Browse Item 120,000/day, Order Item 2000/day)

Response Time: 4 Seconds maximum

3. Analyze Candidate Use Cases

So, you have identified Use Cases that represent Typical and Peak workloads for the different user groups. You know which Use Cases need to be engineered for performance and scalability. What’s next? You’ve probably guessed it, delving into the design and implementation details of such Use Cases. The higher and more stressful the Peak Workload is expected to be, the more attention you’ll have to give to the details. At the extreme, you may have to aggressively tune algorithms and optimize memory usage.

4. Prepare Your Tools

Finally, you need two kinds of tools when doing Performance Engineering: a Profiler and a Performance Testing tool. The Profiler is your informant that gives you valuable insider information – and sometimes even amazing insights – about your application’s run-time behavior including Memory footprint, execution times of Class methods, and even code coverage reports. However, these features are highly tool-dependent.

As for Performance Testing tools, they are invaluable for helping you test the performance of your system under different Workloads. JMeter is one open-source performance testing tool that is pretty stable now and has been around for a while. Also, I recommend IBM’s Rational Performance Tester if your organization is willing to invest some cash.

Performance Testing tool makers are aware how challenging it is to design, implement, and most of all: execute Performance tests. That’s why you’ll find some advanced features common in most tools; such as proxy server-based recording of HTTP activity for example, that allow you to record test cases and then replay them against your system in volumes.

Hey! What About Scalability!

Last, a concluding remark. A concept that is closely related to Performance is Scalability. Scalability is literally the ability of a system to “scale” with the increase in workload.

I had to mention Scalability because there’s a widely spread misconception about it. Contrary to a very common belief, Scalability is not a system’s ability to handle a heavy workload. That should be called “a properly engineered system” instead.

Scalability is actually all about how easily an entire system – both hardware and software – can grow (e.g. by adding more hardware or replacing aging machines by more powerful ones) or be modified (by virtue of flexibility of application architecture and design) to handle Workload increase beyond the point it was originally engineered to handle. Especially if more functionality is added to the system over time as part of the system’s evolution.

That’s all for now, folks!