Why You Should Use a Performance Budget

I’d like to suggest a fundamental concept that will allow you to implement a true performance culture for your web projects: the performance budget.
It’s a kind of budget that isn’t expressed in dollars, but in seconds, megabytes or even the number of files served!

It’s a concept that should be used and understood by all kinds of online professionals, to create faster websites and to support them throughout their life-cycle, because speed matters.

Initial findings

The web keeps getting bigger. Pages are growing in size, and the number of different resources (like images, scripts, and style sheets) required to display a webpage keeps growing too.

Why? Because today’s websites are fully-fledged applications in their own right, with functionalities that can be very rich with a wide range of multimedia content.

If you consider the 1,000 highest traffic websites in the world, this is how the situation looks right now:

  • Average page weight is about 2MB
  • Average page requires about 128 requests
  • Average page requires 27 JavaScript files

Even if you put performance at the heart of the development philosophy when you first develop your site, the site will inevitably evolve and grow over time, with new features, new content, and new underlying technologies.

Your site will add richer functionality, including A/B testing, recommendation engines, retargeting, click-to-chat buttons, and so on: features that are becoming increasingly common to the extent that they are in a position to shape your site’s life-cycle.

The performance budget is a framework: a tool that enables you to place performance at the heart of everything you do and to avoid unnecessary distractions that lead to your site slowing down. Let’s not forget that a fast website is vital to a successful business.

Speed is a feature

In 2010, Urs Hölze reported a remark made by Larry Page, co-founder of Google:

As a product manager you should know that speed is product feature number one.

Speed is a philosophy at Google, something that they have considered to be a major differentiator right from the outset.
My suggestion aims to help you discover why this approach makes so much sense, and how it leads us to the concept of the performance budget.

You are probably familiar with the concept of uptime monitoring, which involves ensuring that a web server is indeed available, without which the site simply isn’t accessible. A site that is down will not generate any revenue until service is restored.
That can lead to long-term losses and you risk losing out entirely, at least until visitors can access your site again.

The speed of your site can be viewed as an extension of its availability: what use is having a site, even if it is technically ‘up’, if it takes 13 minutes to load? Obviously, none. Understanding that extremely long page loading times get you nowhere is completely intuitive.

But what about if your page took a minute to load? Perhaps a few patient souls would wait that long, but they would be the exception, not the norm!

Perceived waiting time is a complex subject, and there are many different aspects that can affect our patience. The first factor is undoubtedly the most significant: the value that we expect to receive at the end of the tunnel.

Who would be ready to wait three hours at the supermarket checkout? No one.
Who would be prepared to wait three hours to get the latest and greatest smartphone? Lots of people!

Be careful nonetheless, most people are far from ready to wait that long: perceived value is personal to each individual.

waiting queue

Let’s return to those brave souls who are prepared to wait in line for three hours. Would they still be ready to do so in hundred-degree (45°C) summer heat? Or in a snowstorm? Undoubtedly they wouldn’t be: our willingness and ability to wait depends on the context.

The difficulty that all web publishers face is putting abstract ideas into their proper context, taking not just their own experiences into account but also the diverse range of profiles that apply to their visitors.

So while the effective loading time for each user can be impacted by the user’s geographic location (e.g., the effects of latency), internet connection, hardware setup, or even the web browser used, the perception of the waiting involved can itself be impacted by the user’s general browsing habits and recent experiences (e.g., if the last site visited was particularly slow or particularly fast), preconceived ideas about your brand, or something as simple as the user’s mood!

Each user will also have his or her own perceived value of any particular function or feature.
Each additional feature will have a technical impact on your site, and can therefore affect the waiting period that each user experiences.

If, like Larry Page, you consider that speed is one of the main features of your product, it will be at the heart of the way you deal with each of these aspects.

A feature is pointless unless the value that it adds makes up for the waiting time that it causes.

The concept of a performance budget addresses this problem, not by forcing you to have a site that loads quickly, but by allowing you to consider each feature as having to justify the speed overhead that it takes. This therefore helps you to reach the judgments necessary to compare the value added by a feature with the trade-off in terms of speed.

What indicators should you use for performance budgets?

A ruler

The concept of a budget is not in itself complex, and the following examples are each very easy to understand:

  • “Our home page must be less than 1MB”
  • “On mobile, over 3G, all our pages must load in less than three seconds”
  • “The web server must respond in less than 200ms”

As we have described above, it’s a self-imposed restriction, hence the concept of a “budget” – you can’t spend more than X seconds or Y MB, etc.

The budgetary range can be more or less specific, and the thresholds that are applied can each work on a very different basis.

The first step is probably to integrate performance concepts at a very early stage of your project, at the initial requirements stage. I am a huge fan of Brad Frost‘s words on this topic:

Statements of work, project proposals and design briefs should explicitly and repeatedly call out performance as a primary goal. “The goal of this project is to create a stunning, flexible, lightning-fast experience…”

This ensures that you never forget the criterion, and can make a specific effort to monitor it closely. This is necessary at each stage of your project.
The later that performance issues are addressed, the more effort that is required to implement corrective measures.

In order to allow you to express your performance budget, Tim Kaldec has come up with a very interesting breakdown of the different metrics that are available, and about which you can learn more here:

Milestone timings

Examples: Time To First Byte, Load time,  domContentLoaded, Time to render, etc.

One benefit of these measures is that they are easy to monitor over time, which is ideal for implementing a framework that the technical team can follow. Be careful, however, as a page’s loading time is more complex than simply measuring it at a given period in time. Happily, speedindex is there to help.

SpeedIndex

This index is widely acknowledged by web performance experts as it provides a consolidated value that reports how a page will load, from the perspective of a user.

This meant that, for Tim, it was worth a category in its own right, which isn’t a bad thing. Personally, however, I would have added the Time to Interact (which forms part of the previous category) as the page only has value, even once it has loaded, when the user is in a position to interact with it.

If you’d like to find out more about speedindex, this article will provide explanations as well as two other metrics that address the time to render the page.

Quantity based metrics

Examples: Total number of requests; Overall page weight; Total image weight

These are measures that lack a direct, causative link to the user experience. Nevertheless, the major benefit is that these values that are much more easily understood and planned for than the impact on the user experience.

As Tim explains, it is much easier to understand that adding a script will increase the size of the page than it is to anticipate the effect of that script on speedindex.

Another example, raised very early in the design process (which I think is highly worthwhile) is provided by Daniel Mall: when the graphic layout is being created, limit the number of fonts that can be used! It’s a typical example of integrating performance budgets at the earliest possible stage: the layout proposed by your graphic designer might look stunning, but what if it inevitably leads to disastrous technical performance?

Rule based metrics

Examples: PageSpeed score, YSlow score, etc

Multiple tools are available on the market to enable you to ensure that best practices to encourage fast loading speeds are being followed. These scores therefore summarize the technical quality of your pages in terms of their performance: e.g., are the resource files compressed? are the images optimized? and so on.

Even then, there is still no direct relationship with the user experience, but by incorporating scores like these into your performance budget, you can be confident of detecting any major issues and make sure that you follow all the obvious steps to optimize your site from a technical perspective.

How should you set a budget threshold?

We have already discussed over the course of the last few paragraphs how performance budgets should be implemented as early as possible during a project, and should support the project throughout its life-cycle.

It is of course necessary to use measurements that are suitable for each stage of the project and the different stakeholders.

But what are the right methods for setting a budget? How much speed do you need? How should you choose the thresholds that will apply to your projects, and should they be hard limits at certain stages?

These questions were the subject of an article by Tim Kaldec, entitled “Fast Enough” that particularly recalls Steven Seow’s 20% rule: you only perceive the difference between two time periods if the gap between them is more than 20%.

If you use this as a rule of thumb, you can set your budget in relation to your competitors, whether to be better (at least 20% quicker) or at the very least to avoid seeming slower. (In other words, at most 20% slower).

Another interesting method that is mentioned throughout the article is to focus on the perception of time. A response in less than 100ms seems instantaneous; one second is the maximum time to retain a web user’s full attention, and after 10 seconds you’ve lost the user’s attention altogether.

Your budgets can also be formulated to take into account the overheads for your visitors: in many countries, mobile connections still have limited bandwidth, and mobile browsing on large sites is far from trivial! To find out more, I invite you to take a look at this excellent project: http://whatdoesmysitecost.com/

These few pieces of data will help you, but as ever, the perceived waiting time depends on the value generated and you need to put your budgets together with that in mind.

Adapt your budgets – or adapt to them?

When you add a new feature or new content, it’s no surprise if your budgets appear to suffer. But your goal is not to fit everything into your budget at all costs!

When a budget is exceeded, you need to continue each of these different options in sequence:

  • Optimize: whether it applies to a change or an existing element of your page, could you implement optimizations that would allow you to meet your budgets again?
  • Simplify: could any old content or features be removed if they are no longer necessary?
  • Give up: do the new features add so much value that you can justify exceeding your budget? If not, it may simply be the case that the new feature isn’t worth the trouble.
  • Adapt: if none of the responses suggested above is feasible, there’s no other solution than revising your challenging performance budgets. It’s not a question of giving up on the budget, so much as it is a question of evolving to allow you to incorporate the change. You’ll have a new framework to guide you in future.

Whatever the outcome, the performance budget is a useful tool: you’ll ask yourselves questions about the impact of your change, and you’ll be in control of the consequences of the change.

I hope that this article will have convinced you that it’s useful to implement performance budgets, and that together, we will be able to move forward to a world of faster web performance!

If you don’t know our tool yet, give a try to dareboost.com now, in a few seconds you’ll get a comprehensive report about your website speed and quality, and we offer website speed monitoring,  allowing you to make sure your website will always comply with your performance budget!

 

One thought on “Why You Should Use a Performance Budget

Leave a Reply

Your email address will not be published. Required fields are marked *

*