This article is a transcript of my conference about Performance Budgets, held during the We Love Speed event (Bordeaux, October 9, 2018).
[ Slides ]
Video (french):
My name is Damien Jubeau and I am CEO and co-founder of Dareboost (startup based in Rennes) which offers a Software as a Service dedicated to front-end web performance. From website speed test to User Journey monitoring, Dareboost provides teams with all the tools they need to work and collaborate on performance. I’m here today to talk about performance budgets, a method to get your website to lose weight.
An anecdote to begin: in 2016 the weight of the average web page reached the size of the complete version of the original Doom video game (from the early 90s), 2.4 MB. Doom is hours of play, a rendering engine, sound effects, graphic textures, etc.
“We made the web in our own image, obese.”
This quote is a good summary of the situation, the web has a tendency to obesity. And when you look at it more closely, it’s not really surprising.
Indeed, a website is something complex. There are a lot of constraints that come from a lot of stakeholders. For example, at the organizational level, you have financial constraints, which will of course impact what can – or can not – be done.
Another important point is the income model related to the website. I am thinking, for example, of press and media sites monetized through advertising. I imagine that most of you are aware of the challenges that come from advertising models in the web performance field… There are many other organizational constraints, such as artistic direction or again the political management than some companies are facing extensively.
At a more operational level, we can speak of software entropy, with a tendency towards technical debt. A website is a system that will live, evolve, and therefore tend to become more complex or even deteriorate itself. You can also face lack of skills in teams. There are also network and infrastructure contingencies, or again all risks related to third-party content quality of service.
To summarize all this: a website is a complex system, involving many stakeholders, bringing all their sets of constraints. A complex system that will evolve and tend to obesity and slowness if the necessary safeguards are not taken.
Still, we need to have fast pages. I really like this quote from Larry Page, who invites us to define speed as the number one feature of the product. Why do we consider the need for speed to be such important? Because your visitors, when they come to your pages, have a particular need they want to fulfill. That might be to find a piece of information, to buy a product, etc. In any case, your visitors have an ability to wait that is related to the nature of their need, that is related to the value they are expecting to gain at the end. Therefore, you have no choice but to meet this need within what would be an acceptable delay for the user. Promoting speed as the number 1 feature of your website is to ensure that you respond quickly enough to this need, before the customer/user/visitor realizing there can be some alternatives to your website. If you’re not fast enough, the rest of your work will be meaningless. Any other feature would become futile if the expected value can’t be delivered in a time frame that your visitor would consider acceptable.
How do we prevent our web pages from obesity and slowness?
How do we prevent our web pages from obesity and slowness? How to ensure that speed is one of the main concern of the teams? With performance budgets.
Let’s start with an example of a budget, perhaps the simplest: “I want the website to be fast”. It is a budget that a non-technical decision-maker could easily express.
This budget is not precise at all, neither it is objective. But it gives a direction to the project, an expectation on performance. It is a first orientation that only needs to be declined in the next steps.
If we want to go with something a little more precise, here’s another example of a performance budget: “the weight of the pages should be less than 1MB”. This is an example of a budget that should allow me to communicate with a designer, for example.
As a designer, I can anticipate that designing a layout using 18 carousels, 13 web fonts and an HD video in the background is not something realistic to comply with the 1MB threshold.
This kind of budget is great because we can easily anticipate the impacts of a change before having it to be implemented. So we can prevent performance issues, rather than correct them in late phases, which would always be way more expensive.
Finally, a third example of a performance budget with something even more precise and webperf-oriented: “Our pages must have a speed index lower than 2000 on mobile phone over 3G connection”.
The Speed Index, about which Jean-Pierre talked about this morning, is an indicator that can be obtained thanks to synthetic speed test. The budget also describing a context for the test. Here we have something we can control throughout the life cycle of the project, to make sure the website is still complying with its budget. And of course we would have to make sure to take actions quickly in case of non-compliance.
How to use performance budgets?
Let’s look at how to use performance budgets on a daily basis. Let us see how they will play their role as safeguards and how they can be a great tool to take better decisions.
Let’s imagine a 800 kB homepage, and a budget of 1MB for the page. My marketing department requested to add a carousel on the homepage to promote more products. The motivation behind: highlighted products are more purchased, and therefore generate more income. My problem is the requested change is too heavy and it can’t comply with my budget when adding it to the page. What can we do?
First of all, may be I can optimize the feature. Here, I can optimize the images, minimize JavaScript code, make sure it is properly gzipped, etc.
Hopefully this will allow me to stick to my budget and we will be done here. Otherwise, I have another action to save me some bytes. I will try to find some room for optimization in the existing content of the page, to do some clean up. For example, may be it’s now a good time to remove the A/B testing solution that we haven’t used for 6 months…
After that, hopefully I’ll stick to my budget and everything will be fine.
And if not? It’s time to have concerns about what we’re doing, to question the added value of the feature. We can’t stick to our budget, the page will be heavier / slower than what we allowed by defining this budget. Is adding a carousel really a good idea? Are not we risking to hurt badly our current revenues by making this change?
If this added value of the feature does not seem quite enough, then we have no interest in implementing the evolution, it was a mistake (on this subject: shouldiuseacarousel.com). The smart move would be to abandon this new feature.
On the other hand, if the team is convinced of the opportunity behind this evolution, whether it is additional value for the visitors or the business, we will keep moving forward and implement the feature, accepting the additional weight and slowness. Still, the idea is not to abandon my budget: it will evolve. So we will have a new budget to be used for the next iteration, for the next evolution.
Indeed, the performance budget was useful here. It allowed to question the value of what we were doing, to optimize both the new and the existing content. The framework has also allowed to communicate, to exchange about the “why” of a feature and its impacts on performance. To make sure that the need for speed was getting the attention it deserves.
How do you choose a budget?
“If you can’t measure it, you can’t improve it”. It will be necessary to monitor your website speed and to compare the results against your performance budgets. Give attention to the choice of indicators. I refer you to Jean-Pierre’s conference on the subject, in particular the User Timing allowing to get performance metrics as close as possible to your business stakes.
Beyond that: not all pages are the same. You probably don’t want to have the same performance budget as Netflix or Facebook. You are part of an identified market, with known competitors. Therefore you have particularities to take into account when setting your budgets. Another important point: not all your pages are the same. Keep in mind that some pages have more important stakes than others. Focus your efforts where there is business opportunities. Having a ROI based approach is ensuring its sustainability.
The first method I like to use to set a budget is to take a look at the competition.
You can benchmark your competitors with synthetic monitoring, or use some data source like Chrome UX Report (that Rick will detail in the next presentation). If you’re not so good compared to most of your competitors, you will try to catch up.
Conversely, if you are already among the best ones, your goal may be to create a real competitive advantage over speed. There are real success stories built around speed and a better UX.
Another way to define a budget can be through experimentation, Stéphane mentioned this earlier. Measuring how much it costs to slow down and what are the critical thresholds. More generally, it is a good way to keep in mind and objectify the stakes of performance, thus relying on data specific to your own business and context.
If your objective is to improve the performance of your existing website to get a “wow” effect from your visitors or project stakeholders, aim for something significantly faster. Indeed, the 20% rule indicates that users will only perceive a change that is more than 20% faster. Slighter can also affects your KPIs and your business, but a user won’t probably be noticing the improvement, the change will not be perceived. Aim for a release with a significant acceleration to spontaneously obtain positive feedback.
Stéphanie talked about performance through the prism of UX, it is a good way to set some performance budgets based on the main thresholds related to user perception.
My last approach to built budgets would be to remind you that some of them are imposed. I am thinking, for example, of Google’s crawl budget. One of the main concerns of SEO professionals is the time Google gives to a website to explore its web pages. The more pages you have, the more critical the crawl budget can be. If Google’s bots fail to visit all your pages within this time limit, you may see some of your pages indexed late (or slow updates of content).
To prevent this, you would need to reduce your TTFB and thus accelerate the exploration of your pages by bots. It would result to define a performance budget on your TTFB extrapolated from the Google crawl budget.
Another example of budget related to an external constraints: the 14kB of data we can use to optimize the critical rendering path. If you want to have a very fast first display of your page, you will need to make sure that the requiered data can be transmitted on a single network round trip.
Over TCP, because of the “slow start”, you can only benefit to do so from a window of about 14kB, no more.
Performance Culture: gain support for the budgets
Budgets are not only a safeguard, but also a useful communication tool. I quickly illustrated this with the example of the decision-maker who expresses a need for speed for a website, which is then adapted to other stakeholders during different stages of the project.
through
It is a discussion framework that will also allow us to facilitate trade-offs, as we mentioned with our example of adding a carousel that would make us exceed the budget.
To get budgets working properly, It is necessary that the tool gets the support of the project stakeholders. It is therefore needed to work collectively rather than trying to impose them. Go step by step, don’t try to set too many thresholds on too many indicators. It is important to keep going with something that will be operable, on which teams can react quickly.
Some of the bolder examples of the adoption of budgets among our clients: taking into account web performance objectives as a criterion for team incentives or individual bonuses.
To conclude, the adoption of performance budgets in your projects will allow you to put performance back where it belongs, as something that matters. You’ll make sure to optimize new features, and will make sure to regularly clean up the existing content.
This method will allow you to better communicate and open the discussion on the trade-offs between the value of the considered evolutions and their impacts on the loading speed of the website. And hopefully you will so participate to stopping the trend of an obese web.