Journal 3, About making an application faster with real user monitoring

This journal entry is about

a recap on how I used real user monitoring to make a customer's web application faster,
a summary of stuff I did within my business,
a report of the new Boostrappers meetup in Graz,
how business is going in general
and what’s next in the coming weeks.

Performance tuning with real user monitoring

In December I was commissioned to improve the performance of a business web application. Before I go into details, I’d like to give a bit of background.

Background

I’m maintaining the application for three years now, and I already worked on its performance in the past.

So far, I accomplished the most significant improvement by fixing an HTTPS issue. A self-signed certificate caused Chrome to cache no resources at all. As a result, pages took up to fifteen seconds to load.

This slowness is all the more a problem as the application is covering almost everything in the company – from time tracking to reporting and from composing offers to creating invoices. Anyhow, by fixing the certificate, page load times went down by about ten seconds.

Rolling my own little real user monitoring

This certificate issue showed me one thing: While I was aware that performance differs between development and production due to different factors like hardware, I wasn’t expecting the difference being so drastic. At this moment it became clear to me that I had to measure performance in the field.

I quickly learned that there’s a term for what I was trying: real user monitoring (RUM).

The cool thing is that modern browsers already provide native APIs for that. The most important ones are Navigation Timing API and Resource Timing API. The first one measures the metrics of the main HTML document. The second one measures the speed of the document's sub-resources such as CSS, JavaScript and image files.

Collecting timing data at the client

Based on this tutorial, writing the data capturing script was a breeze. In my case, it boiled down to ~20 lines of JavaScript. It was good enough for the job at hand. For a more advanced scenario, one might wanna use boomerang.

Navigator.sendBeacon is the preferred way to transmit metrics. It's an API that allows sending something to the back-end without blocking the unloading of a page, which in turn would slow down the loading of the next page.

One gotcha in practice is the limit of the sendBeacon buffer size. In Chrome it appears to be 64KB according to this blog post. Therefore boomerang’s compression feature would be of particular interest. My experiments showed that Chrome omits to send the request if the payload is larger than 64KB.

Back-end and analysis

Anyhow, I quickly cobbled together the back-end code and a node.js script to do the analysis. For the analysis part, I thought about taking the opportunity to learn a bit of Python and pandas. However, I already spend slightly too much time on getting to know the APIs mentioned above.

With my self-made RUM in place, I went on to measure some real-world performance data. How does that data look? In essence, it’s all data you would need to render a waterfall diagram developers know from browser dev tools.

Analyzing what was slowing the app down

My analysis showed that TTFB (time to first byte) was quite acceptable overall. That meant back-end code and database queries for most pages were in good shape. Still, simple pages like the login and time tracking forms were experiencing page loads times greater than two seconds.

Some might think that this is already a good enough performance. Indeed, the total waiting time isn’t that much. The real problem is that employees lose focus on their current task. According to this article from Google “beyond 1000 milliseconds (1 second), users lose focus on the task they are performing.”.

So where was the time spent?

Although I had thought about it before already, the data clearly showed this: I had to bite the bullet and produce a custom production build of an ancient DojoToolkit version. It was a “learning experience” by itself *sigh*.

Nevertheless, I was able to cut the loading time again. Simple pages, such as the aforementioned time-tracking form, load almost instantly, so users are less disrupted in their tasks. However, the best thing for me personally is that I can now measure those improvements with my new self-made RUM.

Real user monitoring tools

I’d also like to mention some products and tools I stumbled upon during my research:

Speedcurve, a commercial tool that seems to be the market leader
mPulse, similar to Speedcurce, now owned by Akamai
Dareboost, not exclusively a tool for web performance
sitespeed.io, a set of open source tools as an alternative to commercial ones
basicrum, an open source RUM solution

The developer of basicrum, Tsvetan Stoychev, has written a nice article on the topic.

Stuff I did in my business

As I hoped in my journal entry last December, I was able to tackle some long-delayed tasks within my business.

Here’s a list of the most notable ones:

Gradually streamlining scalingcurve.at – I fixed some issues reported by the Google mobile usability report, added new customer logos, changed some copy and so on. It’s already a slightly more consistent user experience, but I’m still in the middle of the process.
Publishing Kommis case study – it’s the most notable update to my website, and I’m pleased I pushed through writing it.
teamvergleich.de update – I adapted some scraping and API consumption code. First and foremost I needed to migrate to version 2 of the football-data.org API. When I initially built this site, football-data.org was a free service. They offer paid plans now. It’s nice to see people can grow a business around an API.
Creating a favicon for this very website – by the way, if you think implementing a favicon in 2019 is easy, I recommend reading Favicons, Touch Icons, Tile Icons, and so forth. Which Do You Need?.
Upgrading to Ubuntu 18.04 – some notes here

New Bootstrappers meetup in Graz

Speaking of products, I visited the first two events of the new Micropreneurs, Bootstrappers and Self-Funded Startups Graz meetup.

It's aimed at startup founders that don’t seek outside investment. As such, it targets a slightly different community than the Startup Spritzer events. My impression so far is that most attendees don’t even know what terms like bootstrapping and self-funding mean. So there’s some room for education.

How is business going?

In my journal entries, I’d also like to include an update on my business itself.

So how is it going?

In the last months, I was busy with smaller jobs for existing customers. These things add up, and I’m grateful to have long-term client relationships.

Next week I’m going to start a mid-sized project. I’ll have to add some features in an existing web application and furthermore carry out some improvements on the UX side of things.

As far as data visualization is concerned, there are two new opportunities on the horizon. One would be building a dashboard for a web-based software product. The other one would be dashboard design and implementation in the realm of accounting and controlling (BMD).

Other than that, I’ve also been asked to become part of a longer-term project in the software product space. However, it's too early to give any further details.

What’s next in coming weeks

The upcoming weeks will be stuffed with the already mentioned client project. After that, I’d like to continue improving my business website. Moreover, I almost forgot to mention that I'll be supervising a bachelor thesis on "Accessibility in state-of-the-art Single Page Applications" in the coming months.