Journal 2, Farewell Plotly.js!

This journal entry is about

me looking back at working on Plotly.js,
what’s next in the coming weeks
and a quick mention of the FTTH initiative in Weiz.

Looking back at working on Plotly.js

My time working on Plotly.js is over, a bit unexpected, however. While I don’t want to talk too much about the reasons, let me note this: as an external part of a development team, there’s always the chance the client finds a more economic replacement. It’s not that it took me by sheer surprise – it’s a quite usual business move I already saw in the past elsewhere.

This may sound like I’m bitter. But quite the opposite is true. I only mention it as a warning sign for myself that especially a “staff augmentation” position isn’t something you should rely on too much.

Enough of that. Overall I’m really grateful for the experience to serve a client well known in the data visualization community (and beyond) and for being able to do that a 100% remote for a company in Canada. On top of that I’m proud to be now named as one of the official creators of Plotly.js.

So this appears to be the perfect time to look back at

how it all began,
what went well and
what went not so well.

How it all began

In late 2017 Rob Monfera (Hi Rob!) shot me a message that his current client at that time was looking for someone to help with a rather unusual task. Plotly was looking for someone to lint an internal code base – a process I like to call code gardening. For a lot of developers this may sound as a low prestige job – and it probably is. But for me it was an opportunity to get another job in data visualization and apart from that I like cleaning up code. Anyhow, I got assigned to lint another JavaScript project after that and in early 2018 I went on-board to help develop the core visualization library most products of the company are based upon.

First, I had some doubts that I’ll be a good enough library developer – as opposed to application development which is the stuff I usually do. But in the end all went well and I’ve certainly grown my skills. At times we need to step out of our comfort zone if we want to improve, right?

What went well

Not wasting the core maintainers’ precious time – I really tried hard to figure things out for myself instead of bothering the core maintainers with beginner questions. There are certainly situations where it makes sense to consult a core dev, but usually it is very well received if you don’t pick their brains too often. The unsolicited feedback “You know how and when to ask the right questions.” was underlining that.
Testing first – I’m not a person proclaiming “always test first”. I’d even go so far to say that you shouldn’t blindly aim for 100% test coverage at all times. Why? Because in my opinion test coverage is a vanity metric. What is 100% test coverage anyway? Anyhow, here it was interesting to see how I slowly progressed from writing tests after to writing tests first. I found the latter tests were easier to write and on top of that gave me a feeling of safety and being on the right track early on. But I’ve some deeper, likely critical thoughts on the safety net aspect, which might be a topic for another post. Only so much, the guardrail programming approach, Rich Hickey called out in Simple Made Easy, reminded me a bit of what I was doing here in this respect.
Sticking to the existing coding style – I hardly got complaints in my PRs about the coding style. I sure would have preferred a different style here and there, but in the end the quality of the code base matters much more than the taste of a single programmer. My effort to stick to the existing coding style was very well received.
Using existing facilities in the code – related to the last point. Simply watch out for and use existing stuff.
Trying to not drop the balls – I think this is of particular importance when maintaining a library where backwards-compatibility matters a lot. To not drop balls in this context means that when developing a new feature, existing functionality (a lot in case of Plotly.js) has to be kept intact. I managed to not drop too much balls by (i) heavily using TODO comments in the code, (ii) keeping an extra list for broader concerns and (iii) making a habit out of noting things down immediately when they came to mind. The book Checklist Manifesto might me worth a look in this regard.

What went not so well

Performance of first click-to-select attempt – In the last journal entry I talked about the click-to-select feature. It certainly was not a simple one. After all it had to integrate with the existing polygon-selection behavior and it had to be supported by all chart types. My first attempt was built around extending the interface that chart modules have to implement to support selection. While the code got cleaner and easier to understand this way, it had some serious (structural) performance issues. Anyhow, it was a first working solution. As a result I had existing tests at my disposal and I gained a deeper understanding of the relevant code parts. That allowed me to come up with a simpler solution that didn’t change the chart module selection interface and instead used the strategy pattern (sort of) to leverage the existing API. I’m not sure if I could have seen the performance problems coming beforehand. I guess in our profession we sometimes need to try stuff out to know things for sure.
Lack of knowledge about some browser inconsistencies – Particularly, I’m speaking about getBoundingClientRect() which at first sight seems to be a wonderful tool to test the position of different SVG elements of a chart. When getting the bounding box of SVG text (or transformed SVG elements), please don’t rely on getBoundingClientRect. It’s inconsistent across browsers and for text the real bounding box also heavily depends on the definite font in use.
Lack of typography knowledge – related to the above, this was very relevant for my last contribution chart title alignment. Users want to be able to align chart titles and labels accurately. For example, chart title should be aligned with the left edge and be positioned 3 pixels above the plotting area. With text-anchor, SVG gives us a wonderful tool for horizontal text alignment. It’s a different, more difficult story for vertical alignment. There are attributes available, but browsers treat dominant-baseline et al. inconsistently. What most people usually end up doing is to leverage dy to shift text along the y-axis using fixed values in em units (which is relative to the text’s font size). And these fixed values are approximations at best that are nearly correct for average fonts. To be precise, you use dy=”.7em” to have the cap line of your text at the text’s y coordinate and dy=”.35em” to have the midline (i.e. half the cap height) sitting at y.

Besides some smaller stuff and bug fixes, my biggest features were fixed-size shapes, click-to-select and chart title alignment. I hope I can use these contributions to land some other gigs in the realm of data visualization development. It’s just so that I’ve not yet really got the hang of how the dataviz freelance market really works. I’d love to do more in this space, but I guess I’m dependent on referrals which are not predictable.

A biased look at the future of Plotly.js as a charting library

With respect to Plotly.js, I see a positive future for the library. Of course, D3.js revolutionized the dataviz library landscape. Plotly.js is even built on top of it. But d3.js operates on a different level because you build everything your own and only your imagination is the limit. That’s a blessing and a curse. For business dashboards and reports, you’ll often want something out-of-the-box. Plotly.js can definitely deliver on this.

Another question is if Plotly.js works well with Virtual DOM based frameworks like React. By taking the control on the DOM away from the charting library, they pose a real challenge. Despite Plotly.js is older than those front-end frameworks, it delivers on that as well. Why? Because Plotly’s new chart editor product is written in React and thus Plotly.js has been adapted to work well with it. For sure there are React-native charting libs. But a lot of them are a byproduct of some Silicon Valley companies (e.g. Uber) where data visualization is not the core business as it is the case with Plotly.

Other stuff the last months

I’d like to mention our districts (Weiz) effort to have FTTH (fiber to the home) internet for every household. The website of the initiative is g31.at. Not very informative at the moment I must say, but the people driving this project are admirable. I wish more politicians in higher institutions would have such foresight and perseverance.

Teaching dynamic web went fine so far. It’s almost finished. Surprisingly little has changed over three years in the platform and libraries in use, namely node.js, express.js and handlebars.js. Not everything in JavaScript land is crazy fast changing.

What’s next? Repositioning?

The next weeks and months will be dominated by filling my pipeline with work again. Lately I was questioning if my current positioning is still the best in my present situation. After three years in business, I now know much better which type of clients I enjoy working with and what kind of projects I like the most. I’m still interested in doing data visualization work. But as I said above, I very much depend on referrals in this space. Because of that, I’m thinking of presenting dataviz development as only one of many services of a broader web development positioning.

With respect to that, I’m even thinking about positioning away from web development, which describes the what, to non-IT centric SMEs, which describes the who. Technically I would still develop and maintain business web applications. But by focusing on communicating whom I serve instead of what I do technically, potential buyers would be able to resonate with my offerings much easier.

In addition to that, I hope to take care of long-delayed administrative tasks postponed in busier times for far too long.