BuildStream metrics: exploration

Metrics and telemetry are fundamental in any engineering activity to evaluate, learn and improve. They are also needed to consolidate a culture in which opinion and experience are continuously challenged, in which experimentation and evidence becomes the norm and not the exception, in which transparency rules so co-workers are empowered, in which data analysis leads to conversations so evaluations are shared.

Open Source projects has been traditionally reluctant to promote telemetry, based on privacy concerns. Some factor that comes to my mind are helping to change this perception:

  • As FLOSS projects grow and mature the need for information grows.
  • It is easier now to process big amounts of data while keeping high levels of anonymity.
  • The proliferation of company driven and consortium driven FLOSS projects, specially those related with SaaS/cloud technologies and products, showing how useful telemetry is. In general, corporations are less concerned about personal data privacy than many Open Source projects though.
  • The DevOps movement is spreading like a pandemic and telemetry is an essential action for practitioners.

So the last few years data analytics is becoming more popular among Open Source projects.

Finding the right metrics is frequently tough. Most of the times projects, teams or departments get drowned in data and graphs before they realize what actually matters, what does it have real business value. When you find the right metrics, somehow it means that the right questions are being asked which I find the hardest part. To identify those questions, I recommend organizations or projects to invest in exploring and learning before moving into automating the data collection, processing, plotting and irradiate the results to be analysed.

So when BuildStream is getting into its third year of life, I thought it could be interesting to invest some effort in digging into some numbers, trying to find a couple of good questions that provide value to the project and the stakeholders involved.buildstream-beaver

The outcome of this exploratory effort was published and spread across the BuildStream / BuildGrid community. The steps taken to publish the report has been:

  • Select a question to drive this exploratory effort, in my case: are we growing?
  • Select data sources: in my case, information from the ticketing system and the git repositories.
  • Collect the data: in this case, the data sets from the BuildStream ticketing system were exported from gitlab.com and the data sets from git obtained through a script developed by Valentin David.
  • Clean the data set (data integrity, duplications, etc.) in this case the data was imported into GSheets and worked there.
  • Data processing: the data was processed and metrics were defined using GSheet since the calculations in this phase were simple enough and the amount of data and processing power did not represent a challenge for the tool.
  • Plot the data: since the graphs were also simple enough, GSheet was also used for this purpose.
  • Initial analysis: the goal here was to identify trends, singularities, exceptions, etc and point them to the BuildStream community looking for debate and answers.
  • Report: provided in .pdf and .odt, it has been publishing in the BuildStream Group in gitlab.com and sent to the community mailing list. The report include several recommendations.

The data set could lead us to a deeper analysis but:

  • It would have also take me more time.
  • I wanted to involve the contributors and stakeholders early in the analysis phase.
  • Some metrics which collection, processing and plotting can be automated has been identified already so to me it is better to consolidate them to bring value to the project on regular basis than to keep exploring.

I understand that my approach is arguable but it has worked for me in the past.  The debate of just half way cooked analysis increases the buy-in in the same way that developers love to put their hands in half-broken tools. Feel free to suggest a better approach that I can try in the coming future. I would appreciate it.

Link to the report on gitlab.com. Download it.

What’s next?

I am looking forward to have a fruitful debate about the report within the BuildStream community and beyond. From there, my recommendation is to look for an external provider (it is all about providing value as fast as possible) that, working with Open Source tools, can consolidate what we’ve learnt from this process and can help us to find more and better questions… and hopefully answers.

What is BuildStream

I have been putting effort on BuildStream since May 2018. Check the project out.

Back from Akademy 2018. What an event!

A few days back I wrote about Akademy 2018 and my expectations and duties at the event. This is a report based on that previous post.

Akademy 2018 in Vienna, Austria, has been the best one in I’ve attended to in recent years. We event got a record number of sponsors, Codethink among them, and some of them for the very first time, which is always a good sign. The organization went smooth, the location was comfortable and several talks were very good. As a point to improve, I would mention the location of the Sunday party (sorry, social event), which was small for so many people.

Codethink as sponsor

This Akademy there were more first timers than in previous editions. Most of them knew nothing about Codethink. So being a sponsor allowed me to talk for 3-4 minutes to the audience, right before the closure, about what Codethink does, specially about the FOSS projects freedesktop-sdk and BuildStream. I will write more about them in the near future.

Representing Codethink also allowed me to attend to the sponsors dinner. I have attended to several of them before, either as a KDE eV Board Member, as the local organiser or as a sponsor representative (SUSE and now Codethink). This one worked very well. I was sit by the openSUSE Leap release manager Ludwig Nussel, former colleague of mine, Chris Lamb, Debian Lead and Alan Pope from Canonical, which allowed us to have interesting conversations about topics like reproducible builds, telemetry in distros, automotive systems, upstream events, etc. I also had the opportunity to introduce BuildStream to them.

Flatpak – Snap BoF

As part of my duties at the event I participated in the Flatpak – Snap workshop to introduce briefly the work Codethink is sponsoring on freedesktop-sdk and BuildStream:

  • freedesktop SDK 1.8 was just released before Akademy. flatpak-gnome and kde runtimes are based on it.
  • freedesktop SDK uses BuildStream as build tool. The integration team at GNOME too. So it was about time to talk about the project among KDE people. When it comes to build tools, the key selling point for BuildStream within KDE community is the advantage of creating a single set of definitions (recipes) and get different outputs like flatpaks, rpm-deb packages and hopefully in the future, snaps. Outputs depend on plugins that can be developed and maintained by contributors to BuildStream if they are not already available.
  • I am happy with the response BuildStream got and hopefully after the coming v1.2 release the project will support some KDE contributors in trying out the tool. It will be interesting also to support the KDE community in building the flatpak-kde runtime.

BuildStream

I introduced BuildStream to several developers. In general, very few knew anything about it which is not surprising since, except among the GNOME community, we have barely made any noise about the project. The coming release will help us to start mitigating this issue. The KDE community produces flatpaks, snaps, .deb and .rpm packages, has Yocto recipes… . A tool set that provides you all these outputs by maintaining a single sets of definitions, being simple to install, update and maintain, becomes attractive at first sight. The challenge is major, but a nice one to face.

Third edition of the Freedesktop Summit?

Several years ago, driven by Alison Lortie and sponsored by openSUSE, I organised the first freedesktop event ever. The following year I participated in the organization of the second one, that also took place in Nürnberg, Germany. During Akademy I talked to some people that might be interested in participating in a new edition. I did have some conversations around this topic at GUADEC too so, although it is still nothing but an idea, I will invest a few cycle to mature it. Who knows, maybe we can organise a third edition of the event next year.

KDE for automotive

As you know, I have been advocating at KDE for putting effort to showcase our software in automotive R&D and pre-production environments, knowing that Qt is the default graphic toolkit in the industry. Last year I provided a lightning talk about it which, together with the improvements in Plasma Mobile, triggered the interest of some developers at KDE, which are professionally involved in Qt companies working in embedded/automotive already. We had a BoF back then and we created a roadmap to shorten the gap between what we had and what would be interesting to show in such environments.

Andreas Cord-Landwehr and Volker Krause has managed to create two Yocto recipes together with the infrastructure to update them regularly. Both had a talk at Akademy. Andreas introduced the audience to Yocto and talked about the created recipes while Volker showed during a lightning talk the plasma mobile shell working on a RPi3 with the Raspberry 7″ touchscreen.

At the BoF we evaluated the technical, coordination and promo actions taken as well as the coming ones. The ultimate goal for the coming year is to showcase at different events KDE apps on top of Plasma mobile on automotive dev as well as to increase the attention of more KDE developers on embedded as a potential target market for KDE software.

I am excited about this progress. I believe putting some effort to move towards embedded can bring more attention and new energy to KDE.

KDE E.V. AGM

I participated at the annual KDE eV AGM (general assembly) on Monday August 13th. As you know the KDE eV AGM is a close doors one, so private. It was a good move to provide room during the event for the working groups to summarise the work they have done. These reports used to be done during the assembly. As consequence, the AGM was shorter.

There used to be a time in which the KDE eV AGM was long and tedious. Over the years we have put effort in making it shorter and more dynamic. Mission accomplished.

openSUSE

It was great to meet all timers but also new openSUSE contributors. I could take some time to talk about points that require some attention in the distro, learn about the improvements on the delivery process that has been implemented on Leap 15, some of the actions in progress and future plans. Finally I did not update the Leap version in my working laptop (a Slimbook) because I had more work than expected during the event. I will update it during the coming weeks. Overall I am happy with Leap 15 in my personal laptop (a Lenovo) so far.

Slimbook

I was happy to see Slimbook booth at the event. As you know I am a Slimbook happy customer. I had the chance to talk to other customers at the event and I would highlight how happy we are with the post-sale service they provide.

Did I mention I would like to have a 11″ SlimNoteBook. I definitely did to them.

Free Qt Foundation

I cannot get tired of telling everybody within KDE and specially outside the community how important is the Free Qt Foundation, not just for KDE and the Qt ecosystem, but also for the entire Free Software movement. I never loose the chance when I am among automotive professionals to highlight the enormous impact that this entity has on their businesses. There is still a lot to do to provide The Free Qt Foundation the attention it deserves. By the way, it is always a pleasure for me to talk to KDE e.V. representatives there, Martin Konold and Olaf Schmidt-Wischhöfer.

What a great contribution they do.

Other topics

Did I mention how cool it is to have Mycroft integrated with KDE? All the demos I have seen are so promising… openSUSE Tumbleweed has packages to enjoy Mycroft on Plasma. The same applies to having KDE software on phones. It seems that we will have Qt6 in 2020 and that the transition from Qt5 is planned to be smooth. Breaking things when updating is never a good strategy in my opinion.

I love KDE Connect and I had the chance to learn about some coming new features, which is always a plus for coming to Akademy.

Sebas and Valorie got awards this year. Well deserved, as the rest of the winners. I like the fact that we as community, put some effort in recognising those who make significant contributions to the project.

I could have chats with people that I respect, new developers, key contributors… . As I said, a great event.

Thanks to the organisers

I will finish this Akademy 2018 report thanking the organization team who worked hard to make Akademy 2018 one of those hard to forget.

Thank you.