BuildStream metrics: exploration

Metrics and telemetry are fundamental in any engineering activity to evaluate, learn and improve. They are also needed to consolidate a culture in which opinion and experience are continuously challenged, in which experimentation and evidence becomes the norm and not the exception, in which transparency rules so co-workers are empowered, in which data analysis leads to conversations so evaluations are shared.

Open Source projects has been traditionally reluctant to promote telemetry, based on privacy concerns. Some factor that comes to my mind are helping to change this perception:

  • As FLOSS projects grow and mature the need for information grows.
  • It is easier now to process big amounts of data while keeping high levels of anonymity.
  • The proliferation of company driven and consortium driven FLOSS projects, specially those related with SaaS/cloud technologies and products, showing how useful telemetry is. In general, corporations are less concerned about personal data privacy than many Open Source projects though.
  • The DevOps movement is spreading like a pandemic and telemetry is an essential action for practitioners.

So the last few years data analytics is becoming more popular among Open Source projects.

Finding the right metrics is frequently tough. Most of the times projects, teams or departments get drowned in data and graphs before they realize what actually matters, what does it have real business value. When you find the right metrics, somehow it means that the right questions are being asked which I find the hardest part. To identify those questions, I recommend organizations or projects to invest in exploring and learning before moving into automating the data collection, processing, plotting and irradiate the results to be analysed.

So when BuildStream is getting into its third year of life, I thought it could be interesting to invest some effort in digging into some numbers, trying to find a couple of good questions that provide value to the project and the stakeholders involved.buildstream-beaver

The outcome of this exploratory effort was published and spread across the BuildStream / BuildGrid community. The steps taken to publish the report has been:

  • Select a question to drive this exploratory effort, in my case: are we growing?
  • Select data sources: in my case, information from the ticketing system and the git repositories.
  • Collect the data: in this case, the data sets from the BuildStream ticketing system were exported from gitlab.com and the data sets from git obtained through a script developed by Valentin David.
  • Clean the data set (data integrity, duplications, etc.) in this case the data was imported into GSheets and worked there.
  • Data processing: the data was processed and metrics were defined using GSheet since the calculations in this phase were simple enough and the amount of data and processing power did not represent a challenge for the tool.
  • Plot the data: since the graphs were also simple enough, GSheet was also used for this purpose.
  • Initial analysis: the goal here was to identify trends, singularities, exceptions, etc and point them to the BuildStream community looking for debate and answers.
  • Report: provided in .pdf and .odt, it has been publishing in the BuildStream Group in gitlab.com and sent to the community mailing list. The report include several recommendations.

The data set could lead us to a deeper analysis but:

  • It would have also take me more time.
  • I wanted to involve the contributors and stakeholders early in the analysis phase.
  • Some metrics which collection, processing and plotting can be automated has been identified already so to me it is better to consolidate them to bring value to the project on regular basis than to keep exploring.

I understand that my approach is arguable but it has worked for me in the past.  The debate of just half way cooked analysis increases the buy-in in the same way that developers love to put their hands in half-broken tools. Feel free to suggest a better approach that I can try in the coming future. I would appreciate it.

Link to the report on gitlab.com. Download it.

What’s next?

I am looking forward to have a fruitful debate about the report within the BuildStream community and beyond. From there, my recommendation is to look for an external provider (it is all about providing value as fast as possible) that, working with Open Source tools, can consolidate what we’ve learnt from this process and can help us to find more and better questions… and hopefully answers.

What is BuildStream

I have been putting effort on BuildStream since May 2018. Check the project out.

Stable: not moving vs. not breaking

There are two terms that brings a heavy controversy in the Open Source world: support and stable. Both of them have their roots in the “old days” of Open Source, where its commercial impact was low and very few companies made business with it.

You probably have read a lot about maintenance vs support. This controversy is older. I first heard of it in the context of Linux based distributions. Commercial distribution had to put effort in differentiating among the two because in Open SOurce they were used indistictly but not in business. But this post is about the adjectivet stable

Stable as adjective has several meanings in English:

  • According to the Cambridge dictionary stable is, among others: firmly fixed or not likely to move or change. I am used to this meaning because of my backgroun in Physics.
  • One of the definitions provided by the Oxford dictionary is slightly different: not likely to change or fail; firmly established.

Can you see the confusion between…?

  1. It is hard to move… or move slow.
  2. It does not fail… it is hard to break.

I am not an English native speaker. Maybe because of its latin root (I am Spanish) or maybe eacuase I studied Physics, I do not provide to the adjective stable the second meaning. It has always called my attention how many people provide both meanings to the word, specially when referring to software releases. Looking at the dictionaries, I can tell why in English there is such link between both ideas.

I read today on Twitter another example of this confusion/controversy. The comment has its roots on a different topic but it has embedded this confusion, I think.

Open Source projects used to have as the main testing strategy the collaboration with a group of beta testers and power users. This strategy helped to create this relation between “not moving” and “not failing” that has become so popular. But this relation is not direct if you have a different release/deploy strategy. The way to increase stability (as hard to break) is by detecting and fixing bugs and that can be done moving fast (constantly updating), or moving slow (backporting). I will not get into which one is better. Both are possible and popular.

I think the word stable should be avoided. Many system and application developers or integrators refer to it as “move slow” as “it will not change much“. But what many users hear is “it will not break” which is why “it moves slow“.

Which adjective can we use to label a release that “moves slow“, so it is not mistaken with “it is not expected to break“?

Is this controversy present in other languages as strong as I perceive it in English? Is it just me who see this controversy?

My new gravatar

More and more applications invite you to add a pic in your profile, a gravatar. In some applications and specially in social media the size and position of these gravatars are becoming prominent, too much for my taste. I am not a fan of pictures, I have never been. I do not like selfies, I have no Instagram, I try to avoid publishing pics of myself on Facebook… .

I use two different gravatars in internet. One for professional profiles and applications, associated to my corporate account ( agustin.benito@codethink.co.uk nowadays ), which was taken about 4 years ago, and one I took over 10 years ago associated to my personal account. I have hair on that one.

A few months back I started to think about my approach to this gravatars, looking for something simpler that could last, reflect my personality and that could be used for both, personal and professional profiles… and substitute the pic I have in my site.

My first idea was to ask a couple of photographers I know to take a good picture of me. One of those super modern, super cool, super fancy I-enjoy-life-you-should-ping-me-and-I-will-teach-you-how-but-only-if-you-are-cool-enough-to-deserve-my-coolness kind of picture. I was not fully convinced on this approach though. It would only work if the pic is really cool and checking LinkedIn, I had little confidence my pic could be at the level of some of my connections. Yep, I am connected to very cool people.

Then, looking at some gravatars from hackers I know who have zero interest in publishing a pic of themselves, I thought that maybe an illustration could work. Something simple, not a portrait. An illustration that looked like me but is not me, that could work in small and medium size formats… .

Exactly, I did not know what I want. It was more like… I do know a few things I do not want.

With this “crystal clear” idea, I contacted Ramón Miranda.

Ramón is a professional illustrator that has contributed to several Open Source projects, like Krita. He lives only a few kilometres away from my place in Málaga, Spain which is… great.

I explained him in a mail my idea. We have a video chat to polish the concept and a few days later he invited me over his place to show me a draft. A few days later my new representation of myself was done… with Open Source tools (Krita). And yes, I have the source file.

toscalix avatar

Thank you Ramón for accepting the challenge and for your dedication. I like the result… a lot.

Dear reader, do you? Yep, I will erase you comment if you don´t.

If you ever need a professional illustrator consider pinging Ramón. He is a hell of a professional. Ah, and he provides online training sessions for those wanting to learn or improve how to draw/paint in the analog and in the digital world (with Open Source tools).

Time to substitute my pics with this new gravatar.

A summary of my participation at OSS EU / ELCE 2018

ELCE 2018 has been a key event each of my 3 years at Codethink. It is an interesting conference from the technical and business point of view. This year the event was collocated again with several others, like the Open Source Summit Europe (OSS EU). It took place in Edinburgh, UK, I city I find particularly beautiful and enjoyable.

CIP, the Linux Foundation Initiative for Civil Infrastructure, has the event as a key milestone. The group organises a booth and a face to face meeting of the Technical Steering Committee (TSC) the day before the event starts. So for me ELCE is a day longer. The other main task for me at this event is supporting Codethink business development and community engagement actions, since I represent the company at the Linux Foundation.

During the CIP TSC face to face meeting, I did a short presentation of one of the projects I am putting effort on lately: BuildStream. I also described the project among several people I know during the event, asking them to try out the integration tool and provide me feedback. The tool has matured quite a lot the last few months and I am interested in collecting feedback from experienced developers and software integrators.

Lukas Bulwahn is leading an interesting effort to create a safety Linux related initiative at the Linux Foundation, including a Safety Critical Systems track at the event. I attended the last day of the conference to these talks. My colleagues Ben Dooks, senior kernel developer and Codehtink’s CEO Paul Sherwood, presented there a couple of topics we have been working on lately. I found the track interesting and learnt a few things. I hope such track consolidates and more automotive companies participate in the future.

Codethink sponsored the event and had a booth to recruit new developers. I backed up my colleague Tim a couple of times during the first two days of the event which allowed me to talk to a couple of of potential hires. It is always interesting to find out what motivates others about the company your are working for.

During the second day of the event I took some time to visit some booths and learn about what other companies are doing, as well as attending to a couple of talks. I also had several interesting meetings and conversations with old friends and former colleagues, visited again the Edinburgh Castle, this time at night, and managed to try a few whiskies from the Highlands. Not everything was to be work, right?

kernelci.org is an initiative I feel attached to since it was born during my time as Director at Linaro, promoted by engineers from my department, specially Kevin Hilman. Time has demonstrated that the project has helped the Linux Kernel tremendously. I hope kernelci.org get a successful second life under the umbrella of the Linux Foundation.

At Akademy 2018 a group of KDE people agreed to put effort in showcasing KDE software at embedded events to demonstrate that Plasma Mobile is a credible option as HMI for automotive R&D environments. At ELCE 2018, we showed Plasma Mobile and a couple of applications on top of a YOCTO based system on a Raspberry Pi 3 with a 7″ touchscreen. Showing the same technologies across different form factors (a mobile, a Pinebook, a laptop with openSUSE and the mentioned RPi3) was impressive. Sadly my RPi3 worked only the first day because it burned. I guess I have taken it to too many events. RIP my dear….

I would like to highlight how excited I got when I saw the KDE booth at this event, the fantastic work that Adrian, Jonathan, Kenny and Paul did there and the significant impact KDE had. Getting out of the comfort zone is never easy, but usually worth it. I am looking forward to show further progress at FOSDEM and the Embedded World early in 2019.

Except surprises, this will be the last event of the year for me. It is time to prepare next year’s agenda. I will see you all at FOSDEM 2019. Thanks Codethink for sponsoring my participation at ELCE 2018.

Back from Akademy 2018. What an event!

A few days back I wrote about Akademy 2018 and my expectations and duties at the event. This is a report based on that previous post.

Akademy 2018 in Vienna, Austria, has been the best one in I’ve attended to in recent years. We event got a record number of sponsors, Codethink among them, and some of them for the very first time, which is always a good sign. The organization went smooth, the location was comfortable and several talks were very good. As a point to improve, I would mention the location of the Sunday party (sorry, social event), which was small for so many people.

Codethink as sponsor

This Akademy there were more first timers than in previous editions. Most of them knew nothing about Codethink. So being a sponsor allowed me to talk for 3-4 minutes to the audience, right before the closure, about what Codethink does, specially about the FOSS projects freedesktop-sdk and BuildStream. I will write more about them in the near future.

Representing Codethink also allowed me to attend to the sponsors dinner. I have attended to several of them before, either as a KDE eV Board Member, as the local organiser or as a sponsor representative (SUSE and now Codethink). This one worked very well. I was sit by the openSUSE Leap release manager Ludwig Nussel, former colleague of mine, Chris Lamb, Debian Lead and Alan Pope from Canonical, which allowed us to have interesting conversations about topics like reproducible builds, telemetry in distros, automotive systems, upstream events, etc. I also had the opportunity to introduce BuildStream to them.

Flatpak – Snap BoF

As part of my duties at the event I participated in the Flatpak – Snap workshop to introduce briefly the work Codethink is sponsoring on freedesktop-sdk and BuildStream:

  • freedesktop SDK 1.8 was just released before Akademy. flatpak-gnome and kde runtimes are based on it.
  • freedesktop SDK uses BuildStream as build tool. The integration team at GNOME too. So it was about time to talk about the project among KDE people. When it comes to build tools, the key selling point for BuildStream within KDE community is the advantage of creating a single set of definitions (recipes) and get different outputs like flatpaks, rpm-deb packages and hopefully in the future, snaps. Outputs depend on plugins that can be developed and maintained by contributors to BuildStream if they are not already available.
  • I am happy with the response BuildStream got and hopefully after the coming v1.2 release the project will support some KDE contributors in trying out the tool. It will be interesting also to support the KDE community in building the flatpak-kde runtime.

BuildStream

I introduced BuildStream to several developers. In general, very few knew anything about it which is not surprising since, except among the GNOME community, we have barely made any noise about the project. The coming release will help us to start mitigating this issue. The KDE community produces flatpaks, snaps, .deb and .rpm packages, has Yocto recipes… . A tool set that provides you all these outputs by maintaining a single sets of definitions, being simple to install, update and maintain, becomes attractive at first sight. The challenge is major, but a nice one to face.

Third edition of the Freedesktop Summit?

Several years ago, driven by Alison Lortie and sponsored by openSUSE, I organised the first freedesktop event ever. The following year I participated in the organization of the second one, that also took place in Nürnberg, Germany. During Akademy I talked to some people that might be interested in participating in a new edition. I did have some conversations around this topic at GUADEC too so, although it is still nothing but an idea, I will invest a few cycle to mature it. Who knows, maybe we can organise a third edition of the event next year.

KDE for automotive

As you know, I have been advocating at KDE for putting effort to showcase our software in automotive R&D and pre-production environments, knowing that Qt is the default graphic toolkit in the industry. Last year I provided a lightning talk about it which, together with the improvements in Plasma Mobile, triggered the interest of some developers at KDE, which are professionally involved in Qt companies working in embedded/automotive already. We had a BoF back then and we created a roadmap to shorten the gap between what we had and what would be interesting to show in such environments.

Andreas Cord-Landwehr and Volker Krause has managed to create two Yocto recipes together with the infrastructure to update them regularly. Both had a talk at Akademy. Andreas introduced the audience to Yocto and talked about the created recipes while Volker showed during a lightning talk the plasma mobile shell working on a RPi3 with the Raspberry 7″ touchscreen.

At the BoF we evaluated the technical, coordination and promo actions taken as well as the coming ones. The ultimate goal for the coming year is to showcase at different events KDE apps on top of Plasma mobile on automotive dev as well as to increase the attention of more KDE developers on embedded as a potential target market for KDE software.

I am excited about this progress. I believe putting some effort to move towards embedded can bring more attention and new energy to KDE.

KDE E.V. AGM

I participated at the annual KDE eV AGM (general assembly) on Monday August 13th. As you know the KDE eV AGM is a close doors one, so private. It was a good move to provide room during the event for the working groups to summarise the work they have done. These reports used to be done during the assembly. As consequence, the AGM was shorter.

There used to be a time in which the KDE eV AGM was long and tedious. Over the years we have put effort in making it shorter and more dynamic. Mission accomplished.

openSUSE

It was great to meet all timers but also new openSUSE contributors. I could take some time to talk about points that require some attention in the distro, learn about the improvements on the delivery process that has been implemented on Leap 15, some of the actions in progress and future plans. Finally I did not update the Leap version in my working laptop (a Slimbook) because I had more work than expected during the event. I will update it during the coming weeks. Overall I am happy with Leap 15 in my personal laptop (a Lenovo) so far.

Slimbook

I was happy to see Slimbook booth at the event. As you know I am a Slimbook happy customer. I had the chance to talk to other customers at the event and I would highlight how happy we are with the post-sale service they provide.

Did I mention I would like to have a 11″ SlimNoteBook. I definitely did to them.

Free Qt Foundation

I cannot get tired of telling everybody within KDE and specially outside the community how important is the Free Qt Foundation, not just for KDE and the Qt ecosystem, but also for the entire Free Software movement. I never loose the chance when I am among automotive professionals to highlight the enormous impact that this entity has on their businesses. There is still a lot to do to provide The Free Qt Foundation the attention it deserves. By the way, it is always a pleasure for me to talk to KDE e.V. representatives there, Martin Konold and Olaf Schmidt-Wischhöfer.

What a great contribution they do.

Other topics

Did I mention how cool it is to have Mycroft integrated with KDE? All the demos I have seen are so promising… openSUSE Tumbleweed has packages to enjoy Mycroft on Plasma. The same applies to having KDE software on phones. It seems that we will have Qt6 in 2020 and that the transition from Qt5 is planned to be smooth. Breaking things when updating is never a good strategy in my opinion.

I love KDE Connect and I had the chance to learn about some coming new features, which is always a plus for coming to Akademy.

Sebas and Valorie got awards this year. Well deserved, as the rest of the winners. I like the fact that we as community, put some effort in recognising those who make significant contributions to the project.

I could have chats with people that I respect, new developers, key contributors… . As I said, a great event.

Thanks to the organisers

I will finish this Akademy 2018 report thanking the organization team who worked hard to make Akademy 2018 one of those hard to forget.

Thank you.