BuildStream metrics: exploration

Metrics and telemetry are fundamental in any engineering activity to evaluate, learn and improve. They are also needed to consolidate a culture in which opinion and experience are continuously challenged, in which experimentation and evidence becomes the norm and not the exception, in which transparency rules so co-workers are empowered, in which data analysis leads to conversations so evaluations are shared.

Open Source projects has been traditionally reluctant to promote telemetry, based on privacy concerns. Some factor that comes to my mind are helping to change this perception:

  • As FLOSS projects grow and mature the need for information grows.
  • It is easier now to process big amounts of data while keeping high levels of anonymity.
  • The proliferation of company driven and consortium driven FLOSS projects, specially those related with SaaS/cloud technologies and products, showing how useful telemetry is. In general, corporations are less concerned about personal data privacy than many Open Source projects though.
  • The DevOps movement is spreading like a pandemic and telemetry is an essential action for practitioners.

So the last few years data analytics is becoming more popular among Open Source projects.

Finding the right metrics is frequently tough. Most of the times projects, teams or departments get drowned in data and graphs before they realize what actually matters, what does it have real business value. When you find the right metrics, somehow it means that the right questions are being asked which I find the hardest part. To identify those questions, I recommend organizations or projects to invest in exploring and learning before moving into automating the data collection, processing, plotting and irradiate the results to be analysed.

So when BuildStream is getting into its third year of life, I thought it could be interesting to invest some effort in digging into some numbers, trying to find a couple of good questions that provide value to the project and the stakeholders involved.buildstream-beaver

The outcome of this exploratory effort was published and spread across the BuildStream / BuildGrid community. The steps taken to publish the report has been:

  • Select a question to drive this exploratory effort, in my case: are we growing?
  • Select data sources: in my case, information from the ticketing system and the git repositories.
  • Collect the data: in this case, the data sets from the BuildStream ticketing system were exported from gitlab.com and the data sets from git obtained through a script developed by Valentin David.
  • Clean the data set (data integrity, duplications, etc.) in this case the data was imported into GSheets and worked there.
  • Data processing: the data was processed and metrics were defined using GSheet since the calculations in this phase were simple enough and the amount of data and processing power did not represent a challenge for the tool.
  • Plot the data: since the graphs were also simple enough, GSheet was also used for this purpose.
  • Initial analysis: the goal here was to identify trends, singularities, exceptions, etc and point them to the BuildStream community looking for debate and answers.
  • Report: provided in .pdf and .odt, it has been publishing in the BuildStream Group in gitlab.com and sent to the community mailing list. The report include several recommendations.

The data set could lead us to a deeper analysis but:

  • It would have also take me more time.
  • I wanted to involve the contributors and stakeholders early in the analysis phase.
  • Some metrics which collection, processing and plotting can be automated has been identified already so to me it is better to consolidate them to bring value to the project on regular basis than to keep exploring.

I understand that my approach is arguable but it has worked for me in the past.  The debate of just half way cooked analysis increases the buy-in in the same way that developers love to put their hands in half-broken tools. Feel free to suggest a better approach that I can try in the coming future. I would appreciate it.

Link to the report on gitlab.com. Download it.

What’s next?

I am looking forward to have a fruitful debate about the report within the BuildStream community and beyond. From there, my recommendation is to look for an external provider (it is all about providing value as fast as possible) that, working with Open Source tools, can consolidate what we’ve learnt from this process and can help us to find more and better questions… and hopefully answers.

What is BuildStream

I have been putting effort on BuildStream since May 2018. Check the project out.

Stable: not moving vs. not breaking

There are two terms that brings a heavy controversy in the Open Source world: support and stable. Both of them have their roots in the “old days” of Open Source, where its commercial impact was low and very few companies made business with it.

You probably have read a lot about maintenance vs support. This controversy is older. I first heard of it in the context of Linux based distributions. Commercial distribution had to put effort in differentiating among the two because in Open SOurce they were used indistictly but not in business. But this post is about the adjectivet stable

Stable as adjective has several meanings in English:

  • According to the Cambridge dictionary stable is, among others: firmly fixed or not likely to move or change. I am used to this meaning because of my backgroun in Physics.
  • One of the definitions provided by the Oxford dictionary is slightly different: not likely to change or fail; firmly established.

Can you see the confusion between…?

  1. It is hard to move… or move slow.
  2. It does not fail… it is hard to break.

I am not an English native speaker. Maybe because of its latin root (I am Spanish) or maybe eacuase I studied Physics, I do not provide to the adjective stable the second meaning. It has always called my attention how many people provide both meanings to the word, specially when referring to software releases. Looking at the dictionaries, I can tell why in English there is such link between both ideas.

I read today on Twitter another example of this confusion/controversy. The comment has its roots on a different topic but it has embedded this confusion, I think.

Open Source projects used to have as the main testing strategy the collaboration with a group of beta testers and power users. This strategy helped to create this relation between “not moving” and “not failing” that has become so popular. But this relation is not direct if you have a different release/deploy strategy. The way to increase stability (as hard to break) is by detecting and fixing bugs and that can be done moving fast (constantly updating), or moving slow (backporting). I will not get into which one is better. Both are possible and popular.

I think the word stable should be avoided. Many system and application developers or integrators refer to it as “move slow” as “it will not change much“. But what many users hear is “it will not break” which is why “it moves slow“.

Which adjective can we use to label a release that “moves slow“, so it is not mistaken with “it is not expected to break“?

Is this controversy present in other languages as strong as I perceive it in English? Is it just me who see this controversy?

My new gravatar

More and more applications invite you to add a pic in your profile, a gravatar. In some applications and specially in social media the size and position of these gravatars are becoming prominent, too much for my taste. I am not a fan of pictures, I have never been. I do not like selfies, I have no Instagram, I try to avoid publishing pics of myself on Facebook… .

I use two different gravatars in internet. One for professional profiles and applications, associated to my corporate account ( agustin.benito@codethink.co.uk nowadays ), which was taken about 4 years ago, and one I took over 10 years ago associated to my personal account. I have hair on that one.

A few months back I started to think about my approach to this gravatars, looking for something simpler that could last, reflect my personality and that could be used for both, personal and professional profiles… and substitute the pic I have in my site.

My first idea was to ask a couple of photographers I know to take a good picture of me. One of those super modern, super cool, super fancy I-enjoy-life-you-should-ping-me-and-I-will-teach-you-how-but-only-if-you-are-cool-enough-to-deserve-my-coolness kind of picture. I was not fully convinced on this approach though. It would only work if the pic is really cool and checking LinkedIn, I had little confidence my pic could be at the level of some of my connections. Yep, I am connected to very cool people.

Then, looking at some gravatars from hackers I know who have zero interest in publishing a pic of themselves, I thought that maybe an illustration could work. Something simple, not a portrait. An illustration that looked like me but is not me, that could work in small and medium size formats… .

Exactly, I did not know what I want. It was more like… I do know a few things I do not want.

With this “crystal clear” idea, I contacted Ramón Miranda.

Ramón is a professional illustrator that has contributed to several Open Source projects, like Krita. He lives only a few kilometres away from my place in Málaga, Spain which is… great.

I explained him in a mail my idea. We have a video chat to polish the concept and a few days later he invited me over his place to show me a draft. A few days later my new representation of myself was done… with Open Source tools (Krita). And yes, I have the source file.

toscalix avatar

Thank you Ramón for accepting the challenge and for your dedication. I like the result… a lot.

Dear reader, do you? Yep, I will erase you comment if you don´t.

If you ever need a professional illustrator consider pinging Ramón. He is a hell of a professional. Ah, and he provides online training sessions for those wanting to learn or improve how to draw/paint in the analog and in the digital world (with Open Source tools).

Time to substitute my pics with this new gravatar.

A summary of my participation at OSS EU / ELCE 2018

ELCE 2018 has been a key event each of my 3 years at Codethink. It is an interesting conference from the technical and business point of view. This year the event was collocated again with several others, like the Open Source Summit Europe (OSS EU). It took place in Edinburgh, UK, I city I find particularly beautiful and enjoyable.

CIP, the Linux Foundation Initiative for Civil Infrastructure, has the event as a key milestone. The group organises a booth and a face to face meeting of the Technical Steering Committee (TSC) the day before the event starts. So for me ELCE is a day longer. The other main task for me at this event is supporting Codethink business development and community engagement actions, since I represent the company at the Linux Foundation.

During the CIP TSC face to face meeting, I did a short presentation of one of the projects I am putting effort on lately: BuildStream. I also described the project among several people I know during the event, asking them to try out the integration tool and provide me feedback. The tool has matured quite a lot the last few months and I am interested in collecting feedback from experienced developers and software integrators.

Lukas Bulwahn is leading an interesting effort to create a safety Linux related initiative at the Linux Foundation, including a Safety Critical Systems track at the event. I attended the last day of the conference to these talks. My colleagues Ben Dooks, senior kernel developer and Codehtink’s CEO Paul Sherwood, presented there a couple of topics we have been working on lately. I found the track interesting and learnt a few things. I hope such track consolidates and more automotive companies participate in the future.

Codethink sponsored the event and had a booth to recruit new developers. I backed up my colleague Tim a couple of times during the first two days of the event which allowed me to talk to a couple of of potential hires. It is always interesting to find out what motivates others about the company your are working for.

During the second day of the event I took some time to visit some booths and learn about what other companies are doing, as well as attending to a couple of talks. I also had several interesting meetings and conversations with old friends and former colleagues, visited again the Edinburgh Castle, this time at night, and managed to try a few whiskies from the Highlands. Not everything was to be work, right?

kernelci.org is an initiative I feel attached to since it was born during my time as Director at Linaro, promoted by engineers from my department, specially Kevin Hilman. Time has demonstrated that the project has helped the Linux Kernel tremendously. I hope kernelci.org get a successful second life under the umbrella of the Linux Foundation.

At Akademy 2018 a group of KDE people agreed to put effort in showcasing KDE software at embedded events to demonstrate that Plasma Mobile is a credible option as HMI for automotive R&D environments. At ELCE 2018, we showed Plasma Mobile and a couple of applications on top of a YOCTO based system on a Raspberry Pi 3 with a 7″ touchscreen. Showing the same technologies across different form factors (a mobile, a Pinebook, a laptop with openSUSE and the mentioned RPi3) was impressive. Sadly my RPi3 worked only the first day because it burned. I guess I have taken it to too many events. RIP my dear….

I would like to highlight how excited I got when I saw the KDE booth at this event, the fantastic work that Adrian, Jonathan, Kenny and Paul did there and the significant impact KDE had. Getting out of the comfort zone is never easy, but usually worth it. I am looking forward to show further progress at FOSDEM and the Embedded World early in 2019.

Except surprises, this will be the last event of the year for me. It is time to prepare next year’s agenda. I will see you all at FOSDEM 2019. Thanks Codethink for sponsoring my participation at ELCE 2018.

Working in distributed / remote environments 4: one on ones (1:1s)

An important part of any (people) manager’s work is to evaluate how things are going for your team members, the people under your responsibility. When managing distributed or remote teams, the one on ones (1:1) become even more relevant than in collocated environments. In extreme cases, they represent the only opportunity a manager to have an honest conversation with a member of your team.

The older I get, the more relevance I provide to 1:1s. I have found myself in several occasions feeling that my manager did not paid enough attention to me, to the problems myself or my department or team were going through, that affected my motivation, the project I was working on or the organization as a whole. If is not a fun feeling, specially in remote environments, where isolation plays against you. In all cases, there was a common factor: 1:1s did not work well.

Whenever I can, I like to have short but frequent 1:1 with my team members. My preferred option is a weekly 25 minutes meeting through video chat, at the beginning of the day. I try to avoid 1:1s late in the afternoons, depending on time zones. This is tough to achieve when working with Asians (I am located in CET/WET), but it is worth making the effort.

Why weekly 1:1s

A significant number of people tend to dislike such frequent meetings initially, specially when working remotely remote. In my experience, it is often underestimated how important it is to have a healthy channel with your manager, specially when times get rough for whatever reason: personal reasons, due to frictions with colleagues, workload peaks, changes within the organization, strategy shifts, etc.

If you experience such resistance, I recommend you to invest some time early on to justify why you will schedule these weekly 1:1s. Do not take for granted they will agree with them. I provide the following arguments:

  • If I am the new guy, I need to learn about how things work so I need help. Having frequent meetings the first couple of months will help me to shorten my on-boarding process.
  • If you are trying to push changes, you want your direct reports to participate in the decision making process. Use these meetings to get feedback from them.
  • If anybody under your direct responsibility just joined the team, you want to make sure their landing process is smooth, specially if they manage a team themselves.

There are two additional points that help to overcome significant resistance to have weekly 1:1s:

  1. When there is no relevant topic to talk about or the team is experiencing a working peak, the meeting can be skipped if there is a common agreement. I rarely skip two 1:1s in a row though.
  2. These meetings are primarily to deal with people management topics, not execution topics.

In most cases, if you have regular performance evaluations, there are always topics to go over during these 25 minutes slots, that might help to develop your colleagues professional career and performance. As a manager, it is a great learning experience too.

There is a special case I try to pay special attention to. There are times in which a member of your team is already overloaded with too many meetings. Your priority then is to offload that person and clear her agenda. 1:1s become then the desired useful tool. The same principle applies to you. If you perceive the 1:1 with your manager as inopportune, you might need to take a closer look at your agenda. They never are, or should be.

Ground rules on 1:1s

Before the first 1:1 with a member of my team, I prepare a shared document including the most important ground rules for the meeting. Most  ground rules are generic so I will focus on those influenced by the special nature of a distributed or remote environment (D.R.E.):

  • Meeting goal: write down clearly the meeting goals and priorities. I recommend to put people management topics first and, only when those are covered or there is none, move on onto other kind of topics. In the absence of face to face interactions with her colleagues, other managers or yourself, these kind of meetings might be the only chance to deal with people management topics a person might have in your organization.
  • Use corporate approved channels only: this might sound like an obvious advice but there are occasions where it might become a topic>
    • I have found many engineers, specially in the Open Source space, unhappy with using proprietary tools or tools that imply the installation of an application or a plug-in that track data from their machines. This specially sensitive for contractors. Be sensitive with this case if you can.
    • When working in Open Source projects, having a clear separation between what is or is not confidential might not be easy. Confidential information should flow in corporate approved channels only. 1:1s should be consider as such so the default video chat tool might not meet legal requirements in this regard.
  • Define the process to change the schedule of the meetings: the idea is to be flexible about when to meet but strict about how the process of changing the schedule of the meeting should be done, specially when it comes to notify changes in advance. Both parties need to consider how difficult it is to manage agendas when working in distributed environments where people is located in a a wide range of time zones.
  • Start on time: add this explicitly along with how to communicate that you will be late. Remember that if people around you systematically joins late, you will end up reproducing such behaviour. This is a hard personal battle to fight.
    • What has worked for me in the past with people that systematically joins late is to agree to make a note in the shared document when your counterpart or yourself arrive late. In the performance evaluation, when you find several of these notes, the point become a topic as something to pay attention to in the coming evaluation period.
    • The toughest situation comes when executives or customers are frequently late, specially when they are not remote. Do not let the situation get into a point in which you become passive aggressive about it. If you give up with them, you will end up giving up with your team too. Make it a point.
  • Describe what is the shared document for and how to use it: I will come to this later.
  • Add links to the description of key processes and HR documentation: mature companies in distributed environments tend to have well defined written processes. The escalation process is just an example. I like to explain it so the other person understand what will it happen when he request you to escalate a topic.
  • Add the contact information of the participants: I find it very handy.

The shared 1:1 document

The mission of this document is to track those conversation points that any of the participants consider relevant enough. Every time I make sure we both agree with the written text. This is extremely important to me. If there is no agreement, we work on the redaction of the text until we do. I there is no possibility to reach an agreement, both entries, mine and hers should be included.

In my experience, the shared document only works well if it is confidential.

The goal is to reflect when a common understanding has been reached and, if it hasn’t, where is the discrepancy. By default, I suggest to track at least the following topics in the shared document:

  • Agenda: during the week I introduce an entry in the document for every topic I want to raise in the meeting, if there is time. Sometimes there is no time so adding some description of the topic prevent you from having a additional meeting to deal with them. It also helps me as a self note. I tend to easily forget the topics I want to raise during these meetings, specially when I have several with different people during the week, or they are bi-weekly instead of weekly.
  • Escalations: when for whatever reason I have to escalate a point, I make sure it is written down in the document, in agreement the other person. This is very helpful when the escalation process or the resolution takes some time.
  • ToDos: sometimes the outcome of a conversation is a task. I keep track of them and who is assigned to.
  • Points related with the performance evaluation process: these points should only be written down under agreement.
  • Outcome of interesting conversations for one of both parties.

Any other topic can be translated to the document if any party wants to. My 1:1 shared documents frequently have the following sections:

  1. Document title: 1:1 (name surname):Agustín B.B. (this helps me to find the document easily.
  2. Contact info of both parties.
  3. Index
  4. Entry for each meeting with the following heading: Meeting #. Weekday YYYY/MM/DD
    1. Closed ToDos (on this week)
    2. Agenda items
    3. Open ToDos

Some other tips I recommend you to consider when it comes to using the document are:

  • Be short and accurate. This is not about tracking everything. It is not about taking minutes but about sharing a common understanding and agreeing on the outcomes.
  • Provide context. This document is meant to be consulted several times per year, like during performance reviews, for instance.
  • Although the document is confidential, write as if it is not. This is a general rule, in my opinion, to every communication.

Meeting how-to

I always start the meeting asking the person on the screen if there is any point she wants to raise. If an additional meeting is needed to go over the agenda, it should be because there was no chance to go over my points.

In my experience people with none or little experience in distributed or remote environments tend to avoid raising topics at the very beginning of the conversation. Being direct, adapted to the environment, takes some time. It is a learning process we all have gone through. If that is the case, I ask general questions about how is she doing. If nothing is raised, I like to check a couple more times during the meeting.

Only if there is time left after people management topics have been discussed, I introduce execution related topics. In any case, I like to dedicate some minutes to work unrelated topics, at the beginning or the end of the meeting depending on the agenda.

I like to configure the calendar to get a notification in my screen 5 and 1 min before the end of the meeting so I avoid cutting down the conversation abruptly. It is so easy to loose track of time…

Some additional points to consider

Do not forget about the intrinsic limitations of a communication through video chat, specially when feelings are involved. Compared to a face to face conversations, this is a serious handicap you will have to overcome as manager. Creating the right atmosphere during the good times will help you a lot when rougher times arrive, and they will.

These meetings might be the only chance for your counterpart to “talk to” or get “first hand information from” the organization. The impact of your words might get magnified by these two circumstances.

Since the nature of the relations between peers is different in remote environments than in collocated ones, managers has a higher chance to get “caught by surprise” during the one on ones by a problem, complain or a demand you had no previous information of.  If that is the case, be transparent about it and ask for some time to digest the information and provide a credible answer. Schedule another meeting if necessary. There is no award for providing a quick answer.

Do not underestimate the language barrier. It is already hard enough to talk with a manager openly through a screen about a sensitive topic. Add the fact that you have to do it in a language other than your mother tongue and you will soon realise that the shared document might work as a great ally. Cultural barriers are also higher to overcome in remote environments, as I have mentioned in previous posts of this series.

Conclusions

1:1 meetings are an essential “resource” for any manager. As expected, the remote or distributed nature of a working environment has a significant impact on this “tool” which makes it harder to master, at least in my opinion.

I prefer to have frequent and short check points than infrequent and long meetings. I also prefer to schedule an additional meeting than overrun the current one, since you loose very little time in joining/dropping from a meeting when working remotely. This might not be the case in distributed environments.

I like to create, pay attention to and put effort in translating the outcomes of interesting conversations into a shared and confidential document, keeping in mind that 1:1s are above all, about having an honest, transparent and direct conversations. Having them through internet makes the communication tougher which requires a higher doses of experience from any manager to make them meaningful, specially during stormy weather.