Software Product Inventory: what is it and how to implement it.

The concept of inventory applied to software, sometimes called catalogue, is not new. In IT/help-desk it usually refers to the software deployed in your organization. Along the history, there has been many IT Software Inventory Management tools. I first started to think about it beyond that meaning when working in deployments of Linux based desktops at scale.

The popularity that Open Source and Continuous Delivering is providing this traditionally static concept a wider scope as well as more relevance. It is still immature though, so read the article with that in mind.

1.- What is Inventory in software product development?

I like to think about the software inventory as the single source of truth of your software product so the main element for product development and delivery auditing purposes.

Isn’t that the source code?

Yes, but not only. The source code corresponding to the product that you ship (distribute) is a big part of it, but there are other important elements that should be considered part of the inventory like:

  • Requirements and/or tests, logs and results.
  • Technical documentation.
  • Tools and pipelines configuration files.
  • Packages, definitions or recipes…
  • Hashes, signatures, crypto libraries
  • License metadata, manifests, etc.
  • Metadata associated to security checks, permissions descriptions… .
  • Data associated with process performance metrics and monitoring/telemetry.
  • Many more…

When defined that way, the Software Inventory is a concept relevant in every stage of the software product life cycle. When you introduce, change, produce, publish, deploy or distribute any element of your product portfolio, your software inventory should change too.

There are two interesting considerations to add.

1.- If your product is part of a supply chain, like in any Open Source upstream/downstream environment, then the software inventory concept expands and become even more relevant since it can become an essential inbound-outbound control mechanism, even at acquisition time.

2.- In critical environments, especially safety critical ones, keeping such single source of truth goes beyond a “good practice”. Integrity, traceability and reproducibility for example, can be simpler to manage with a Product Software Inventory.

When you think about this particular case, it becomes clear to me that the elements that belongs to the inventory go beyond the actual deliverables or “product sources”. It should also include those elements and tools necessary to generate them, transform them, evaluate, deploy/ship them and evaluate its purpose.

2.- Static vs dynamic concept

Considering the above, the Software Product Inventory is a living construction, so dynamic, with the capacity to be frozen at any point in time (snapshot). This might seem obvious but it implies a different approach than supply and release management has traditionally considered (deliverables).

If evaluating, adding, modifying or managing elements of the inventory requires any action that significantly increases the cycle time of any specific stage, decompose those actions, parallelize them when possible and, when there is no choice, push it right in the pipelines. Ideally, no Software Product Inventory related activity should produce any friction in the code flow.

In a Continuous delivery environment, implementing the inventory requires to take actions across the entire development and delivery processes. Here are some points to consider at key stages:

2.1.- Inbound process: stage 0 of the development process

No software or any other element can become part of the product portfolio if it is not present in the Software Inventory. It make sense to implement the Product Inventory concept as part of the inbound stage (stage 0). Following Continuous Delivery principles and practices, here are some things to avoid vs promote:

  • Handovers or manual/committee-based vs code-review-like approval processes (pull vs push).
  • Completeness vs walking skeleton approach.
  • Management oriented (document based) vs engineering oriented (git based) tooling whenever possible.
  • Reports (manual) vs evidences (automated and reproducible) as output.
  • Access control vs signing and encrypting (if needed).

Avoid gate keeping activities. It is better to promote high throughput and short feedback loops than “quality gates” to improve product quality. If an evaluation is not completed, it is better to tag such piece of software as pending for a decision and letting the code flow than to have the engineers waiting for a decision of third parties.

I recognize that the concept might be too abstract to be easy to buy at first beyond the inbound and outbound (release/deploy) stages. Sadly, there is a strong tendency to pick up the concept at the inbound stage to establish early on a gate keeper, committee-based process to control the software that the developers use in the project, frequently compromising the code flow at a very early stage.

I prefer to focus on the procurement stage in the case of suppliers or how the relation is established with partners first. These are hand-over processes that heavily benefit from restructuring them, reducing the acquisition and on-boarding time and conditions.

More frequently that I would like to admit, Open Source is becoming a driver in this wrong direction, in many cases due to the proliferation of Open Source Offices in corporations that prefer to focus their initial attention in establishing specific policies for their own developers than in to changing their relations with partners and suppliers.

This is frequently due to a lack of understating of software product development at scale and what Continuous Delivery is about. In a nutshell, having their own engineers selecting the right Open Source software is prioritized over changing the relation with their existing commercial ecosystem, a more difficult but higher impact activity in many cases, according to my experience.

2.2.- Outbound process (deployment or release): last last stage of the delivery process.

The inventory accumulates all the elements required to ship/deploy the product plus all the elements required to recreate and evaluate the development and delivery process as well as the product itself, no matter if they are released or deployed. Ideally, this elements are evidence-based instead of report-based.

Like in the inbound case, each element of the Inventory should signed/encrypted as well as the overall snapshot, associated to the deployed/released product version. In case you are consuming or producing Free Software, please see Open Chain Project specification for more information about some good practices.

2.3.- Intermediate stages

As previously mentioned, the concept of Inventory is relevant at every stage of the development/delivery process. In general, it is all about generating additional/parallel outputs within the pipelines, signing and storing them in association with the related source code and binaries in a way that those evidences become “part of the product”. Using proprietary tools might break the trust chain in your process. This is something to consider carefully in safety critical environments. You will also need to consider the hardware, including dev. versions and prototypes.

A very interesting and open field is the Inventory concept in the context of safety critical certifications that traditionally have been very report-heavy-oriented. In this regard, I find the usage of system thinking very promising. Check Trustable Software, for instance.

3.- Some practical advice

3.1.- Walking skeleton vs completeness

I love the walking skeleton concept to design and implement processes in product development. It is significantly better to establish and end-to-end approach to the Inventory, where it has a light/soft/incomplete presence along the entire development/delivery cycle, than trying to implement it stage by stage following completeness, preventing you from having process-wide feedback loops.

It is not so much about doing it right as it is about moving fast in the right direction.

For instance, a frequent mistake is to concentrate most of the activities related to software license compliance on the inbound and outbound stages. Software license conformance and clearance has traditionally been perceived in many industries as a validation process performed by specialists, just like testing was done not so long ago, or a procurement action (acquisition).

Although lately more and more corporations are promoting the execution of license compliance activities at both stages (inbound and outbound), since they consume and ship more and more FOSS, they are still very report-based, specialists driven and management controlled activity.

I have witnessed enough dramatic situations to understand and promote that software license compliance is everybody’s job, just like tests or technical documentation (everything as code approach). Software license conformance and clearance, together with security, testing and technical documentation, can become the key drivers of the implementation of the Product Inventory concept. They share the same principles in this regard. The history of testing is the mirror to look at.

Decompose the software license compliance activities (conformance and clearance) and perform them across your pipelines. Start by executing simple conformance checks (REUSE) early on (inbound process, for instance). Coordinate such activities with the security team to also perform simple static code analysis. Agree with the architects or tech leaders in checking coding guidelines or other elements that can have a future impact in quality taking advantage of the Inventory concept. Add not just the software and the checks to the Inventory but also the results, logs and simple tools/scripts used.

More time consuming and intensive activities using more complex static code analysis or code scanning (licenses) tools can use the inventory as source (pull approach), instead of requesting the teams to perform such activities on their own (outside the pipelines) or establishing hand-over processes with specialists.

Be careful about how and when you include such activities in the pipelines though. Again, decompose and parallelize. And only when there is no choice, push these activities right in the delivery process. But do not break the code flow.

3.2.- Keep It Simple Stupid until it is not stupid anymore.

Here are some simple actions to start with…

At the beginning at least, use for the Inventory the same tools you are already using for the development of the product. Initially, your inventory can be nothing more than a file with a list of repos, hashes and links pointing at product elements location. This already have a value for security and software license compliance teams.

If you use a multi-repository approach pay attention to where the build tool pull the software from (definitions/recipes) to integrate the product. Make sure your initial inventory and the build tool are “in sync”. This will have a tremendous impact later on.

Export, sign and export the technical documentation living in your repositories (markdown, plantUML, .svg etc) as documents if they need to be part of the product deliverables, so you can establish simple checks to confirm their presence , integrity, etc.. This outputs should be also part of the Inventory as well.

Many of you already perform these and many other activities as part of the development and delivery process. The question is what to do with the associated metadata, the tooling used to generate them, the intermediate states required to get the output, the executed scripts, how to related them with the code and with other elements from previous stages of the process, how you guarantee their persistence, integrity, how you manage them at scale, how to store them etc..

3.3.- Act as an auditor

I always ask myself the following question: if I replicate the complete product development system providing all the product inputs for auditing purposes, how can I save time to the auditor so she does not need to understand the system itself and perform all the actions again to fully trust the output and the system itself (what/how we did, evidence based) and not myself or the workforce (who did it, report based)? Remember that you or your team will be such auditors in the future.

3.4.- Do not push the concept too early

If your stating point is very management (so reporting) driven, like for instance an environment where requirements or license information is generated and kept in .docx documents, the usage of binaries over source code is the norm, where proprietary tools are not questioned or where trust in the actual processes and outputs are based on who signs the associated reports (Officers) instead of in evidences, save your self headaches and do not try to push the Inventory concept beyond your area of direct influence. In my experience, it is worthless. You have a different battle to fight, a different and deeper problem to solve in such case.

In such cases, simply get ready in your area of influence for the future crisis to come, that will hit your product, so you get a chance to become part of the potential solution. Look for allies like the security, software license compliance and technical writing teams for instance, to expand that ring of influence. Hopefully they will see the value and the inventory how it can become a trigger to support modern and widely accepted practices within their domains across the organization.

4.- Summary

The Software Product Inventory is a high level (abstract) concept that will help you to move towards creating trustable software. Since it is part of the development process, following Continuous Delivery principles and practices to design and implement it is essential.

Some of the concepts behind this idea are probably present already in your organization but frequently using a management-driven approach, associated to the release and/or inbound stages, implemented in ways that generates a negative impact (friction) in the product code flow, so throughput and stability.

System thinking and asking questions as if you were an auditor will help you to implement simple measures first, and habits later, that will raise the quality of the product over time, even if the Product Inventory concept does not fly in your team or organization. That approach has helped me, at least.

Interview candidates with an Open Source background

I often say that there are two actions that define the line management role: one-on-ones and hiring people. This is specially true in growing organizations. If you nail these two actions, you have a good chance to truly influence, the ultimate goal, in my view, of any line manager.

One of the common strategies to speed up the journey from being an Open Source contributor to becoming a Good Open Source citizen is to hire talent with a solid Open Source background. The process of hiring such talent is different from what most organizations and recruiters are used to. That is so true that we have now companies specialized in hiring these profiles.

One key part of the hiring process is the interview, obviously.

This article is another one of those I have written the last couple of years about management topics, based on my experience working in Open Source as manager and consultant. More specifically, it is an attempt to describe some of the key points that hiring managers with little or no experience in hiring Open Source talent need to consider to increase their hit rate.

As usual, I would appreciate if you add in the comments section or to me directly your experience, criticisms or missing points. I would add them to this article as update.

Evaluate the candidate’s work in advance

If you only have room in your hard disk for one takeaway, it should be this point.

Most of your candidate’s work is public. Take the time to evaluate it or have somebody else doing it for you, before the interview. If the candidate have public talks, watch the videos or check the slides. If she has written articles, read them. If she has code available, study it, test it, install it… .

Having the candidate’s work in the open is a double edge sword. It is an advantage for the hiring organization because it can reduce significantly the hiring process, since you can evaluate in detail many of the candidate skills in advance, so the interview process can be reduced to a single interview, for instance (I strongly recommend this, by the way).

At the same time, if the hiring manager do not take the time to evaluate the candidate’s work or even worse, if the manager acts during the interview as if that work is not public, trying to technically evaluate the candidate prior or during the interview, for instance, the failure is almost guaranteed.

I always mention how wrong it is to ask a music group with published records or videos of their concerts available on the group’s website if they know how to sing. Or even worse, to request an audition during the hiring process for your birthday party. Who does that? Instead, watch their performances, skip the audition and go directly to negotiate other relevant points. Their work is public, did you check it?

By the way, the world is significantly larger than GitHub, thanks God. The candidate might have a strong Open Source background without having a great GitHub profile… or no profile at all.

Interview goal

The interview is not just an opportunity to evaluate if the candidate fits in your organization, it is a chance to attract that professional and her colleagues. Remember, the candidates code, processes followed, tooling used… are public. Your organization’s most likely not.

Who has more to inform about?

As result of this reality, the interview should look more like hiring a senior manager/expert than about hiring an engineer.

In summary, the goal of the interview shifts. It should be balanced between what both, the organization and the candidate looks for. This shift represents a challenge for hiring managers with limited experience in hiring senior talent.

Candidate’s future upstream participation

During the interview, be clear about the amount of time the candidate will have to invest in upstream work, specially if he candidate is working upstream already a significant amount of time.

There are plenty of developers who prefer to keep their work in the open as a personal activity. On the contrary, some others want to make a living out of their participation upstream. The best way to ensure a sustainable relation over time is to be transparent about the expectations related with the effort put in internal work vs work done in the open as well as in which projects.

If by working in your organization, the developer will have to reduce her current involvement in the open, specially in specific periods of time, say it clearly. If you cannot guarantee that the technologies the candidate works with in the open will be part of the company’s tools set, state it clearly. If your organization has plans to move towards Open Source and you bring the candidate to speed up that journey, make sure she understands the timelines and expectations.

No matter how experienced you are as hiring manager, there is always room for a candidate to surprise you about her motivations to participate in the interview and potentially join your organization. This is most likely true with professionals coming from an industry you might not be very familiar with.

Tooling, languages, technologies…

One of the great advantages of being an Open Source professional is that, because you use open source tooling, you can take them with you throughout your career, which makes you very efficient. Professionals with a solid Open Source background do not depend on certifications paid by their companies to learn how to use their tools, or having to change their base tool-set regularly, limited by commercial decisions. In other words, Open Source professionals have made their career supported in specific tooling and associated practices they have chosen and mastered. Often, they have even helped to develop or polish those tools. Some of them love to try new tooling on regular basis and some others love to focus their energy in other aspects, not in the tooling or associated processes, so they are conservatives.

In some cases, this specific point is so important for the candidate that using those tools and associated practices represents a requirement to join your organization.

You will need to consider and potentially embrace this reality. If you cannot or do not want to use those technologies or tools, make the point during the interview. But before you do, I recommend consider the following…

Are you open to reconsider such decisions if somebody prove there is something better? Is it something that can be reconsidered next year, when the current support contract for tool X expires? Do you have any specific department or project where you are willing to try new tools, processes, languages…? Are you open to assume that you might have made a mistake in such decision or that those decisions will not stand the test of time?

Many Open Source professionals, for instance, consider that, doing Open Source with Open Source tools is significantly better that using proprietary ones. Why?

At scale, adapting the tooling to the needs of the specific environment and integrate them with other tooling to improve the development experience is essential. Even more that to use a feature rich tool. To have full control of the tool’s evolution in your specific environment is another interesting point… there are many other strategic and practical reasons that leads to that mindset.

Learn from the candidate about those arguments during the interview.

Very often hiring managers see the candidate’s position in this front as religious or philosophical. Instead, I would take it as an opportunity to improve and innovate within their organizations. I suggest to embrace such experience the candidate can bring to your organization as starting point during the interview and take the conversation from there, instead of questioning her criteria and define this point as a no-go early in the interview or the overall process.

Recommendations and referrals

Open Source is about trust and networking, based on the work the potential candidate does within one or several communities. In many cases, the candidate recommendations and referrals come from peers from those communities.

If you are a hiring manager, take those recommendations as seriously as any traditional referral from a senior manager or a relevant figure in any big corporation. They are frequently equally valuable.

In Open Source, you are judged by your work and your influence, not by your position in a hierarchy of any organization.

Get help for the interview

If you have little or no experience with Open Source, bring somebody that do have it to the interview, in addition to a technical expert, specially if you are interviewing engineers. It is not relevant if such person is not an expert in the candidate’s knowledge area of expertise.

I am not a technical person and, when I join such interviews, the hiring manager often learn new things about the candidate because we both (candidate with solid Open Source background and myself) share a similar working culture. This is specially true when the candidate starts to ask questions to evaluate if the company is a good fit for her. In addition, it often plays in favor of the company to show to the candidate that there is Open Source knowledge internally and peers that feel comfortable working there.

Assume the candidate knows about your organization already

Do not underestimate how much candidates with solid Open Source background know about your company before the interview, even if your organization is not an Open Source contributor.

These kind of professionals might learn about your organization not just by interacting directly with your engineers or evaluating your products, but also by interacting with your suppliers and partners in the open, in some cases on regular basis. They might know about the hardware you use, the versions of specific software you ship, tools you use, release strategies you have, updates policies, etc. They might even point at clear deficiencies and strengths of your products and services as well.

I suggest to ask the candidates early during the interview what do they know about your organization and why do they want to join it. It is also interesting to ask about their evaluation/perception of your organization as a whole and your products/services compared to others.

It is not only about getting information from the candidate but also about informed perceptions. Such conversation might help hiring managers to tune the description about what the organization does and avoid providing superfluous information during the interview. It also helps to understand if the candidate is one of those who want to work in industry leaders or help you to become one, which is something I am always interested to learn about them.

Why do you need a CV?

Even today, I still come across former colleagues and great Open Source professionals who do not have a CV. I have helped several of them throughout the years to create one or update it.

For an Open Source professional, their work speak for themselves. It is a better presentation that any CV will ever be, in my view. It is the outdated business culture the one that should change and adapt to this “not so new” way of “filtering candidates”. It is simply better for all parties involved.

Many great Open Source professionals have a mediocre CVs or no CV at all because they never needed it. For some, this lack of CV is a way to filter organizations they do not want to work with. Their thinking goes in the line of… if you need my CV to filter me as candidate or evaluate my work, I am not interested in working with you. Check my work!

Instead of questioning the quality of their CV, take the opportunity to ask yourself why do you or your organization need a CV when the candidate’s work is publicly available. It might be a great introspective exercise. Maybe you end realizing that only a bunch of links and his contact information are really needed, instead of a formal CV.

Myth: Open Source developers are not good communicators

Open Source professionals, specially developers, are used to great levels of exposure to other professionals worldwide. In general, they need to become good communicators to succeed within their communities.

Open Source developers as people incapable to communicate is a myth. Such perception often depends on what you understand by communicating and to whom, but specially it is based on myopia often from people with a limited technical background.

It is true in my experience though that in general, many Open Source developers do not do great interviews. But that is mostly because they do very few of them. They do not go to 10 interviews with four different companies to get a job. They do not need to. And what is more important, throughout their careers, interviews are not used to evaluate them technically, again, because their work speak for themselves.

Evaluate the candidate’s written communication skills. Check the mails they send to dissent with others or describe their position, read the blog posts they write summarizing their latest work, check their presentations, their slides… Check how they interact with their users, code maintainers… You will soon realize they interact with profiles or groups of people on regular basis that many corporate engineers do not. And those interactions often face ore limitations that within any corporation, due to cultural differences.

When talking to managers unfamiliar with Open Source about this point I always ask them how many professionals they know that having done good interviews, are still incapable of standing behind their opinions in a meeting with senior colleagues or customers. Or how many people in their teams excel at describing complex ideas in simple words (in a 10 min presentation, for instance).

Public exposure to peers and users in Open Source projects are an outstanding learning environment for developing certain communication skills. Be careful with this myth.

English level

In Open Source, most of the communication among developers take place in written form. This is due to the distributed and asynchronous nature of the environment they work in (software development at scale). Most of those developers have a better level of written English than spoken.

I always recommend to be careful when evaluating the candidate English skills only through the interview. To have a full perspective of the candidate skills, I always recommend to check her written communications and articles. If they have videos of their talks available, review them. You might perceive noticeable differences between their written and spoken English levels.

When doing an interview, it is very hard to be quick and provide sharp answers without thinking when you do not speak English on daily basis. I have experienced this myself.

My point here is that you might evaluate her English skills as poor based on the interview when, in many cases, they have solid written skills, so they need little time to develop the minimum level required to work in English.

Agile fans tend to underestimate how important it is to have a good written level in English compared to the spoken/conversation level. In Open Source (develop software at scale), the trend is exactly the opposite… for good reasons.

Remote Work

Many professionals prefer to live by the beach, in a cheaper place, than to commute for an hour to go to the office in a big city. At the same time, when developing software at scale, it is unrealistic to think you will find the talent you need in a single location; even in two different locations. It should not come to a surprise that the market is moving towards developing software in distributed environments where remote workers are not the exception anymore.

The number of “remote first” and “remote friendly” companies is growing in number and size at a fast pace. Open Source is about highly distributed, asynchronous and high latency environments. Some of the larger software engineering projects has grown and mature with those characteristics by design.

The candidate has probably made a career out of mastering working on such environment.

Instead of questioning the efficiency of distributed teams during the interview (common practice among remote work detractors, like radical agilists and traditional managers), take the opportunity to learn about it.

If you think that co-located, synchronous and low latency environments are more efficient, ask about how current tasks at your company could be performed remotely, how ceremonies can be adapted, how testing can be performed remotely. Learn from the candidate about how to develop software at scale, how he has acquire such experience and levels of efficiency in such environment.

I have worked in organizations where testing was done remotely, in a heavily distributed environment, for instance. Technology and creativity have made Open Source projects the best environment for developing software at scale. Even if your organization is small, the candidate skills might become an asset.

The network effect

Working in the open provides any professional the possibility of building a large network. That interview might well be the first and only direct interaction that such professionals might have with your company. If the experience is not satisfactory, her network will know.

Would you recommend a person you trust or respect to apply to a company that did not provide you a good experience during the hiring process? And I am not talking about hiring them. I have had good experiences with companies that chose a different candidate during the process. We are grownups. We can take rejection.

By ignoring the point on this article (or others i skipped), you are not just simply making harder for your organization to hire Open Source talent. You are shouting with a speaker your deficiencies.

It is not a coincidence that some corporation with pedigree have such a hard time hiring good Open Source professionals while some others less popular are so successful. It is partly because of the networking effect.

Do it right a couple of times and more talent will come. On the contrary, fail badly and you will pay the prize for some time.

How to move the candidate out of her comfort zone

Part of any interview’s goal is to get from the candidate things she did not plan to show or surface.

People used to work in the open, with high levels of exposure, develop a great sense of correctness and politeness when expressing opinions. That does not mean they are not passionate about what they do or that they do not have “radikal opinions”. They are and you want to know them.

Ask the candidate for her opinions about tools, technologies, projects from your competitors or technologies that the community project they participate on are struggling with. I believe these kind of questions are a fair way to move the candidate out of her comfort zone and get their “essence”.

But when you do, be aware about the trends and debates the candidate and/or the community she participates on have had about such topics. Be careful with making statements that are a no-go for the candidate simply because you did not prepare the interview well enough.

Avoid the typical myths around Open Source, the stupid arguments about battles between communities, superfluous statements about Open Source business models and their feasibility, etc. Instead, ask about, for instance, how do they see their community project in five years, what are the challenges they are facing, what are the deficiencies they need to overcome, etc. Do your homework about Open Source and the specific community the developer is involved in before the interview. Nobody wants to work for a person that seems a fool or lazy.

Again, if you do not know about these topics or had no preparation time, bring somebody with knowledge to the interview or state your ignorance up front.

Summary

Radical exposure to peers and users, development of public code, knowledge share with professionals from different industries and global organizations, etc. are characteristics about the candidate that cannot and shouldn’t be ignored during the hiring process, specially during the interview.

Instead of applying the same template that has worked fine in the past for your organization, hiring managers will need to evolve it or even change it in order to attract people with solid Open Source background. This is yet another of those transformations and culture changes that any organization will need to go through to fully embrace Open Source.

To me is very simple, if you want to become a Good Open Source Citizen, you will need to hire people with a strong Open Source background if you intend to speed up the journey. Either you adapt your hiring processes, including the interviews, or you will most likely fail in such goal.

To adapt, hiring managers and experts need to develop new habits. That can only come if they question the current ones. This evolution will take place faster and with a smaller impact for their organizations if they get help.

I think this topic has gotten less attention that it deserves.

OpenSouthCode 2019 recap and new information added to my site

OpenSouthCode 2019 recap

OpenSouthCode is a FLOSS event that takes place in Málaga, Spain, every year. I have written about it before:OLYMPUS DIGITAL CAMERA

The event tool place this year in a new venue, significantly better than the previous one, in my opinion. More than 300 people were registered which is not bad at all for a free of charge event about Open Source that does not require pre-registration to participate.

Some workshops and talks were packed, although not the majority of them. Some people has commented that there did not feel a “sense of packed” which is was due to the fact that, during 2 days, the event offered 2 to 4 tracks and workshops simultaneously. Saturday was busier than Friday, I think.

I don’t feel that there is anything bad in having only a few people at your talk if they are truly interested. With such an interesting and diverse offering, motivated participants is almost guaranteed. I understand though that if you come from far away or your company send you to give a talk, having a full room is a good thing.

The event is little by little growing. The organization in general goes smoother, the quality of the talks and the speakers is better every edition, the workshops, specially those for kids, are gaining traction, the venue is better, there were sponsors this year… All signs are positive.

As a suggestion for the 2020 edition, I would organise a closing keynote so participants can get together afterwards for some drinks. This would improve the sense of community and would provide a good opportunity to thank the sponsors.

I am happy with how my talk went. Around 15 people attended. I could attend to 3 additional talks ramon_agustin_paul_opensouthcode_2019which were very good. I learned a lot. It was great news to see Ramón Miranda giving a talk about Krita, by the way. Thanks Paul for your advises about my slides and Gaby for the pics.

Special thanks to the OpenSouthCode organisers for putting the event together once again. See you next year. Follow them on Twitter to know more about the next edition.

Latest updates on my site

The past weeks I have updated some information on my site:

  • I have added the slides of my OpenSouthCode 2019 talk to the Talks page, together with some additional links from previous talks.
  • I have added a couple of great books I have read lately and/or use widely. Check them out in the Reads section of this site. A couple more will be added the coming weeks.

FOSDEM 2019 and CHAOSS EU 2019 report

FOSDEM is over and it is time to recap.

Last year I decided to take a break and did not attend to the event. This year I was really looking forward to attend.

I will start by thanking Codethink Ltd for sponsoring my trip. It is always a pleasure to work in a company that supports their employees in attending to Open Source community events. Codethink sponsored FOSDEM once again by the way.

It has not been the easiest edition for me because I have been sick the past days and was not fully recovered. The cold weather didn’t help so I decided to stay away from late nights and Trappist beers. It was hard to go to bed at a decent time every night and miss some night gatherings like the KDE and GNOME ones or the FOSDEM party on Friday at Delirium Cafe.

On February 1st I attended to the CHAOSScon EU conference. I liked it. It was well organised and I could have several interesting conversations about what to measure and why when it comes to Open Source communities. I attended to most of the talks and I participated in one of the workshops. I think I can add some value in the GMD working group. Let’s see if I have the time to contribute. It would be fun.

I would like to highlight the prominent role that Bitergia, a Spanish company, plays in the CHAOSS project, a Linux Foundation Initiative. Despite being a small organization, they are in the front line when it comes to software analytics, specially in the Open Source space. Well done Bitergians!

As you probably know, I am putting some effort, together with some KDE developers, in calling the attention within the KDE community about the immense opportunity this project has in embedded, now that Plasma Mobile and Kirigami are a reality. KDE project is making an efforts also to show devices with this new shell at events, so professionals and corporations can identify the value that the KDE as community can add in ecosystems like the (open) automotive ones.

ELCE 2018 was the first event were we showed the outcome of our efforts in embedded ()in mobile we have for some time now). FOSDEM has been the second one. There will be more during this 2019.

dav
Plasma Mobile + Yocto in a RPi3 and Risc-V boards at the KDE booth. FOSDEM 2019

It was a pleasant surprise to see several boards and devices at the KDE booth, my RPi3 and a Risc-V board among them, both with Yocto and Plasma Mobile. I think attendees were both, surprised and happy to see KDE showing new and attractive stuff.

krita demo
Wolthera sketching live. Krita demo. FOSDEM 2019

Krita had a strong presence at the booth and the last day there was a KDEnlive demo, among other activities. A Pinebook, a Slimbook and an ARM based

Rockchip board completed the show. I think the booth worked extremely well. Some of the messages published in social media reflected it.

Special thanks to every KDE contributors that made the booth possible. I was really proud to be part of such an amazing group of people.

I attended to the automotive/embedded dinner on Saturday night. There is a group of people interested in reviving the Embedded devroom at FOSDEM 2020. The dinner’s main goal was to find out how many people wanted to help and coordinate them. Mission accomplished! Thanks Jan-Simon and Leon Avani for organising it.

On Sunday night I attended to the OpenChain informal dinner. Thanks Shane for organising it. I had a really good time. Lawyers are very cool people. There were several interesting conversations there about the FLA, which is not well known yet among the legal and developer communities despite being around for several years.

I tried to attend to three talks during FOSDEM. I couldn´t even get close to the door in any of them because, not just were the rooms packed, but there was a long queue of people waiting to get in. I got a little frustrated and decided to stop trying. Videos will come to the rescue.

On the bright side, the organisers opened an additional cafeteria this year. I usually take some sandwiches and water with me to the the venue so I can skip the long lines to get something to eat. On Sunday I didn’t and it worked out well. I guess that the days when it was impossible to get a sandwich are over. Yay!

As usual I could talk with lots of people which is the part I like the most about this event. I could also chat with some of the many Codethings (colleagues from Codethink) that attended to the event. I also take with me new contacts and plenty of new technologies and project to evaluate.

In general it has been a very good event. I will spend a week in Manchester after FOSDEM and then go back home. My next stop will be Embedded World, in Nuremberg, GE at the end of the month.

Thanks to the FOSDEM organisers and volunteers for your effort and dedication.

FOSDEM rocks!

BuildStream metrics: exploration

Metrics and telemetry are fundamental in any engineering activity to evaluate, learn and improve. They are also needed to consolidate a culture in which opinion and experience are continuously challenged, in which experimentation and evidence becomes the norm and not the exception, in which transparency rules so co-workers are empowered, in which data analysis leads to conversations so evaluations are shared.

Open Source projects has been traditionally reluctant to promote telemetry, based on privacy concerns. Some factor that comes to my mind are helping to change this perception:

  • As FLOSS projects grow and mature the need for information grows.
  • It is easier now to process big amounts of data while keeping high levels of anonymity.
  • The proliferation of company driven and consortium driven FLOSS projects, specially those related with SaaS/cloud technologies and products, showing how useful telemetry is. In general, corporations are less concerned about personal data privacy than many Open Source projects though.
  • The DevOps movement is spreading like a pandemic and telemetry is an essential action for practitioners.

So the last few years data analytics is becoming more popular among Open Source projects.

Finding the right metrics is frequently tough. Most of the times projects, teams or departments get drowned in data and graphs before they realize what actually matters, what does it have real business value. When you find the right metrics, somehow it means that the right questions are being asked which I find the hardest part. To identify those questions, I recommend organizations or projects to invest in exploring and learning before moving into automating the data collection, processing, plotting and irradiate the results to be analysed.

So when BuildStream is getting into its third year of life, I thought it could be interesting to invest some effort in digging into some numbers, trying to find a couple of good questions that provide value to the project and the stakeholders involved.buildstream-beaver

The outcome of this exploratory effort was published and spread across the BuildStream / BuildGrid community. The steps taken to publish the report has been:

  • Select a question to drive this exploratory effort, in my case: are we growing?
  • Select data sources: in my case, information from the ticketing system and the git repositories.
  • Collect the data: in this case, the data sets from the BuildStream ticketing system were exported from gitlab.com and the data sets from git obtained through a script developed by Valentin David.
  • Clean the data set (data integrity, duplications, etc.) in this case the data was imported into GSheets and worked there.
  • Data processing: the data was processed and metrics were defined using GSheet since the calculations in this phase were simple enough and the amount of data and processing power did not represent a challenge for the tool.
  • Plot the data: since the graphs were also simple enough, GSheet was also used for this purpose.
  • Initial analysis: the goal here was to identify trends, singularities, exceptions, etc and point them to the BuildStream community looking for debate and answers.
  • Report: provided in .pdf and .odt, it has been publishing in the BuildStream Group in gitlab.com and sent to the community mailing list. The report include several recommendations.

The data set could lead us to a deeper analysis but:

  • It would have also take me more time.
  • I wanted to involve the contributors and stakeholders early in the analysis phase.
  • Some metrics which collection, processing and plotting can be automated has been identified already so to me it is better to consolidate them to bring value to the project on regular basis than to keep exploring.

I understand that my approach is arguable but it has worked for me in the past.  The debate of just half way cooked analysis increases the buy-in in the same way that developers love to put their hands in half-broken tools. Feel free to suggest a better approach that I can try in the coming future. I would appreciate it.

Link to the report on gitlab.com. Download it.

What’s next?

I am looking forward to have a fruitful debate about the report within the BuildStream community and beyond. From there, my recommendation is to look for an external provider (it is all about providing value as fast as possible) that, working with Open Source tools, can consolidate what we’ve learnt from this process and can help us to find more and better questions… and hopefully answers.

What is BuildStream

I have been putting effort on BuildStream since May 2018. Check the project out.