Software Product Inventory: what is it and how to implement it.

~ toscalix ~ Leave a comment

The concept of inventory applied to software, sometimes called catalogue, is not new. In IT/help-desk it usually refers to the software deployed in your organization. Along the history, there has been many IT Software Inventory Management tools. I first started to think about it beyond that meaning when working in deployments of Linux based desktops at scale.

The popularity that Open Source and Continuous Delivering is providing this traditionally static concept a wider scope as well as more relevance. It is still immature though, so read the article with that in mind.

1.- What is Inventory in software product development?

I like to think about the software inventory as the single source of truth of your software product so the main element for product development and delivery auditing purposes.

Isn’t that the source code?

Yes, but not only. The source code corresponding to the product that you ship (distribute) is a big part of it, but there are other important elements that should be considered part of the inventory like:

  • Requirements and/or tests, logs and results.
  • Technical documentation.
  • Tools and pipelines configuration files.
  • Packages, definitions or recipes…
  • Hashes, signatures, crypto libraries
  • License metadata, manifests, etc.
  • Metadata associated to security checks, permissions descriptions… .
  • Data associated with process performance metrics and monitoring/telemetry.
  • Many more…

When defined that way, the Software Inventory is a concept relevant in every stage of the software product life cycle. When you introduce, change, produce, publish, deploy or distribute any element of your product portfolio, your software inventory should change too.

There are two interesting considerations to add.

1.- If your product is part of a supply chain, like in any Open Source upstream/downstream environment, then the software inventory concept expands and become even more relevant since it can become an essential inbound-outbound control mechanism, even at acquisition time.

2.- In critical environments, especially safety critical ones, keeping such single source of truth goes beyond a “good practice”. Integrity, traceability and reproducibility for example, can be simpler to manage with a Product Software Inventory.

When you think about this particular case, it becomes clear to me that the elements that belongs to the inventory go beyond the actual deliverables or “product sources”. It should also include those elements and tools necessary to generate them, transform them, evaluate, deploy/ship them and evaluate its purpose.

2.- Static vs dynamic concept

Considering the above, the Software Product Inventory is a living construction, so dynamic, with the capacity to be frozen at any point in time (snapshot). This might seem obvious but it implies a different approach than supply and release management has traditionally considered (deliverables).

If evaluating, adding, modifying or managing elements of the inventory requires any action that significantly increases the cycle time of any specific stage, decompose those actions, parallelize them when possible and, when there is no choice, push it right in the pipelines. Ideally, no Software Product Inventory related activity should produce any friction in the code flow.

In a Continuous delivery environment, implementing the inventory requires to take actions across the entire development and delivery processes. Here are some points to consider at key stages:

2.1.- Inbound process: stage 0 of the development process

No software or any other element can become part of the product portfolio if it is not present in the Software Inventory. It make sense to implement the Product Inventory concept as part of the inbound stage (stage 0). Following Continuous Delivery principles and practices, here are some things to avoid vs promote:

  • Handovers or manual/committee-based vs code-review-like approval processes (pull vs push).
  • Completeness vs walking skeleton approach.
  • Management oriented (document based) vs engineering oriented (git based) tooling whenever possible.
  • Reports (manual) vs evidences (automated and reproducible) as output.
  • Access control vs signing and encrypting (if needed).

Avoid gate keeping activities. It is better to promote high throughput and short feedback loops than “quality gates” to improve product quality. If an evaluation is not completed, it is better to tag such piece of software as pending for a decision and letting the code flow than to have the engineers waiting for a decision of third parties.

I recognize that the concept might be too abstract to be easy to buy at first beyond the inbound and outbound (release/deploy) stages. Sadly, there is a strong tendency to pick up the concept at the inbound stage to establish early on a gate keeper, committee-based process to control the software that the developers use in the project, frequently compromising the code flow at a very early stage.

I prefer to focus on the procurement stage in the case of suppliers or how the relation is established with partners first. These are hand-over processes that heavily benefit from restructuring them, reducing the acquisition and on-boarding time and conditions.

More frequently that I would like to admit, Open Source is becoming a driver in this wrong direction, in many cases due to the proliferation of Open Source Offices in corporations that prefer to focus their initial attention in establishing specific policies for their own developers than in to changing their relations with partners and suppliers.

This is frequently due to a lack of understating of software product development at scale and what Continuous Delivery is about. In a nutshell, having their own engineers selecting the right Open Source software is prioritized over changing the relation with their existing commercial ecosystem, a more difficult but higher impact activity in many cases, according to my experience.

2.2.- Outbound process (deployment or release): last last stage of the delivery process.

The inventory accumulates all the elements required to ship/deploy the product plus all the elements required to recreate and evaluate the development and delivery process as well as the product itself, no matter if they are released or deployed. Ideally, this elements are evidence-based instead of report-based.

Like in the inbound case, each element of the Inventory should signed/encrypted as well as the overall snapshot, associated to the deployed/released product version. In case you are consuming or producing Free Software, please see Open Chain Project specification for more information about some good practices.

2.3.- Intermediate stages

As previously mentioned, the concept of Inventory is relevant at every stage of the development/delivery process. In general, it is all about generating additional/parallel outputs within the pipelines, signing and storing them in association with the related source code and binaries in a way that those evidences become “part of the product”. Using proprietary tools might break the trust chain in your process. This is something to consider carefully in safety critical environments. You will also need to consider the hardware, including dev. versions and prototypes.

A very interesting and open field is the Inventory concept in the context of safety critical certifications that traditionally have been very report-heavy-oriented. In this regard, I find the usage of system thinking very promising. Check Trustable Software, for instance.

3.- Some practical advice

3.1.- Walking skeleton vs completeness

I love the walking skeleton concept to design and implement processes in product development. It is significantly better to establish and end-to-end approach to the Inventory, where it has a light/soft/incomplete presence along the entire development/delivery cycle, than trying to implement it stage by stage following completeness, preventing you from having process-wide feedback loops.

It is not so much about doing it right as it is about moving fast in the right direction.

For instance, a frequent mistake is to concentrate most of the activities related to software license compliance on the inbound and outbound stages. Software license conformance and clearance has traditionally been perceived in many industries as a validation process performed by specialists, just like testing was done not so long ago, or a procurement action (acquisition).

Although lately more and more corporations are promoting the execution of license compliance activities at both stages (inbound and outbound), since they consume and ship more and more FOSS, they are still very report-based, specialists driven and management controlled activity.

I have witnessed enough dramatic situations to understand and promote that software license compliance is everybody’s job, just like tests or technical documentation (everything as code approach). Software license conformance and clearance, together with security, testing and technical documentation, can become the key drivers of the implementation of the Product Inventory concept. They share the same principles in this regard. The history of testing is the mirror to look at.

Decompose the software license compliance activities (conformance and clearance) and perform them across your pipelines. Start by executing simple conformance checks (REUSE) early on (inbound process, for instance). Coordinate such activities with the security team to also perform simple static code analysis. Agree with the architects or tech leaders in checking coding guidelines or other elements that can have a future impact in quality taking advantage of the Inventory concept. Add not just the software and the checks to the Inventory but also the results, logs and simple tools/scripts used.

More time consuming and intensive activities using more complex static code analysis or code scanning (licenses) tools can use the inventory as source (pull approach), instead of requesting the teams to perform such activities on their own (outside the pipelines) or establishing hand-over processes with specialists.

Be careful about how and when you include such activities in the pipelines though. Again, decompose and parallelize. And only when there is no choice, push these activities right in the delivery process. But do not break the code flow.

3.2.- Keep It Simple Stupid until it is not stupid anymore.

Here are some simple actions to start with…

At the beginning at least, use for the Inventory the same tools you are already using for the development of the product. Initially, your inventory can be nothing more than a file with a list of repos, hashes and links pointing at product elements location. This already have a value for security and software license compliance teams.

If you use a multi-repository approach pay attention to where the build tool pull the software from (definitions/recipes) to integrate the product. Make sure your initial inventory and the build tool are “in sync”. This will have a tremendous impact later on.

Export, sign and export the technical documentation living in your repositories (markdown, plantUML, .svg etc) as documents if they need to be part of the product deliverables, so you can establish simple checks to confirm their presence , integrity, etc.. This outputs should be also part of the Inventory as well.

Many of you already perform these and many other activities as part of the development and delivery process. The question is what to do with the associated metadata, the tooling used to generate them, the intermediate states required to get the output, the executed scripts, how to related them with the code and with other elements from previous stages of the process, how you guarantee their persistence, integrity, how you manage them at scale, how to store them etc..

3.3.- Act as an auditor

I always ask myself the following question: if I replicate the complete product development system providing all the product inputs for auditing purposes, how can I save time to the auditor so she does not need to understand the system itself and perform all the actions again to fully trust the output and the system itself (what/how we did, evidence based) and not myself or the workforce (who did it, report based)? Remember that you or your team will be such auditors in the future.

3.4.- Do not push the concept too early

If your stating point is very management (so reporting) driven, like for instance an environment where requirements or license information is generated and kept in .docx documents, the usage of binaries over source code is the norm, where proprietary tools are not questioned or where trust in the actual processes and outputs are based on who signs the associated reports (Officers) instead of in evidences, save your self headaches and do not try to push the Inventory concept beyond your area of direct influence. In my experience, it is worthless. You have a different battle to fight, a different and deeper problem to solve in such case.

In such cases, simply get ready in your area of influence for the future crisis to come, that will hit your product, so you get a chance to become part of the potential solution. Look for allies like the security, software license compliance and technical writing teams for instance, to expand that ring of influence. Hopefully they will see the value and the inventory how it can become a trigger to support modern and widely accepted practices within their domains across the organization.

4.- Summary

The Software Product Inventory is a high level (abstract) concept that will help you to move towards creating trustable software. Since it is part of the development process, following Continuous Delivery principles and practices to design and implement it is essential.

Some of the concepts behind this idea are probably present already in your organization but frequently using a management-driven approach, associated to the release and/or inbound stages, implemented in ways that generates a negative impact (friction) in the product code flow, so throughput and stability.

System thinking and asking questions as if you were an auditor will help you to implement simple measures first, and habits later, that will raise the quality of the product over time, even if the Product Inventory concept does not fly in your team or organization. That approach has helped me, at least.

Logbook, a great practice for distributed teams, long term projects and products.

~ toscalix ~ 2 Comments

Logbook, diary, journal, bitacora… there many names for the same practice: write down what you do. This practice has been used among sailors and scientists for centuries, for good reasons. Nowadays there are plenty of tools to keep track of what you do but I haven’t seen anything as powerful as a team, a project or a product logbook.

Five years ago I wrote an article about my first interactions with this practice, back in the beginnings of my professional life, in the Canary Islands, Spain. The article describes the basics of any team or project logbook. You would benefit from reading it before you keep reading this article.

I would like to provide some additional insides about the diary, together a few tips and practices I have used throughout the years.

Why is the logbook useful

Keeping a logbook is useful for:

  • Engagement: you will know about what your colleagues are doing. They will know more and better about what you do. All in a contextualize manner.
  • Alignment: the logbook will reduce the need of coordination and providing status to your colleagues. It will also help in advancing divergences among team or project members. Managing dependencies better is another outcome.
  • Reporting: the logbook allow to turn the reporting activity into a pull system. Instead of providing a report to your manager, for instance, she can pick up the relevant information from the logbook. A similar thing happens when you need to provide a status report to a customer or another team. A significant part of such reports should be already in the logbook or link to it.
  • Analysis / retrospective: quarterly or yearly reviews are simpler when a logbook exists. Postmortem analysis also becomes simpler.
  • On-boarding: when a new developer joins the team for instance, the diary helps a lot. The newcomer can learn a lot about the team activity, hot topics, who is who etc. by reading it as one of her initial activities.

Who is the logbook for

Everybody would benefit from contributing to a logbook but in some situation or environment, this practices provide significant benefits:

  • Remote teams, specially those distributed across different time zones. The fewer overlapping times, the more benefit you get from the diary. Engagement and alignment become key factors here.
  • Long term projects or products: the analysis / retrospective element makes the logbook a beneficial practice.
  • Teams working on more than one project (multitasking): teams that work in more than one project benefit a lot from having a log book per project and/or a team log book. In this case last case, you might want to consider a per project tag. Reporting is another obvious argument for using a logbook.
  • SWAT teams: engagement is the main factor here. Alignment is also an obvious factor, specially in management teams.
  • Large teams: the benefit here, beside those explained already, is the reduction of the need to provide status in the different team ceremonies, maximizing discussion, retrospective or brainstorming time, for instance. The status information is already recorded on the diary. Go and read instead of repeating already documented information.

Git based vs wiki

I always recommend to use a git-based tool for the logbook. It is not just that collaboration is easier, especially for developers, but also allows to integrate the habit of writing in their workflow easily. It will also be easier to structure and visualize the information through tags. Git is specially convenient for distributed teams too, which are the ones who benefit the most from this practice in terms of alignment.

Often the diary is used by people who does not know how to use git or is not part of their day to day workflow. I have had jobs in which I did not use git on regular basis. In such cases, a wiki can be the best option. Make sure you use a wiki with conflict resolution capabilities. Otherwise, the logbook will not scale. If you use a wiki that structure pages in editable sections, that might work too.

There are tools that combine the best of both worlds, like Gitlab, Github or zim-wiki. These are my favorites.

Structure and archive

I recommend to structure the logbook per day and per user. It doesn’t matter if we are talking about a product/project diary or a team one. Other options are possible but it is simpler to write down your entries one after the other one and use the tags to open the possibility to structure and visualize the information later on in different ways.

To use the logbook as reporting tool, at some point the write permissions should be removed. From that moment on, past entries should be amended in today’s section, not modified in the corresponding date.

When should the write permission be removed?

I suggest to do it weekly by default. I usually lock it down the following Monday at lunch time, providing the possibility to those who forgot to add their entries on the Friday before or were absent for whatever reason, to add their entries.

Project or product logbooks might use the sprint as the iteration, specially in cases where those sprints are two weeks long. But in general, if you have a good number of people writing in the logbook, I prefer a week long iteration.

To archive the diary, I usually move it away from the current one. I personally prefer to avoid scrolling or information from many days where I write. So I use a different file for past entries and current ones (that week). If you are using a wiki, move the content to a different page in read-only mode.

When and what to write?

As mentioned in the original article, you should include in the logbook what is…

  • Relevant to you.
  • Relevant to your peers.
  • Relevant to those you interact with like, managers, other teams, customers, etc.

Write down all the relevant events or any relevant information as soon as you have it. Write down what has been completed, solved, blocked… State facts and ask questions instead of writing opinions. Leave the judgement for the later analysis of the logbook. Keep them out of the logbook entries unless you create a specific tag for it (#self-note could be an example).

I add an entry at least every couple hours, before taking a break, except when I invest a longer period of time in an intensive task. So I usually have four entries or more daily in my logbook. I tend to write a lot more though. Other people write less than I do and that is OK as long as each one of us write down the relevant information for ourselves first, for our colleagues and finally to others.

Include decisions and agreements, conclusions, achievements, pitfalls, mistakes, external events that have influenced your work, external references and data sources that were relevant, etc. But remember, the journal is about what you did, not about what you need or should do, what is coming or what you think. Later on, with practice, you can expand the nature of the content. Make sure you agree worth other writers on this topic. It helps to keep the logbook clean.

One special type of entry is a comment on somebody else’s entry. I recommend to use it at the beginning only to add information or additional context to somebody else’s statement, not to provide any opinion or question. This is a journal, not a forum.

Ask yourself this question, what would I like to read about what any of my colleagues or myself are doing in 3 weeks or 3 months time?

How should I write?

Be concise. One or two lines maximum per entry is the best approach. Add links to the information sources, to the tickets, messages, bug reports, patches, logs, web pages… where the information is really generated and stored. Provide context to those.

Use ways to shorten the links to those common tools you use at work on daily basis. Add tags like dates, names and any other metadata that can help to contextualize and structure the information later on.

Remember, whatever you do, make it simple so adding information to the logbook is easy and fast.

Tags

One of the key elements of the journal is the capacity to structure and visualize later on the information through tags. As mentioned, the logbook is structured by date and user (two tags), but there are others you can and should use. The more experienced the team or project is with the logbook, the more tags can be used.

Warning: agree on the tags with your colleagues up front, otherwise you loose part of the filtering and visualizing of the information capabilities later on. If there is more than one logbook at your organization, agree on the tags with the other teams and projects. Define common tags.

I recommend to start with very few tags and increment their number over time. The tags to start with might depend a little on the environment, the type of team or project. These are the ones I commonly start with:

  • date and user
  • Mood: 😀 , 🙂 and 😦 Use these tags in entries but specially by the user tag, to let others know how you feel and to be able to evaluate later on your mood in different time windows.
  • Escalation: #red #amber #green Agree with your team and/or manager the meaning and reaction to each one of these tags.
  • Absence: #sick and #vacation Use them by the user tag to tell others that you or your colleague are absent.
  • Others: #decision, #agreed and #self-note

The tags are usually located at the end of the entry. There is one exception. When you are commenting on somebody else’s entry, start with the user tag and some indent. Check the example below for this case.

Different tools might have different restriction about how to define a tag. Some have restricted keywords. I added the symbol “#” because it is a common one. For users, many tools use the “@” symbol, for instance.

Use standard emojis instead of tool specific ones. Again, make it simple. Consider emojis a tag.

Example

This is a simple example of how the journal would look like:

May

02/05/2020

@toscalix 🙂

  • Blog post finished. Referenced previous entries.
  • Could go out to do exercise for the first time in over 50 days of confinement. I will go back out again tomorrow. #decision
  • A new tag #off-topic has been created #agreed

@peter 😦

  • Blog post reviewed. Ready to publish #decision
  • Network issues again. Third time in two days #amber
  • Send any policy to @toscalix before approval for review #self-note

Conclusion

Keeping team, project or product logbooks is a very useful practice. In the short term increases engagement and alignment, in the mid terms reduces the effort in reporting. In the long terms keeping a journal improves analysis and retrospectives. on-boarding is another topic that heavily benefits from the existence of a diary.

I would like to know about your prior experience with logbooks or similar practices. If you want to try it out, feel free to contact me for questions or advice.

The Remote Journey: references to start

~ toscalix ~ Leave a comment

Introduction

I started my professional career in an archipelago and I have been involved in Open Source for years so managing remote software related teams, departments and even organizations has been the default for me. I have been also working as consultant in a remote-friendly environment and now I am working at MBition remotely. I believe I am familiar with many aspect of the The Remote Journey, which is a topic I am interested on beyond my work, since it is tightly related to the way of life I want to live.

Remote work is a fairly mature topic at individual (software development), team and department level. It is maturing at company level too which means that there are already resources in internet that will cover most of the basic questions and topics that most of the companies struggling today with moving from colocated directly to remote-only environments might have.

This forced shift will have a major impact in every aspect of the company, in you as professional and in the way you understand your work, which is why I strongly recommend that you make yourself and your colleagues aware of the challenge and embrace it, which means to me at least two things:

  • Be ready to challenge most of what you currently know about how you work, how your colleagues work.
  • Be open to learn. Read and talk about what you and your colleagues are going through assuming that no, you do not have the situation under control. You will need to learn again what under control means.

The good news is that this worldwide crisis will change the way we all see remote work, hopefully for the better.

The Remote Journey

The journey from being co-located to a remote-only environment has different stages. There is no agreement on how to name them but in general, I will define them in the following way:

  • Co-located: teams/departments are located in the same physical location (office).
  • Distributed: teams/departments are located in different physical locations (office).
  • Remote-friendly: the workforce of the organization can work at a different location than the office part of the time. A mature remote-firnedly environment has a minority but significant part of the workforce working remotely and coming to the office frequently. Usually these workers are related with sales, support, business development… When it comes to product development or services, those remoters are senior professionals with wide experience in remote working so they can overcome the technical, process and cultural gaps they face on daily basis due to living in a colocated culture.
  • Remote-first: most of the workforce works remotely. The office is usually reduced to specific areas of the company like labs, administration, HR, junior developers… Mature remote-first environments usually have their workforce distributed across different time-zones/countries.
  • Remote-only: there is no office or when there is, working from it is voluntary. Employees are supposed to work from home/coworkings, including supporting services and departments like admin, HR, etc.

You can read a little about these definitions here:

There is one undeniable fact though, remote-only organizations not just exists, they are successful. In my opinion, it is up to each organization how they want to transit through this journey and which stage is their target. Obviously the current crisis has left many companies with no choice but to jump most stages and go directly to becoming a remote-only organization for a while, but still they can learn from other people journeys. There are plenty of additional articles about the different stages and how routines, processes, methodologies, performance, evaluations, etc. are affected at every level (individual, team, dept. and organization). They should be easier to find now that you know some of the nomenclature.

Team ceremonies need to adapt

It is my belief that in general, habits change mindsets instead of the other way around. When walking through The Remote Journey together with teams and organizations, I put emphasis in the ceremonies as a way to drive the needed change at every level: personal, team, department and organization. If you successfully adapt the ceremonies, your are in a great position to modify people’s habits.

Personal ceremonies are that, personal. I will not get into them. There is plenty of literature in internet about how to face remote work, the advantages, the challenges and how to approach them. I have my own routines. They are not static although some of them have been with me for some time now. Some have been affected due to the confinement state we are in right now in Spain so I am adapting them to evaluate how they work. My advice in this regard is that you read about other people routines, identify yours, track them and experiment to find the right combination. Again, assume they will evolve over time.

I have written in the past several articles about team ceremonies. These articles have helped me to explain certain basic topics. You will need to find the routines that work for you and your colleagues though, in the same way that it happens at individual level. The articles were written thinking mostly about team leads at any level but hopefully there is plenty of useful stuff for team members too:

  • Working in distributed / remote environments 0: presentation.  Motivations, introductions and some definitions. The article includes the link to the rest of the series.
  • Working in distributed / remote environments 1: daily short meetings. Tips and recommendations about adapting the team daily meeting that is so popular on colocated environments.
  • Working in distributed / remote environments 2: the calendar tool. Comments about the increase of relevance for any team that the calendar has in remote environments together with some tips on how to use it.
  • Working in distributed / remote environments 3: weekly meetings (I) and (II). Teams meetings are an essential ceremony to drive change, detect issues early, solve conflicts. They are essential in remote environments. These two articles provide an overview of how relevant they are, why and how to adapt them to the remote nature of the team.
  • Working in distributed / remote environments 4: one on ones (1:1s). in co-located environments, 1:1s are not perceived as a priority by many. Only when the organization grows beyond certain point, this ceremony gets the attention that in my opinion it deserves. In remote environments you cannot wait to get “big enough”. The nature of the work environment force you to establish proactive measures to align, define expectation based on company, department, team and individual goals and evaluate, together with the workforce, progress. The articles provide tips to adapt 1:1s to a remote environment.

You will see that in the articles I use the term distributed and remote environments (DRE) to avoid referring to the different stages of the journey. This is for simplicity. Ceremonies might slightly change depending on the stage the organization is at.

Companies

It is always good to have references, right?

Remote-first and specially remote-only companies need to pay an extraordinary amount of attention to company culture. They usually provide plenty of resources to their employees about this topic. These organizations start hiring experienced remoters at first but as they grow, they realize they need to educate their workforce in remote working, which requires the development of contents. These two links might be a good starting point to find the right companies, what are they doing and why:

  • FlexJobs is an online job board specialised in remote work. They publish the main list of companies walking through their Remote Journey regularly.
  • If you are focusing in tech companies culture, you probably will prefer this article.

Reports about remote work

There are three interesting reports I recommend to read if you like the remote work topic:

I have used them in the past to open conversations about this topic with managers and HR departments, for instance.

Books

I have to admit that I haven’t found yet THE book about the topic, and I have been searching for years. This is the one I recommend:

  • Remote. Office Not Required. It is from 2013 but still (sadly) the best. It is based on the experience of 37signals (now Basecamp). Their authors have accumulated an extensive experience in remote organization since then.

If you are not into buying books (what?), this is an online free book written by Zapier, a popular company in this field.

Social media

  • Twitter: follow the hashtags #remotework #workfromhome . Besides plenty of advertisements from coworkings, you will find useful resources once in a while as well as blog posts.
  • Other social media references: those of you interested in digital nomads or digital travelers, can follow #digitalnomad hashtag on Twitter. I joined some time ago the Digital Nomads Telegram channel.

Events

There are really good ones out there to learn about this topic:

  • The Remote Work Summit: a remote event with many interesting talks and material. You can get free passes to some content and online talks.
  • Nomad City: this event takes place in Gran Canaria, Spain (in English). It is a great one to meet digital nomads, digital travelers and remote workers as well as remote organizations leaders.
  • CoworkingEurope: it is not directly related with remote work but with flexible working spaces, but you can find useful references to companies and processes to follow from there. I have worked in coworking spaces at different locations around Europe. They are a great source of remote work knowledge.
  • I have been the last couple of years trying to join the Nomad Cruise. Let’s see if this year…

Summary

In my experience, going from colocated to remote-only environments changes way more things that you can expect at first. Keeping high levels of efficiency, alignment and workforce satisfaction requires time and effort. Do not underestimate them. The good news is that although not at the speed this crisis is forcing many to do, plenty of people and tech organizations have experienced such transition. Some have published lots of contents about their experiences moving through The Remote Journey or living and growing at a specific stage. Look for those content in internet. Hopefully they are easier to find after reading this article.

A single man experience is very limited though. You probably have experience too. Please share it as well as links to further content.

Interview candidates with an Open Source background

~ toscalix ~ 3 Comments

I often say that there are two actions that define the line management role: one-on-ones and hiring people. This is specially true in growing organizations. If you nail these two actions, you have a good chance to truly influence, the ultimate goal, in my view, of any line manager.

One of the common strategies to speed up the journey from being an Open Source contributor to becoming a Good Open Source citizen is to hire talent with a solid Open Source background. The process of hiring such talent is different from what most organizations and recruiters are used to. That is so true that we have now companies specialized in hiring these profiles.

One key part of the hiring process is the interview, obviously.

This article is another one of those I have written the last couple of years about management topics, based on my experience working in Open Source as manager and consultant. More specifically, it is an attempt to describe some of the key points that hiring managers with little or no experience in hiring Open Source talent need to consider to increase their hit rate.

As usual, I would appreciate if you add in the comments section or to me directly your experience, criticisms or missing points. I would add them to this article as update.

Evaluate the candidate’s work in advance

If you only have room in your hard disk for one takeaway, it should be this point.

Most of your candidate’s work is public. Take the time to evaluate it or have somebody else doing it for you, before the interview. If the candidate have public talks, watch the videos or check the slides. If she has written articles, read them. If she has code available, study it, test it, install it… .

Having the candidate’s work in the open is a double edge sword. It is an advantage for the hiring organization because it can reduce significantly the hiring process, since you can evaluate in detail many of the candidate skills in advance, so the interview process can be reduced to a single interview, for instance (I strongly recommend this, by the way).

At the same time, if the hiring manager do not take the time to evaluate the candidate’s work or even worse, if the manager acts during the interview as if that work is not public, trying to technically evaluate the candidate prior or during the interview, for instance, the failure is almost guaranteed.

I always mention how wrong it is to ask a music group with published records or videos of their concerts available on the group’s website if they know how to sing. Or even worse, to request an audition during the hiring process for your birthday party. Who does that? Instead, watch their performances, skip the audition and go directly to negotiate other relevant points. Their work is public, did you check it?

By the way, the world is significantly larger than GitHub, thanks God. The candidate might have a strong Open Source background without having a great GitHub profile… or no profile at all.

Interview goal

The interview is not just an opportunity to evaluate if the candidate fits in your organization, it is a chance to attract that professional and her colleagues. Remember, the candidates code, processes followed, tooling used… are public. Your organization’s most likely not.

Who has more to inform about?

As result of this reality, the interview should look more like hiring a senior manager/expert than about hiring an engineer.

In summary, the goal of the interview shifts. It should be balanced between what both, the organization and the candidate looks for. This shift represents a challenge for hiring managers with limited experience in hiring senior talent.

Candidate’s future upstream participation

During the interview, be clear about the amount of time the candidate will have to invest in upstream work, specially if he candidate is working upstream already a significant amount of time.

There are plenty of developers who prefer to keep their work in the open as a personal activity. On the contrary, some others want to make a living out of their participation upstream. The best way to ensure a sustainable relation over time is to be transparent about the expectations related with the effort put in internal work vs work done in the open as well as in which projects.

If by working in your organization, the developer will have to reduce her current involvement in the open, specially in specific periods of time, say it clearly. If you cannot guarantee that the technologies the candidate works with in the open will be part of the company’s tools set, state it clearly. If your organization has plans to move towards Open Source and you bring the candidate to speed up that journey, make sure she understands the timelines and expectations.

No matter how experienced you are as hiring manager, there is always room for a candidate to surprise you about her motivations to participate in the interview and potentially join your organization. This is most likely true with professionals coming from an industry you might not be very familiar with.

Tooling, languages, technologies…

One of the great advantages of being an Open Source professional is that, because you use open source tooling, you can take them with you throughout your career, which makes you very efficient. Professionals with a solid Open Source background do not depend on certifications paid by their companies to learn how to use their tools, or having to change their base tool-set regularly, limited by commercial decisions. In other words, Open Source professionals have made their career supported in specific tooling and associated practices they have chosen and mastered. Often, they have even helped to develop or polish those tools. Some of them love to try new tooling on regular basis and some others love to focus their energy in other aspects, not in the tooling or associated processes, so they are conservatives.

In some cases, this specific point is so important for the candidate that using those tools and associated practices represents a requirement to join your organization.

You will need to consider and potentially embrace this reality. If you cannot or do not want to use those technologies or tools, make the point during the interview. But before you do, I recommend consider the following…

Are you open to reconsider such decisions if somebody prove there is something better? Is it something that can be reconsidered next year, when the current support contract for tool X expires? Do you have any specific department or project where you are willing to try new tools, processes, languages…? Are you open to assume that you might have made a mistake in such decision or that those decisions will not stand the test of time?

Many Open Source professionals, for instance, consider that, doing Open Source with Open Source tools is significantly better that using proprietary ones. Why?

At scale, adapting the tooling to the needs of the specific environment and integrate them with other tooling to improve the development experience is essential. Even more that to use a feature rich tool. To have full control of the tool’s evolution in your specific environment is another interesting point… there are many other strategic and practical reasons that leads to that mindset.

Learn from the candidate about those arguments during the interview.

Very often hiring managers see the candidate’s position in this front as religious or philosophical. Instead, I would take it as an opportunity to improve and innovate within their organizations. I suggest to embrace such experience the candidate can bring to your organization as starting point during the interview and take the conversation from there, instead of questioning her criteria and define this point as a no-go early in the interview or the overall process.

Recommendations and referrals

Open Source is about trust and networking, based on the work the potential candidate does within one or several communities. In many cases, the candidate recommendations and referrals come from peers from those communities.

If you are a hiring manager, take those recommendations as seriously as any traditional referral from a senior manager or a relevant figure in any big corporation. They are frequently equally valuable.

In Open Source, you are judged by your work and your influence, not by your position in a hierarchy of any organization.

Get help for the interview

If you have little or no experience with Open Source, bring somebody that do have it to the interview, in addition to a technical expert, specially if you are interviewing engineers. It is not relevant if such person is not an expert in the candidate’s knowledge area of expertise.

I am not a technical person and, when I join such interviews, the hiring manager often learn new things about the candidate because we both (candidate with solid Open Source background and myself) share a similar working culture. This is specially true when the candidate starts to ask questions to evaluate if the company is a good fit for her. In addition, it often plays in favor of the company to show to the candidate that there is Open Source knowledge internally and peers that feel comfortable working there.

Assume the candidate knows about your organization already

Do not underestimate how much candidates with solid Open Source background know about your company before the interview, even if your organization is not an Open Source contributor.

These kind of professionals might learn about your organization not just by interacting directly with your engineers or evaluating your products, but also by interacting with your suppliers and partners in the open, in some cases on regular basis. They might know about the hardware you use, the versions of specific software you ship, tools you use, release strategies you have, updates policies, etc. They might even point at clear deficiencies and strengths of your products and services as well.

I suggest to ask the candidates early during the interview what do they know about your organization and why do they want to join it. It is also interesting to ask about their evaluation/perception of your organization as a whole and your products/services compared to others.

It is not only about getting information from the candidate but also about informed perceptions. Such conversation might help hiring managers to tune the description about what the organization does and avoid providing superfluous information during the interview. It also helps to understand if the candidate is one of those who want to work in industry leaders or help you to become one, which is something I am always interested to learn about them.

Why do you need a CV?

Even today, I still come across former colleagues and great Open Source professionals who do not have a CV. I have helped several of them throughout the years to create one or update it.

For an Open Source professional, their work speak for themselves. It is a better presentation that any CV will ever be, in my view. It is the outdated business culture the one that should change and adapt to this “not so new” way of “filtering candidates”. It is simply better for all parties involved.

Many great Open Source professionals have a mediocre CVs or no CV at all because they never needed it. For some, this lack of CV is a way to filter organizations they do not want to work with. Their thinking goes in the line of… if you need my CV to filter me as candidate or evaluate my work, I am not interested in working with you. Check my work!

Instead of questioning the quality of their CV, take the opportunity to ask yourself why do you or your organization need a CV when the candidate’s work is publicly available. It might be a great introspective exercise. Maybe you end realizing that only a bunch of links and his contact information are really needed, instead of a formal CV.

Myth: Open Source developers are not good communicators

Open Source professionals, specially developers, are used to great levels of exposure to other professionals worldwide. In general, they need to become good communicators to succeed within their communities.

Open Source developers as people incapable to communicate is a myth. Such perception often depends on what you understand by communicating and to whom, but specially it is based on myopia often from people with a limited technical background.

It is true in my experience though that in general, many Open Source developers do not do great interviews. But that is mostly because they do very few of them. They do not go to 10 interviews with four different companies to get a job. They do not need to. And what is more important, throughout their careers, interviews are not used to evaluate them technically, again, because their work speak for themselves.

Evaluate the candidate’s written communication skills. Check the mails they send to dissent with others or describe their position, read the blog posts they write summarizing their latest work, check their presentations, their slides… Check how they interact with their users, code maintainers… You will soon realize they interact with profiles or groups of people on regular basis that many corporate engineers do not. And those interactions often face ore limitations that within any corporation, due to cultural differences.

When talking to managers unfamiliar with Open Source about this point I always ask them how many professionals they know that having done good interviews, are still incapable of standing behind their opinions in a meeting with senior colleagues or customers. Or how many people in their teams excel at describing complex ideas in simple words (in a 10 min presentation, for instance).

Public exposure to peers and users in Open Source projects are an outstanding learning environment for developing certain communication skills. Be careful with this myth.

English level

In Open Source, most of the communication among developers take place in written form. This is due to the distributed and asynchronous nature of the environment they work in (software development at scale). Most of those developers have a better level of written English than spoken.

I always recommend to be careful when evaluating the candidate English skills only through the interview. To have a full perspective of the candidate skills, I always recommend to check her written communications and articles. If they have videos of their talks available, review them. You might perceive noticeable differences between their written and spoken English levels.

When doing an interview, it is very hard to be quick and provide sharp answers without thinking when you do not speak English on daily basis. I have experienced this myself.

My point here is that you might evaluate her English skills as poor based on the interview when, in many cases, they have solid written skills, so they need little time to develop the minimum level required to work in English.

Agile fans tend to underestimate how important it is to have a good written level in English compared to the spoken/conversation level. In Open Source (develop software at scale), the trend is exactly the opposite… for good reasons.

Remote Work

Many professionals prefer to live by the beach, in a cheaper place, than to commute for an hour to go to the office in a big city. At the same time, when developing software at scale, it is unrealistic to think you will find the talent you need in a single location; even in two different locations. It should not come to a surprise that the market is moving towards developing software in distributed environments where remote workers are not the exception anymore.

The number of “remote first” and “remote friendly” companies is growing in number and size at a fast pace. Open Source is about highly distributed, asynchronous and high latency environments. Some of the larger software engineering projects has grown and mature with those characteristics by design.

The candidate has probably made a career out of mastering working on such environment.

Instead of questioning the efficiency of distributed teams during the interview (common practice among remote work detractors, like radical agilists and traditional managers), take the opportunity to learn about it.

If you think that co-located, synchronous and low latency environments are more efficient, ask about how current tasks at your company could be performed remotely, how ceremonies can be adapted, how testing can be performed remotely. Learn from the candidate about how to develop software at scale, how he has acquire such experience and levels of efficiency in such environment.

I have worked in organizations where testing was done remotely, in a heavily distributed environment, for instance. Technology and creativity have made Open Source projects the best environment for developing software at scale. Even if your organization is small, the candidate skills might become an asset.

The network effect

Working in the open provides any professional the possibility of building a large network. That interview might well be the first and only direct interaction that such professionals might have with your company. If the experience is not satisfactory, her network will know.

Would you recommend a person you trust or respect to apply to a company that did not provide you a good experience during the hiring process? And I am not talking about hiring them. I have had good experiences with companies that chose a different candidate during the process. We are grownups. We can take rejection.

By ignoring the point on this article (or others i skipped), you are not just simply making harder for your organization to hire Open Source talent. You are shouting with a speaker your deficiencies.

It is not a coincidence that some corporation with pedigree have such a hard time hiring good Open Source professionals while some others less popular are so successful. It is partly because of the networking effect.

Do it right a couple of times and more talent will come. On the contrary, fail badly and you will pay the prize for some time.

How to move the candidate out of her comfort zone

Part of any interview’s goal is to get from the candidate things she did not plan to show or surface.

People used to work in the open, with high levels of exposure, develop a great sense of correctness and politeness when expressing opinions. That does not mean they are not passionate about what they do or that they do not have “radikal opinions”. They are and you want to know them.

Ask the candidate for her opinions about tools, technologies, projects from your competitors or technologies that the community project they participate on are struggling with. I believe these kind of questions are a fair way to move the candidate out of her comfort zone and get their “essence”.

But when you do, be aware about the trends and debates the candidate and/or the community she participates on have had about such topics. Be careful with making statements that are a no-go for the candidate simply because you did not prepare the interview well enough.

Avoid the typical myths around Open Source, the stupid arguments about battles between communities, superfluous statements about Open Source business models and their feasibility, etc. Instead, ask about, for instance, how do they see their community project in five years, what are the challenges they are facing, what are the deficiencies they need to overcome, etc. Do your homework about Open Source and the specific community the developer is involved in before the interview. Nobody wants to work for a person that seems a fool or lazy.

Again, if you do not know about these topics or had no preparation time, bring somebody with knowledge to the interview or state your ignorance up front.

Summary

Radical exposure to peers and users, development of public code, knowledge share with professionals from different industries and global organizations, etc. are characteristics about the candidate that cannot and shouldn’t be ignored during the hiring process, specially during the interview.

Instead of applying the same template that has worked fine in the past for your organization, hiring managers will need to evolve it or even change it in order to attract people with solid Open Source background. This is yet another of those transformations and culture changes that any organization will need to go through to fully embrace Open Source.

To me is very simple, if you want to become a Good Open Source Citizen, you will need to hire people with a strong Open Source background if you intend to speed up the journey. Either you adapt your hiring processes, including the interviews, or you will most likely fail in such goal.

To adapt, hiring managers and experts need to develop new habits. That can only come if they question the current ones. This evolution will take place faster and with a smaller impact for their organizations if they get help.

I think this topic has gotten less attention that it deserves.

Contributor License Agreement and Developer Certificate of Origin references

~ toscalix ~ 6 Comments

In the last few years I have come across the CLA topic several times. It is and will be a popular topic in automotive the coming years, like in any industry that moves from being an Open Source Producer towards becoming an Open Source Contributor.

In my experience, many organizations take the CLA as a given by looking at the google, microsoft or intels of the world and replicate their model. But more and more organizations are learning about alternatives, even if they do not adopt them.

What I find interesting about discussing the alternatives is that it brings to the discussion the contributor perspective and not just the company one. This enrichs the debate and, in some cases, leads to a more balanced framework between any organization behind a project and the contriibutor base, which benefits both.

Throughout these years I have read a lot about it but I have never written anything. It is one of those topics I do not feel comfortable enough to write about in public probably because I know lots of people more qualified than I am to do so. What I can do is to provide some articles and links that I like or that have been recommended to me in the past.

Articles

  • “Why you probably shouldn’t add a CLA to your open source project“, written by Ben Alter back in 2018 is a must read. Ben is a United States-based lawyer who works for GitHub as their “evangelist” to government agencies.
    • “CLAs are Not a Sham” by Kyle E. Mitchell, a lawyer, is a respond to the previous article.
  • “The trouble with Harmony: Part 1 and Part 2″ by Richard Fontana 2011, a Red Hat lawyer. He came back to the topic this year (2019) in his article “Why CLAs aren’t good for open source” summarising Red Hat’s position.
  • “Why Your Project Doesn’t Need a Contributor Licensing Agreement” by by Bradley M. Kuhnon June 9, 2014. Bradley is a well known member of the Software Freedom Conservancy.
  • “Contributor Agreements Considered Harmful” by Eric S. Raymond on July 8, 2019, represents a clear view from the individual contributors perspective on the CLA topic.
  • “Community contributions and copyright assignment” By Jonathan Corbet, October 28, 2009. This article analyses from the contributor’s perspective the Canonical Ltd CLA which started a heated debate back then, leading to the Harmony project creation first and the initiative by the Mark Shuttleworth Foundation later on, two of the main attempts to standardise CLAs up to date.
  • “Some thoughts on Copyright Assignment” by Michael Meeks, 2011, is still one of the most relevant articles on the topic, covering not just the theory but also putting examples of how harmful unbalanced relations with an entity that controls an Open Source project might be for contributors. Specially relevant has been over time the Recommendations section of this article.

DCO

It is impossible nowadays to talk about CLAs without talking about DCO (Developer Certificate of Origin). Here are some articles that I find interesting:

  1. Ben’s Cotton article “CLA vs DCO What’s the difference“, from 2018 is a good article to read because it does not favour one over the other one.
  2. I like the article “CLAs and using DCO clearly” from Hen written back in 2018 because it summarises well the costs associated to CLAs and how they (do not apply) apply to the DCO.
  3. It is worth reading the summary of James talk at Linux Foundation Collaboration Summit 2014, “The most powerful contributor agreement” done by Jonathan Corbet.
  4. Julien Ponge wrote “Developer Certificate of Origin versus Contributor License Agreements” back in 2016.

For companies that “do not buy into the anti-CLA” case, R. Fontana propose another two options:

  1. Eclipse Contributor Agreement
  2. The Software Freedom Conservancy’s Selenium CLA, which are not proper CLAs but DCOs in his view.

FLA

It is always interesting to learn about the Fiduciary License Agreement FLA, developed by the FSFE:

  1. Fiduciary Programme
  2. FLA FAQ
  3. FLA-2.0 announcement.

Inbound = outbound

A serious conversation about CLA requires to understand the concept of inbound=outbound:

GitHub explains inbound=outbound as:

  • “outbound licensing”, refers to granting a licence to another party to use, copy, distribute…. (depending on the license) your intellectual property.
  • ”inbound licensing” means obtaining a licence from another party, to use, copy, distribute…. (depending on the license) its IP for your own consumption (distribution, etc. again depending on the license).

GitHub explains inbound=outbound as:

“Whenever you make a contribution to a repository containing notice of a license, you license your contribution under the same terms, and you agree that you have the right to license your contribution under those terms.[…]“

In the CLA discussion context, the general idea behind inbound=outbound is that the project should make evident to every contributor the contribution conditions, including those related with licenses rights and restriction. The contributor will then contributes her code with the same license, giving little room for later claims, either by the project, the contributor or third parties, based on those conditions being unclear or not easily reachable.

The project license and its description should be prominent and located at least where it is common practice in Open Source to find them. The same applies to all the assumptions and conditions affecting the contributors and the project in this area.

Common practice, many will claim, but sadly some projects are better than others on this.

Who is doing what?

In CLA discussions, it helps to know what others are doing. Here you have some links I have collected over the years and I still find relevant. I have to recognise that in a couple of cases cases I do not remember exactly why they called my attention at the time.

Company driven projects:

Consortium driven projects:

Community driven projects

Organizations with different agreements for individuals and entities:

Organizations with CLA:

Additional links

And finally, I have several links that I think are worth reading for different reasons:

  1. Project Harmony is worth investigating as an interesting attempt to standardize CLAs:

2. GitHub uses an Individual and an entity CLA. It is interesting their CLA-assistant.

3. James Bottomley’s ideas about the Corporate Contribution Pledge published back in 2016 as complement to the DCO are worth reading.

Additional suggestions from readers

I hope my effort triggers the contributor in you so you provide additional links or challenge the ones above. I will substitute this text for those links you provide, obviously giving you credit by default. It would be a great way to help others in the future. Thank you very much in advance.