Embrace Open Source culture: the 5 common transformations.

Article originally published at Linkedin on May 15th 2016.

This is a story of what I have lived or witnessed a few times so far. A story of an organization that used to consume, develop and ship proprietary software for many years. At some point in time, management took the decision of using Open Source. Like in most cases, the decision was forced by its customers, providers, competitors… and by numbers.

A painful but unavoidable transformation was required.

1.- Open Source consumer

Engineers had to learn a new system, adapt or re-write those features that used to made the organization unique, together with many other painful actions. It was expensive at the beginning but, due to the cost reduction in licenses and the change that Linux represented in the relation with providers, in a few years it was clearly worth it. And if it wasn’t, it didn’t matter since it was what the market demanded. There was no way back

Every software organization has gone through their unique journey, but the final sentence of the story has been the same for all of them: they became Open Source consumers.

2.- Open Source producer

This organization gained control over its production and, by consuming Open Source, it could focus many resources in differentiation, without changing the structure, development and delivery processes. At some point, it was shipping products that involved a significant percentage of generic software taken “from internet”.

It became an Open Source producer.

You can recognise such organizations because they frequently create a specific group, usually linked to R&D, in change of bringing all the innovation that is happening “in the Open Source community” into the organization.

Little by little this organisation realised that giving fast and satisfactory answers to their customer demands became more and more expensive. They got stuck in what rapidly became an old kernel or tool chain version…. Bringing innovation from “the community” required back-porting, solving complex integration issues, incompatibilities with what your provider brings, what your customer wants.

So they have to upgrade.

This organization will be able now to take advantage of all the common features and compatibility that the new kernel, the new tool chain… brings. But, guess what, forward porting all the differentiation features this organization has developed, all the bug fixes, is so much work and so complicated that the challenge put the organization at risk.

3.- Open Source contributors

The organization feels now the downsides of becoming a blind Open Source consumer and producer. Execs feels like when a bubble explodes and they are inside of it. They has less control than they thought, which turns out to be expensive, and what is worse, they lack the expertise within the organization to gain it….

But, after struggling for some time, this organization survived, which means that it has learned some lessons:

  • Upstream those features that are not differentiation factors any more.
  • Increase the investment in that reduced groups of rock-stars that are up to date of what’s going on “in the different communities”.
  • Invest in those Open Source projects that develop the key software you consume.
  • Even better, start your own Open Source project to promote your technologies and be perceived as a leader…
  • Reduce the upgrade cycle, so the “upgrade pain” is lower. As a side effect, the organization has the opportunity to increase the cash flow when doing two smaller upgrades instead of a big one. At the very end, the real profit comes with the first update, not with the new version, right?

This organization ended up upstreaming features when they could, normally very late, because “they do not have time”, frequently assigning that task to young inexperienced developers or, even “better”, subcontracting it, which is “cheaper”.

You can recognise that this organization has gone through the described process when attending to conferences given by any of its executives. They cannot stop talking about how much they contribute to this and that community, about how awesome the community is, how important it is to be open and share…. they are referring all the time to communities/upstream as “us” and “them”. They think they got it, they really do.

Most of them believe they are in the crest of the wave after going through this third transformation process. They are innovative, they has been able to reduce their time to market, they are gaining reputation within a variety of communities… They are not just Open Source consumers and producers any more. They are also contributors. Some of them even heavy and “successful” contributors.

But if you look closer, they have not adopted “the Open Source way”.

This organization keeps their traditional processes intact. It is managed in the same way. Decision processes are taken like when it was a proprietary company, it has not improved transparency significantly, it does not share code and practices among departments…. there is a totally different reality in front and behind its firewall, between production and R&D, between management, engineering, customer support, etc..

This organization face friction because of this reality. It still cannot move fast enough. Upgrading is still too expensive since now they have to do it more often than years ago, upstreaming goes so slow, when it happens. They cannot control the communities they are investing on…

So going through a forth transformation becomes unavoidable. Some refer to this transformation as “upstream first”.

4.- Upstream first or becoming a good Open Source citizen

This fourth transformation basically means that upstreaming is part of your development process, not an aside task. It also means that communities are part of your delivery strategy, not an after market topic, that R&D is a two way road where you do not just consume innovation created by others but you share yours, not just “promote it”. You really need to get involved.

This organization will learn that by becoming more open, their engineers learn more and faster, so the organization itself. It is at this stage where the organization really understand where the real value is in the software they produce compared to what is commodity…or that is what its executives and managers most likely believe, once again. 🙂

But open source (no capital letters any more) is not about being open, but about being transparent, which means that is not just about seeing what is behind the glass, but also understand it.

I believe the fictional organization I am talking about will have to take one more step, the fifth one. It will be about “becoming upstream”.

5.- Becoming upstream or being an open source organization

This is about understanding that, if you consume, produce and contribute Open Source, the smart thing to do is becoming an open source company. I think it is naive to pretend taking full advantage of  Open Source while keeping your traditional corporate culture, which collides with the one of those who produces most of the software you consume and ship, who are your “upstream”. You are building your business on top of them. Since you cannot control them, become “them”.

The smart thing to do is to surf the wave, not fight against it, generating friction. Any manager knows that friction is expensive, reduces focus and drives away talent.  It is bad for the business.

The required culture change to succeed in this fifth transformation involves thinking less about us (company and customers) and them (community), and more about us (ecosystem). It can’t be any more about upstream and downstream but about technology and service. It has to be less about upgrading and more about updating, less about “manage” and more about “lead” at every level of the organization, not just referred to execs and managers.

It is a transformation in which engineers are empowered, where management is more focused on collecting information for execs instead of producing it, and after decisions are taken, their key focus is alignment. A transformation in which execs get closer to where the real value is, to people, because they are the “masters”. A transformation in which engineers not just follow, they get exposed, they take responsibility and assume the consequences… getting paid for it.

An environment in which accessing to key information does not depend on your position within the organization chart, which means that power does not depend so much  on what others ignore, but decisions are taken based on shared knowledge. A culture in which transparency is the norm not the exception.

In summary, a transformation that leads to a stage in which the organization steers its ecosystem instead of driving it. So it leads it in a sustainable way.

A quimera?

I understand it might sound like a quimera, but:
  1. No more than it would have sounded 15 years ago any of the stories that so many CEOs or Open Source Program managers from leading corporations are telling nowadays in popular FOSS events about “their transformation”.
  2. I do not think the debate is if this fifth transformation will be needed, but about when and how to go through it.
  3. My +10 years of experience in Open Source and +17 as manager tells me that, waiting to face any of the first four transformation processes until you have no choice is an unnecessary risk. I suspect the same will apply to the fifth one.
So my message is,
  1. Consume, produce and contribute to open source being a good citizen.
  2. Embrace Open Source culture… better sooner than later.

 

Apply agile methodologies to upstream development environments…. if you can.

 Introduction

When the Agile Manifesto became popular and based on them, agile methodologies like Scrum, XP or Kanban, upstream development was in its early stages as collaboration ecosystems of companies.
Only a few for profit organizations embraced developing upstream back then. Most of them were small and heavily influenced by FLOSS engineers vision. Free software communities were basically driven on personal basis or the very lucky ones, together with “sponsored developers”. In general, these ecosystem were not part of companies strategies.
Today, more and more companies are getting fully involved in community projects as stakeholders, not just consumers or simple contributors.
They frequently start as consumers, then, little by little they become “upstreamers”, that is, they share/publish their code with the goal to have it merged (upstream code). Not without effort, many of them become successful contributors.  After some time, some of them end up understanding that is “cheaper” to play by the project rules. In summary, they learn to become good citizens.
A subgroup of the above companies end up including these collaboration ecosystems as part of their own strategies, going from contributors to  key stakeholders. A necessary step to achieve this goal is to work upstream.
Walking this path present many challenges. One of the toughest ones is related with the differences in development methodologies used internally (mostly agile) and those used in the collaboration ecosystems.
There are two fundamental variables that, in my opinion, determine this challenge:
  1. Environment
  2. Culture

Challenges

1.- Environment

There are two dependent variables that were not taken into account (or just partially) when the agile methodologies were defined, that are relevant in upstream development:
  1. Community projects are global environments, that is, contributors are located in different “offices”, frequently in different time zones.
  2. Probably due to the original amateur condition of early contributors, together with the “distributed condition”, the development processes (so the tools) in most mature community projects, consider, manage and tolerate high levels  of latency.  “Real time” is restricted to IRC discussions and events/conferences.
These two factors has made open source what it is today. They have been “success factors”.
Agile methodologies do not embrace “distribution” environments. The widely accepted recommendation is that teams should share a physical space. It is way more than a recommendation. It is somehow a requirement.
The second case, “latency”, is considered by agile methodologies as a waste. It is not tolerated.

2.- Culture

Free Software was born as a reaction to a system that promoted corporation interests over developers, so users. The agile movement was a reaction to those methodologies that put process first, not people. Hence, it is obvious that both movements share a lot: people first
This is reflected by some when saying that FLOSS development is agile.
In my opinion, there is a big difference between what agile methodologies and what Open Source development propose in terms of principles.
Agile methodologies promotes a strong team culture. Open Source was born “based on champions”. FLOSS culture normally applies the meritocracy concept to individuals.  Open Source projects are organized around contributors, around specialists, not around teams, as we understand them in corporate environments.
This is no surprise since Agile was born in companies/corporations and Open Source was born as a viral movement, grown “by aggregation“.

The conflict

In my opinion, the more the industry embrace open source, and as result, open collaboration, the higher the conflict developers and managers will face due to the above challenges.  Companies are becoming more distributed environments and are working more and more upstream, instead of simply being consumers or occasional contributors.
In consequence, it would not surprise me if we hear more and more about  “corporate development methodologies” (a.k.a. agile) vs. “upstream development methodologies” (a.k.a. FLOSS).
Scrum, XP, Kanban -ish fans will need to face those challenges and find solutions in order to succeed in open collaboration environments. In the same way, based on the increasing influence that companies are gaining in these ecosystems, FLOSS methodologies in a few years will differ from what we knew 10 years ago.

This conflict will not be (is) about a R&D vs a product/service vision, it is not about creativity vs efficiency, it is not about micromanagement vs autonomy or teams of juniors vs specialists either. It is about methodologies applied to specific environments and its limitations. Maybe a simple update of the most successful agile methodologies will do the job…. or maybe we need to revisit some of the principles.

If you got here, maybe you want to take an extra step and answer these questions. I would appreciate it:

  1. Do you perceive this conflict as I do?
  2. Am I missing other key elements in the diagnosis?
  3. How do you think we can adapt agile methodologies so they can be adapted to FLOSS environments?
  4. I am interested in knowing how you adapt agile methodologies to overcome the above challenges. I plan to write about my experience these coming days.