Blogs

SpringSource Blog

Nonsense about Open Source

Rod Johnson

The production of nonsense on open source is a highly competitive field. However, I've just come across something that raises (lowers?) the bar: a post by an OpenLogic blogger entitled What's your time worth?

It's not a long piece, which is handy, as it makes it easier to deconstruct paragraph by paragraph. I'm focusing on enterprise Java, about which I can speak from experience.

The blogger gets to the point right away with a concise statement of why she doesn't understand open source in the enterprise:

Developers that work on open source software typically have day jobs that pay pretty well. So they work on open source software for free and write code during the day for big bucks.

Wow, I thought we'd got beyond this "hobbyist" idea years ago. Let me quote some statistics about Linux, from a 2004 article called Linux is now a Corporate Beast. The emphasis is mine:

Dispelling the perception that Linux is cobbled together by a large cadre of lone hackers working in isolation, the individual in charge of managing the Linux kernel said that most Linux improvements now come from corporations. "People’s stereotype [of the typical Linux developer] is of a male computer geek working in his basement writing code in his spare time, purely for the love of his craft. Such people were a significant force up until about five years ago,� said Andrew Morton, whose role is maintaining the Linux kernel in its stable form. Morton said contributions from such enthusiasts, "is waning." Instead, most code is generated by programmers punching the corporate time clock. About 1,000 developers contribute changes to Linux on a regular basis, Morton said. Of those 1,000 developers, about 100 are paid to work on Linux by their employers. And those 100 have contributed about 37,000 of the last 38,000 changes made to the operating system.


That's 97% of commits coming from people paid to work on Linux. And that transformation has corresponded with the increasing penetration of Linux in the enterprise. Looking at the most successful complex projects in enterprise Java, such as Spring, Hibernate and JBoss, shows a similar picture. All of these are overwhelmingly written by developers who work for the companies behind them. Volunteerism plays little part. As a result, those products have exhibited rapid progress.

The post now moves onto economics–or, to be precise, an attempt to argue that the development of open source software is divorced from economics:

So does that mean if I get paid $50/hour at work I'd rather help the local school set up their network for free rather than take a job setting it up for $10/hour? As strange as it sounds, I think it's true. As a volunteer doing it for free I get more control (I can leave at any time), more prestige (I'm helping!), and more recognition (I'm not a lowly flunky, I'm a cool volunteer.)

This explains exactly why the model advanced in the blog doesn't make sense. If enterprise open source relied on volunteerism, developers would be prone to leave at any time. Let's suppose I'm a Top 10 bank. How would it make me feel imagining that the development and maintenance of my mission-critical software was tied to the enthusiasm of hobbyists? As for prestige and recognition: Sure, that is important. Most people who do outstanding things are motivated by recognition to some degree. And employees need recognition as well as salary. But if you make that the main motivation it's a recipe for disaster in the long term. What happens when something someone does is disappointing and gets rewritten? Does that person go off in a huff? What happens when the work isn't "cool," like writing documentation or reproducing an obscure bug on an esoteric environment? What happens when developers motivated by individual egos come into conflict–a likely situation if they are self-selected by ego size? There are numerous examples of projects that have fallen apart due to such conflicts. What happens if a developer has to stay up all night to help diagnose and resolve a Tier 1 production problem for a customer with a support contract? How do 24×7 SLAs work, without an investment in follow-the-sun support capability?

Developing high quality software requires an effective team of dedicated individuals. Those people need to stick around, and that requires a viable economic model. Developers need to pay for mortgages, school fees, gas and all the other costs of modern life, and it's naive (or worse) to try to duck that issue and leave it to the employers offering "day jobs." Few people will be able to make an open-ended commitment of personal time to support a project; they will want their work to have economic benefit to them. (And why shouldn't they: it provides economic benefit to users?) The reason that this matters so much is that the role of open source is becoming way too important to the enterprise to rely on pure volunteerism. As Entiva Group analyst Alex Fletcher writes, "Earlier this year, Gartner DataQuest reported that the compound annual growth rate (CAGR) of open source software (43%) between 2006 and 2011 will more than quintuple that of proprietary software (8%). The firm also projected open source software sales to reach $4.23 billion in 2007, and triple that figure by hitting $13.10 billion in 2010." Unless there's a sustainable model behind open source for the long term, that's an awful lot of potential flaky software.

While Gartner may crunch numbers, the blogger claims that working on open source is essentially outside the realm of economic activity:

As cheap skilled labor all I've done is lower my hourly worth. I've now said I'm only worth $10/hour – I'm working for peanuts instead of helping out someone who needs it

So ideally open source developers shouldn't be paid, because that would drop the hourly rates for their day jobs. We'll get to OpenLogic's payment model shortly, but it seems consistent with this viewpoint. While developers aren't supposed to think about money, naturally OpenLogic isn't outside the realm of economic activity.

I'm not saying this makes sense at all or that it's a good thing but I think it's a reality. What do you think?

I think it's fortunate that this is not a reality, because it makes no sense at all, is a Very Bad Thing and could potentially destroy the software industry.

The blog concludes that:

This is one of the reasons we worked hard to make sure our OpenLogic Expert Community compensates fairly.

So I followed the link to their "Expert Community" FAQ to understand exactly how this works in practice. The interesting answers are:

Do I get paid to be a part of the Expert Community? Yes, the OpenLogic Rewards program pays Expert Community members in points (redeemable for your choice of cash or rewards) upon successful resolution of an incident. OpenLogic charges enterprise for support. OpenLogic's internal technical support team resolves basic issues. OpenLogic, in turn, contracts with members of the community to resolve more complex issues.

Right, so it works like this. OpenLogic hope that enterprise customers will pay them money up front for support contracts. OpenLogic don't appear to believe in paying developers salaries, so they expect that they won't be able to handle complex support issues internally. So in that case they reach out to the "community," and pay only if an incident is resolved. That works like this:

What will I have to do as a Member of the Expert Community? As a member, you will have access to a list of customer incidents that we need help resolving. You can choose which incidents to work on by "grabbing" them from the Expert Community Member portal. Once you sign up for an incident, we ask you to work on it expeditiously until resolution. Most incidents will take less than 4 hours to resolve… Through the OpenLogic Reward program, you will receive 100 points per incident you resolve. If an incident takes an unusually long time to resolve, you can appeal for more points. Points can be traded in for cash (100 points = $100 dollars) or merchandise (such as an Xbox 360). You can also choose to donate your points as cash to your favorite open source project or one of the charities on our list.

So developers only get paid when there are incidents. Despite signing up, they get nothing unless OpenLogic get stuck themselves trying to resolve an issue. Then they get paid at a notional hourly rate of $25. Developers are incented to fix problems as quickly as possible, because they aren't paid to polish or refactor. (Often an incident may require significant rework to fix properly. This is unlikely to happen under this model.) Developers will naturally compete for incidents, as remuneration being targeted at individuals rather than a team can distort priorities. There is no reward for the kind of teamwork that forges and maintains in great software. There is also no real concept of a support team. In a real support team, managers not only consider geographical distribution and shift patterns to ensure that there's always coverage, but also ensure a balance of skills. For example, if there are only 2 developers with expertise in a particular area, there's likely to be a plan to skill up others, and in the meantime they are unlikely to be approved to take holidays at the same time. You can't say to a volunteer "sorry, you can't go on holiday until we backfill you–but of course we won't pay you unless we get a call".

A casual rate of $25/hr isn't a lot compared to developer salaries. However, it may not be so bad because it's possible to join the expert community when you're an open source developer on the make. It's not like you need to have made a big contribution to the project. My emphasis:

What are the qualifications needed to join the Expert Community? You'll need to fill out a brief Membership Application. We are looking for committers or contributors for over 200 open source projects that we support. If you are not a project committer, we will ask for a reference from one of the project committers to sponsor or vouch for you. OpenLogic's Expert Community program is a great way to boost your experience and contributions to a project.

That wouldn't inspire me with confidence if I was a customer. My "support" may be coming from non-committers. So OpenLogic can't guarantee to resolve issues strategically. Dependable support involves the ability to commit to the main tree–as well as, in some cases, branches maintained for a particular revision of customer. It also involves teams rather than potentially very junior individuals. It's unlikely that I will get to the people who actually wrote the code my incident might relate to.

So what's missing here? Let's leave aside the obvious question of fairness: OpenLogic are happy to take the fixed payment upfront from customers without giving the people who do most of the work security of income. (They leave that to the employers in the day jobs.)

There are many flaws in this model, but one huge problem that you can drive an SUV through. Developers get paid only to resolve incidents. No one gets paid to write the software. No one gets paid to innovate. No one gets paid to document. No one gets paid to refactor and improve design. So OpenLogic contributes a big fat zero to sustaining the projects they hope to profit from. From OpenLogic's point of view, that's fine: if one project falls apart they can move onto another one, as they provide similar "support" for all projects. But it could prove little short of disastrous from the perspective of enterprise customers, who may need to run the software they introduce today for years.

The key answer from the FAQ is this:

Can I make enough to do this full-time? At this point, we would not recommend that anyone quit their job to become a member of the Expert Community.

Exactly. We're talking about a rapidly growing part of the software industry that is mission critical. And this model does not give anyone the ability to make a living from developing great software. If the effect of open source is that people can't make a living from developing great software, everyone will end up suffering. You can't divorce the process of maintaining software from the process of creating software.

Perhaps there are areas in which this model makes sense. There are products (largely outside the enterprise space, or in simpler technologies) where volunteers do most of the work. But it makes no sense at all in enterprise Java. The problem is not the notion of aggregation–which can make sense if a company possesses the resources to invest and sustain projects, or partners with others that do–but the idea that an industry can be sustained in disregard of the laws of economics, and that motivating people with games consoles rather than security of income will provide a basis for enterprise quality support.

It's proven that the most scalable revenue around open source software (disregarding commercial add-ons) is in support. OpenLogic's model divorces that entirely from the creation of the software in the first place. That's not the future of enterprise open source — unless open source has no future.

Similar Posts

Share this Post
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google Bookmarks
  • DZone
  • LinkedIn
  • Slashdot
  • Technorati
  • TwitThis
 

9 responses


  1. Hi Rod,

    Since you didn't comment on my post or leave a trackback for me, I thought I'd leave a comment here so we can start a conversation. (For those reading this, I am the author of the above post that Rod criticizes.)

    First off, you question my understanding of open source in the enterprise. I suggest you google me before you do that – I've spent the last seven years helping enterprises use open source software and I set up and managed HP's Open Source Program Office complete with policies and procedures.

    I totally agree that most developers are paid to work on open source these days – I think the stat is over 70% – but I think that most developers still believe in open source software and the open source software model and have many motivations besides money for working on it. That's one of the great things about the community – I love the idealistic, "we are going to do the right thing" attitude. I think that's one of the main things that will make open source successful. Speaking of which, it sounds like you don't think open source will be successful if it stays in the "free" space. I would challenge that – many successful open source software projects don't have large corporate sponsors – take some of the Apache projects for example!

    As for the OpenLogic Expert Community – we try very hard to pay fair and nobody has to take any of the issues we post – they can evaluate each issue and what we are offering and sign up if it interests them for that amount of money. So far we haven't had any issues on getting takers and we compensate fairly. We also continuously ask our members for feedback. You are welcome to join! You can see issues, take them if they are interested and give us feedback. Constructive feedback only though, please!

    Conversation welcome!

    Stormy


  2. Rod,

    I guess I disagree with the notion that there is only one "right" way to pay developers. I don't think this is an "either/or" situation. Instead, I prefer to leave the choice to individuals. At various times in my life, I have chosen to do consulting and get paid an hourly rate. I think developers deserve the same options and can make their own choices. If they choose to provide support for an open source package and get paid on a per incident basis because it makes sense for them, that's great. If not, they won't choose to do it.

    As far as how much OpenLogic pays developers, I think you mistook the phrase "less than 4 hours" with "equal to four hours". In many cases, issues are resolved in a very short period of time — we often get responses back within 30-60 minutes. That nets out to $100-$200 per hour which likely equals or exceeds the equivalent hourly rate (even with benefits) that many companies pay developers.

    Bottom Line: The "proof is in the pudding" — and in this case the model is already working. Open source developers have chosen to be members of the OpenLogic Expert Community. They have resolved issues and gotten paid. And customers have been happy with the results. At OpenLogic, we are focused on how we can help enterprises use more open source and get more value out of open source — and that will be good for both enterprises and open source developers.

    Best Regards
    Kim


  3. Stormy

    Thanks for your comments.

    [quote comment=\"26875\"]Hi Rod,
    I totally agree that most developers are paid to work on open source these days – I think the stat is over 70% .[/quote]
    Right (and my statistics indicate it is much higher than 70% in the spaces in which I\'m interested). But the blog with which I disagrees begins with the black and white statement that
    [quote]Developers that work on open source software typically have day jobs that pay pretty well. So they work on open source software for free and write code during the day for big bucks.[/quote]You made no mention in your blog that the majority of developers do get paid for developing open source.

    [quote]…but I think that most developers still believe in open source software and the open source software model and have many motivations besides money for working on it. That\'s one of the great things about the community – I love the idealistic, \"we are going to do the right thing\" attitude.[/quote]
    I think my contribution to open source makes it pretty clear that I believe in open source. Idealism is great–and everyone, whether or not they are in open source, should do whatever they do because they feel it is a worthwhile thing to do. (I certainly do.) But ultimately, if developers develop open source software that has economic value to enterprise customers, they should be able to be paid to develop it–not as pocket money but as income that can sustain their family and put their kids through college. I\'m idealistic too, and I feel that it is the right thing for people who develop valuable software to be able to make it a career.

    [quote]Speaking of which, it sounds like you don\'t think open source will be successful if it stays in the \"free\" space. I would challenge that – many successful open source software projects don\'t have large corporate sponsors – take some of the Apache projects for example![/quote]
    I didn\'t say that open source would not be \"free\" (as defined, for example, by being freely available under a recognized, liberal license like the ASL). However, I am sure you will agree that the definition of \"free\" is problematic. My blog refers to enterprise software, and I am sure that you and I both agree that any enterprise customer requires a support contract to achieve best possible results with open source, hence open source does not equal free for such customers and it does them a disservice to make that simplistic equation.

    In the enterprise Java space (my prime concern) there are many projects without corporate sponsors. But those that are widely adopted are mainly relatively simple (Log4j or Struts) and I\'ve given examples of how leading, complex products typically do have corporate sponsors. Even at Apache, there are numerous examples. Most of the most active Tomcat developers, for example, are employees of companies who offer Tomcat support. IBM has invested heavily in Geronimo. Maven has a corporate sponsor. Many of the Apache Java projects without corporate sponsors are less successful and less vital.

    Regards
    Rod


  4. Hi Kim

    Good talking to you at OSBC, btw.

    [quote comment="26884"]Rod,
    I guess I disagree with the notion that there is only one "right" way to pay developers. I don't think this is an "either/or" situation. Instead, I prefer to leave the choice to individuals. At various times in my life, I have chosen to do consulting and get paid an hourly rate. I think developers deserve the same options and can make their own choices. If they choose to provide support for an open source package and get paid on a per incident basis because it makes sense for them, that's great. If not, they won't choose to do it.[/quote]
    Indeed, developers should choose what makes sense for them. Our opinions differ on this and we'll see how it pans out. Again I come back to the enterprise Java space. Look at the most critical open source projects that folk use in that space, like Spring and Hibernate. And look at the choices the active developers on those projects have made.

    [quote]Bottom Line: The "proof is in the pudding" — and in this case the model is already working. Open source developers have chosen to be members of the OpenLogic Expert Community. They have resolved issues and gotten paid. And customers have been happy with the results. At OpenLogic, we are focused on how we can help enterprises use more open source and get more value out of open source — and that will be good for both enterprises and open source developers.[/quote]
    Yes, but you are ignoring the core problem of where does the open source come from and you are failing to invest in the goose that lays the golden egg. My blog entry is trying to grapple with the long term future of open source. Currently I think it's fair to say that OpenLogic is a pretty small company. There's a difference between what works today for a small company and what will sustain the enterprise software industry as open source moves front and center. Currently your model is not having more than a local impact; if it were rolled out across the enterprise space I stand by my view that it would not generate the sustained investment (financial and otherwise) that is required.

    Rgds
    Rod


  5. Stormy

    [quote comment=\"26875\"]As for the OpenLogic Expert Community – we try very hard to pay fair and nobody has to take any of the issues we post – they can evaluate each issue and what we are offering and sign up if it interests them for that amount of money.[/quote]
    So what if an issue takes 2 developer days to resolve, working continually to get to a solution? That requires handovers, and it requires the ability of developers to focus on the fix because that is what pays their bills. No sane developer would sign up to handle such an issue. (In fact, I assume they don\'t get paid if they don\'t fix it, so I assume that would get paid nothing if they spend several hours before realizing that this issue is extremely difficult to resolve.) If I\'m a customer I want the hard things fixed, not just the easy things. If my production system is down due to an issue so complex my own developers can\'t help me with it I want to know someone is working on that issue somewhere in the world until it is fixed. That\'s what I\'m paying for. Enterprise open source requires the same quality of support as closed source products or it will not deliver real benefit to customers. I also want to know that the people working on the issues have the necessary infrastructure to do so properly. For example, let\'s suppose it\'s an issue that involves open source and a particular version of WebLogic. Without the necessary infrastructure it could take hours just to download and install the software mix involved, let alone fix the issue.

    I feel strongly about this because I hate open source to be sold short. A model that can\'t handle this kind of major problem doesn\'t meet the standards of production support offered by tradtional software vendors. For open source to justify its rapidly growing place in the enterprise, the support has to be at least as good.

    [quote]We also continuously ask our members for feedback. You are welcome to join! You can see issues, take them if they are interested and give us feedback.[/quote]
    I admire your recruiting energy :-) But I also want to be paid for creating software, not just when something goes wrong. And I care enough about open source to want to spend my working hours with it, too.

    Rgds
    Rod


  6. One thing that the 97% number does not indicate is when people are committing. I'm paid to do open source in the day, and when I get home at night, and at the weekend, I do more open source. So does that make me a paid-coder or a hobbyist? Have I stopped volunteering? Or is it that open source developers are increasingly finding a way to have their cake and eat it.


  7. Rod…

    I'm a little late to the game here, but oh well, I'll comment anyway.

    For the sake of full disclosure, let me introduce myself. My name is Bryan Noll, and I'm a developer employed by OpenLogic. I'm no big deal, if you google me the first two hits you'll get are a now-defunct website about Bryan Noll the woodworker, and a pretty decent site about Bryan Noll the guitar player, neither of which are me. You can see some of my thoughts on Stormy's original post by reading my response to it. As for my thoughts regarding what you have to say…

    First and foremost, I think your assertion that it is not healthy for a project or open source in general when people who have no real investment in a particular project offer support for it is an interesting one… one I've not heard before. I think there's enough validity to it to make a company like ours consider it and genuinely examine our responsibility to the open source projects we support. The result of this examination, in my mind, would be a demonstrable policy OpenLogic would have in order to mitigate the potential concerns you're raising. I'm sure I don't know what exactly that would be, so allow me to be vague at this point. This dovetails nicely though into some of the issues I have with what you're saying.

    You can't have your cake and eat it too. You say:

    [quote post="167"]My "support" may be coming from non-committers. So OpenLogic can't guarantee to resolve issues strategically. Dependable support involves the ability to commit to the main tree–as well as, in some cases, branches maintained for a particular revision of customer.[/quote]

    Interesting to note that in the case of Spring, Interface 21 would be the one and only entity capable of doing what you're proposing. (Please correct me if I'm wrong here.) Seems like a sweet deal for Interface 21 there. If you open source your stuff, and it's a great product that is a big hit like yours is, do you really expect that no one else in the market will attempt to engage in business surrounding it? Surely not… for another example of this, just take the books written about the Spring Framework not written by folks working for Interface 21, or the countless number of developers out there who are highly marketable because they've learned the framework you open-sourced. Are they to feel guilty for the health of the project as well? Interface 21 is allowed to reap such great benefits because the product is A) great, but also because the product is B) being delivered to the marketplace en masse because you've open-source it and made it free. While not being exactly the same, this argument has that “nobody else should be able to make and sell cars because I invented themâ€? feel to it.

    Secondly, while I again confess that I think you make good points about potential project health ramifications, another fact driving aggregated support of the market remains. What enterprises have to do today to obtain support for the software they're developing and deploying into production is a painstaking process… the idea of managing support contracts for x number of projects across y number of stove-piped divisions seems like an inefficient nightmare to me. Aggregating these services makes sense from the enterprise's standpoint, however shortsighted you may claim it is. If an opportunity to do business in a market exists, you can't fault someone for swooping in and taking advantage of that opportunity.

    Good, thought-provoking, original post on your part.


  8. Stormy,

    Did you really just play the "google me" card? How about instead of throwing your credentials around, you actually address Rod's points?

    By the way, I DID Google you, and it looks like every page that mentions you and Open Source together is a page written by you. Not that impressive.


  9. Hi!

    Today there are plenty of jobs in Open Source or Open Source related positions. Unless you live in the midwest there is no reason why you can't find employment doing what you enjoy. This wasn't the case 15 years ago, but it has been the case for the last 7 to 10 years.

    Supporting open source software is an open field. I don't think you will find companies that will be all that successful at it unless they produce it, for the reason that followers will almost always be a day late in understanding a given product. There are certainly cases where some companies have grown larger then the companies who originally produced the software, but I think that in almost all cases you will find that they did added value in the form of installation (since installation is something that not all open source people get the importance of getting it right).

    Cheers,
    -Brian

8 trackbacks

Leave a Reply