Linux kernel development – Some insight on the ecosystem of the commons, and motives

Linux story is not just about technology development, it is also what it means for a community and what the business is becoming.  Linux kernel grew under a new deal of a collaborative effort investment and sharing the technological return; somehow a rebellious mood to push back against the exclusionary and closed systems. Overall it is a major paradigm shift how a business is conduct because this is a project that demonstrates that cooperation can be useful in developing platforms.

There is an interesting talk of Yochai Benkler on the new open-source economics having his thesis that huge cost of  developing a product will ultimately lead to a social production with the ownership of the capital largely distributed is different to the well known methods (market and governmental ). Furthermore Benkler says in his Coase’s Penguin, or Linux and the Nature of the Firm paper:

”

In this paper I explain that while free software is highly visible, it is in fact only one example of a much broader social-economic phenomenon. I suggest that we are seeing is the broad and deep emergence of a new, third mode of production in the digitally networked environment. I call this mode “commons-based peer-production,” to distinguish it from the property- and contract-based models of firms and markets. Its central characteristic is that groups of individuals successfully collaborate on large-scale projects following a diverse cluster of motivational drives and social signals, rather than either market prices or managerial commands.

”

Thanks to Alun Williams I found an interesting 2009 report on Linux Kernel Development revealing facts on “How Fast it is Going, Who is Doing It, What They are Doing, and Who is Sponsoring It”

The top five individual companies sponsoring Linux kernel contributions include:
* 12.3% Red Hat
* 7.6% IBM
* 7.6% Novell
* 5.3% Intel
* 2.4% Oracle

”

WHY COMPANIES SUPPORT LINUX KERNEL DEVELOPMENT
The list of companies participating in Linux kernel development includes many of the most
successful technology firms in existence. None of these companies are supporting Linux
development as an act of charity; in each case, these companies find that improving the kernel
helps them to be more competitive in their markets. Some examples:
•     Companies like IBM, Intel, SGI, MIPS, Freescale, HP, Fujitsu, etc. are all working to ensure that Linux
runs well on their hardware. That, in turn, makes their offerings more attractive to Linux users, resulting
in increased sales.
•     Distributors like Red Hat, Novell, and MontaVista have a clear interest in making Linux as capable as it can
be. Though these firms compete strongly with each other for customers, they all work together to make the
Linux kernel better.
•     Companies like Sony, Nokia, and Samsung ship Linux as a component of products like video cameras,
television sets, and mobile telephones. Working with the development process helps these companies
ensure that Linux will continue to be a solid base for their products in the future.
•     Companies which are not in the information technology business can still find working with Linux
beneficial. The 2.6.25 kernel included an implementation of the PF_CAN network protocol which was
contributed by Volkswagen. 2.6.30 had a patch from Quantum Controls BV, which makes navigational
devices for yachts. These companies find Linux to be a solid platform upon which to build their products;
they contribute to the kernel to help ensure that Linux continues to meet their needs into the future. No
other operating system gives this power to influence future development to its users.
There are a number of good reasons for companies to support the Linux kernel. As a result, Linux has a broad
base of support which is not dependent on any single company. Even if the largest contributor were to cease
participation tomorrow, the Linux kernel would remain on a solid footing with a large and active development
community.

”

It took personal volunteering until gained weight and height, into becoming an attractor factor. Quite our days  a snowball effect. Why? There is the resultant of rising cost of design of adding more and more complex platform features and the price squeeze which will lead commercial companies to rally with the open source phenomena as the last is less driven by the market.

On this token there is an interesting position in Collaboration is the way out of a crisis, says TSMC – IEF 2009 which reflects the mood to reinvent of the industries:

”
“It has to be made more profitable”, said Marced, “and it can only be done by collaboration. We have to make sure that the whole industry makes more money.”

Marced argued that collaboration reduces waste and shares investment while individual efforts lead to redundant initiatives and heavier investment.

”

There is also the 2008 revision for those interested in some sort of history snapshot reference of the Linux kernel development.

Robustness – a succes factor

One success factor of the Internet is carried by the content of the originator of the Internet Protocol Jon Postel recommendation:
“In general, an implementation must be conservative in its sending behaviour, and liberal in its receiving behaviour. That is, it must be careful to send well-formed datagrams, but must accept any datagram that it can interpret.”

This recommendation is generalized in a common sense Robustness principle that stays unequivocally within the cooperative spirit (of playing nice): “Be conservative in what you do; be liberal in what you accept from others”.

Technology – paraphrasing Clausevitz

Von Clausevitz uses the bold statement “Der Krieg ist eine bloße Fortsetzung der Politik mit anderen Mitteln” – war is merely a continuation of politics (with different means). Technology can be paraphrased being a continuation of the business.

The applied technical solutions are sponsored with financial means for financial return. There are standards, stakeholders, active and passive players, and wars. There are not only the technical merits that will impose a winner solution, the entire business context will tell, and in certain cases even the political regulation will come to play a role. The success stories are many times mystified, there are nothing else than personal merits, alignments of the stars and of many persons interest.

Software development cost, some measurements and available references

I got a few significant references on some of the major software platforms developments. Although there are a many themes that are interesting, but this time I am focusing strictly on the estimated development cost. For example Vista development cost is estimated in 2006 to be about 10 bilion USD, and the estimated total development cost of a Linux distribution to be 10.8 billion USD

The first comments that came to my mind: this is a lot of money! It is just difficult to justify such effort.

All the above are mammoth projects. What about smaller scale projects? Is any way to get a feeling what would be a metrics for a given project? The most handy study material is related to open source projects and there are a number of websites that provides metrics for such open sources projects:

1. Ohloh and its project search page
2. Koders and its project search page
3. Krugle and its project search page
4. Codase and its search page
5. Merobase and its search page
6. JExamples and its search page

To satisfy my intellectual curiosity I have been checking the GStreamer (a multimedia framework) metrics provided by Ohloh and Koders. There are a number of common sense questions that comes up once that a multimedia framework is considered. What is the cost of it? Would be build in house, purchased or open source? What are the legal liabilities? The GStreamer home page provides limited information to such questions, however the code is available for study for gathering more information.

For the software licenses structure and programming languages distribution and usage for this project the Ohloh analysis is providing the following information:

GNU Lesser General Public License 2.1 2822 files

GNU General Public License 2.0 188 files

MIT License 49 files

Mozilla Public License 1.0 38 files

Simplified BSD License 18 files

Academic Free License 16 files

GNU General Public License 3 or later 5 files

BSD Copyright 2 files

Python 2.4.2 license 1 files

	Language	Code Lines	Comment Lines	Comment Ratio	Blank Lines	Total Lines
	C	742,710	171,976	18.8%	174,728	1,089,414
	XML	124,289	1,124	0.9%	1,678	127,091
	C++	22,364	9,922	30.7%	5,743	38,029
	Python	18,885	2,708	12.5%	3,341	24,934
	C#	18,022	2,589	12.6%	4,203	24,814
	Scheme	13,167	296	2.2%	1,981	15,444
	Automake	10,540	1,098	9.4%	2,780	14,418
	Autoconf	5,221	1,140	17.9%	1,171	7,532
	Perl	4,652	450	8.8%	771	5,873
	shell script	3,397	730	17.7%	643	4,770
	HTML	2,724	1	0.0%	7	2,732
	Objective C	1,624	291	15.2%	508	2,423
	Assembly	1,273	301	19.1%	236	1,810
	XSL Transformation	1,209	67	5.3%	188	1,464
	Make	67	4	5.6%	15	86
	cmake	21	0	0.0%	7	28
	CSS	13	0	0.0%	0	13

Development Cost for multimedia  plugins codecs is estimated to be 1.1 mil USD (as provided by Koders analyze), however this is not a real life case and I am expecting the costs to be higher. I would guess it is is based on the Constructive Cost Model (COCOMO).

$1,102,910
Assumptions
Lines of code:	// 220,582
Person months (PM):	220.58
Functions required:	100.0%
Effort per KLOC:	1.00  PM
Labor Cost/Month:	$5000

An interesting complementary view is provided by Coverty architecture library. The Coverty Architecture Analyzer tool uses information gathered during the build of a codebase to create a comprehensive list of interdependencies in the code and to generate diagrams like the shown below.

Precautionary statement: Don’t believe in the obvious!

I read yesterday Ruth Malan‘s August journal about the farmers  “don’t believe everything you think” precautionary sign. This was a deja-vu encounter, I listen fascinated a while ago a precautionary axiom given by a friend of a friend: don’t believe in obvious.  It just struck me, it accelerated my mind! It came to my mind the most famous motto of Delphi, which Socrates said he learned there, the Gnothi Seauton “know thyself!”, or Plato’s Allegory of the Cave,  or Far East’ Maya, or even the culmination of the conspiracy theory.

I am aware of the human capacity to adapt which consists in mind’s ability to simulate realities, to dream. This synthetic ability has to be guided, and there is a medicine against the illusions, it requires a constant mind repositioning to shade new angles and perspectives to the issues. Find a spin and it will feed a fresh light for a while (there are infinite things to know and there is limited life to learn).

After all we are experimenting the physics effect of the observer, the observer measurements is changing the reality …

It is instructive to check the history of ideas with its amalgam of various positions and cycles of thoughts to understand there is a constancy to find and express the reality, and gain personal salvation.

Having some skin in the project

I remember my awe as a child when I have been traversing the Anghel Saligny‘s famous bridge. This bridge was build in 1890 and it was the longest bridge in Europe at that time,  along with technical achievements its value stays foremost in its strategic fasten of the country after its war of independence. After a century this bridge is considered  to be in top 10 most beautiful European bridges.

The bridge daring thrown forth is equal by its builder warranty, his life. The bridge was inaugurated on 26 September 1895 and as a test on the opening, Anghel Saligny stayed with his workers on a boat under the bridge while a convoy of 15 locomotives sped at 85 km/h.

I have to put things in perspective, there are engineering failures. There are professional licensing practices, even symbolic iron ring.

There is always the moment of truth: to stand under the  capstone as it is lowered into place, to be held to that accounting (I like Ruth Malan’s drawing). Anghel Saligny hold himself through this ancient roman ritual, a different stance to what the profane “just another job” might be. This ritual becoming a matter of life or death turns to hold the commitment true and the success measurable.

How is your project for you?

Landscape changes

Big technology players are learning from each other, seems to reinvent themselves by reaching beyond their publicly well known capabilities. Recently Google is set to launch its own OS package, Microsoft is taking its Office to Web. Not long ago Intel was starting to invest in software, compiler and OS. Microsoft was architecting its hardware Xbox platform to keep it entirely in house. Apple started long time ago to build its OS and now is working on building (creating?) its own processor.

Why would they go to great lengths to do such things? Because they have to find and make work a new business model. Ultimately this is driven by hard-wired intrinsic ludic capabilities to explore something new.

Landscapes are not changed only by cataclysm!

cataclysm

Principles of X and awareness of the temperance

I have been advocating in my organization for many months now for principles similar of Windows X. I wished reading earlier such principles, this is just a matter of chance and maturing process to recognize such realities; perhaps I have my teachers to instil such intuitive awareness of the temperance hard acquired. or applied festina lente wisdom of the software development strategy.

”

In 1984, Bob Scheifler and Jim Gettys set out the early principles of X:

Do not add new functionality unless an implementor cannot complete a real application without it.
It is as important to decide what a system is not as to decide what it is. Do not serve all the world’s needs; rather, make the system extensible so that additional needs can be met in an upwardly compatible fashion.
The only thing worse than generalizing from one example is generalizing from no examples at all.
If a problem is not completely understood, it is probably best to provide no solution at all.
If you can get 90 percent of the desired effect for 10 percent of the work, use the simpler solution. (See also Worse is better.)
Isolate complexity as much as possible.
Provide mechanism rather than policy. In particular, place user interface policy in the clients’ hands.
The first principle was modified during the design of X11 to: “Do not add new functionality unless you know of some real application that will require it.”

“

Wabi-Sabi or appreciation of the imperfection

It is fascinating reading now about wabi-sabi ,  in the midst of recession.

“It (wabi-sabi) nurtures all that is authentic by acknowledging three simple realities: nothing lasts, nothing is finished, and nothing is perfect.”

“From an engineering or design point of view, “wabi” may be interpreted as the imperfect quality of any object, due to inevitable limitations in design and construction/manufacture especially with respect to unpredictable or changing usage conditions; then “sabi” could be interpreted as the aspect of imperfect reliability, or limited mortality of any object, hence the etymological connection with the Japanese word sabi, to rust.”

Philippe Kruchten measures the architect likeness against his tolerance to imperfection: “The life of a successful software architect is a long series of suboptimal decisions often made in the dark and under pressure. A perfectionist who spends a lot of time on issues would not be appropriate for the job.” Beware of perfectionism.

There is freshness in the impermanence.  Life spring of the ephemera.

Tokugawa Ieyasu’s festina lente

I am contemplating Tokugawa Ieyasu‘s frugal measure of action through uncharted waters; not hasten and balanced stepping within the awareness of the personal fallibility and of the unpredictable events.

“Life is like unto a long journey with a heavy burden. Let thy step be slow and steady, that thou stumble not. Persuade thyself that imperfection and inconvenience are the natural lot of mortals, and there will be no room for discontent, neither for despair. When ambitious desires arise in thy heart, recall the days of extremity thou has passed through. Forbearance is the root of quietness and assurance forever. Look upon the wrath of the enemy. If thou knowest only what it is to conquer, and knowest not what it is like to be defeated, woe unto thee; it will fare ill with thee. Find fault with thyself rather than with others.”

He has another sentence lightning self-restraining as not hurried jump into the emotions fall down:

“The strong manly ones in life are those who understand the meaning of the word patience. Patience means restraining one’s inclinations. There are seven emotions: joy, anger, anxiety, adoration, grief, fear, and hate, and if a man does not give way to these he can be called patient. I am not as strong as I might be, but I have long known and practiced patience. And if my descendants wish to be as I am, they must study patience.”

Who was Tokugawa Ieyasu? He was the founder and first shogun of the Tokugawa shogunate of Japan. There is a Japanese proverb: “Ieyasu won the Empire by retreating”, but this is another story about the courage of taking risks and  adaptability.