What happens when a small company gets too big for its boots? It runs into trouble with its mates -- or at least that appears to be what's happening at Red Hat (RH).

The nature of capitalism means every company wants to expand its territory. RH, in spite of its near-impeccable credentials as a flag-waver for the open source movement -- is no different. While comfortably pigeon-holed as an OS-only vendor -- of which there aren't too many -- RH could build alliances and partnerships with any number of companies developing application software and selling services on top of those applications. And so it did: with the likes of IBM and Oracle, two of the biggest gorillas in the software industry.

But now, stimulated by the growing number of reports that major customers were using open source middleware from the likes of JBoss, RH spent $340 million last week on acquiring open source middleware vendor JBoss. And in so doing, tiny RH has placed itself in competition with its partners.

Tiny? Just as a point of comparison, Red Hat's annual revenues are running close to $300 million -- a fleabite from Wall Street's point of view. IBM's recent results showed revenues of $8 billion -- 26 times as much -- in the last quarter alone.

Small Red, Big Blue

Red Hat -- Small Red? -- has just plonked itself in the path of this Big Blue juggernaut, starting with its initial discussions with JBoss some nine months ago. For most companies, this could a somewhat scary place to be. But RH boss Matthew Szulik in a recent interview denied this, saying that RH didn't pursue JBoss "to go out and compete with the existing interests in the middleware space."

From a competitive standpoint, he said he believed the deal to be a value-add to his agreements with IBM, under which Big Blue redistributes Red Hat Enterprise Linux (RHEL) on its servers, and provides services on top. He reckoned that the next rev of RHEL, version 5.0, would counter any advantage that rival Novell might grab as a result of the potential for conflict between Small Red and Big Blue.

He then refused to comment further until the deal with JBoss was concluded.

For the future however, Szulik could find that the cosy relationship he's had with the two big players could cool somewhat. Analysts such as Scott Donahue of Tier 1 Research have predicted that Small Red will not be as favoured in future -- and that Szulik and Fleury could clash. This is backed by a Goldman Sachs report, which says that both the dominance of RHEL along with the JBoss purchase is now stirring the two large enterprise software vendors out of their on-the-sidelines stance.

What's more, IBM has a major stake in Novell, and has an interest in seeing it succeed. While it can afford to hedge its bets and support both vendors until one succeeds in becoming the overwhelmingly dominant player, from now on, it's more likely to back Novell, other factors notwithstanding.

One hardware vendor that could find a chink of light in the news is HP, which is likely to attempt to lever RH away from IBM if it sees an opportunity. It could leap in if, for instance, Red Hat and IBM don't renew one or more of the marketing or channel deals they have in place.

Defying the Oracle

With respect to the meetings he had with JBoss chief Marc Fleury prior to the acquisition, Szulik said the two didn't talk about the discussions Fleury's company had undertaken with Oracle, which has shown considerable interest in moving into open source. Here the momentum for a split appears greater. Oracle boss Larry Ellison told the Financial Times that, now that RH competes with Oracle, "We have to re-look at the relationship -- so does IBM. I don't think Oracle and IBM want another Microsoft in Red Hat."

Ellison has also publicly discussed that Oracle is looking for a way into the Linux market -- whether that means buying Novell or RH, both of which he seemingly discounted. This suggests that the big database vendor, whose revenues are getting on for half those of IBM, could eventually squeeze RH.

Small Red is widely regarded as the bellwether of the open source movement, a company that's succeeded in showing that a viable open source business model exists, and one which broadly retains the support of the community from which it sprang. With RH's share price plunging seven per cent on the news of the JBoss move, can RH square up to the challenges that face it?