It makes a lot of business sense for Sun to open source the Java J2SE, J2EE and J2ME framework libraries and a fork of the Solaris kernel under the GPL licence.
It would benefit the entire Java-based industry, including the free software, open source, and proprietary-based vendors, since Java's primary strength - the ability to write code which is constantly portable across many vendors platforms - would be greatly enhanced if all vendors were using the same core libraries.
To ensure that the standard base core would not become polluted with incompatible forks, the source could be licensed with a clause requiring any incompatible changes, or any additional classes or methods, to be moved to, and occupy, only the vendor's namespace. Another clause would require that the vendor version of the Java-to-bytecode compiler, and any GUI IDE, defaults to generating portable bytecode, without embedding any vendor-specific references.
The OSF definition of an open source licence clause five explicitly states that "The license may require derived works to carry a different name or version number from the original software."
Developers and vendors would only be required to shift changes to the vendors'/developers' namespace if the changes were incompatible with the JCP JSR open standards. This would not prevent the development/distribution of additional optimisations, ports, or bug fixes. Since adoption of standards has for a long time been an open source tradition, it would not be much of an imposition on the open source community.
Vendors don't have to use all the same core libraries, just provide the same standard interface. The open source Java core can been seen as a starting common base. Each vendor would be free to "short circuit" their implementation, as long as the standard API behaviour remained the same. Vendors would still be free to compete on their JVM performance along with how well it performs interfacing databases, integrated development tools, etc.
Sun could require contributors to the Java Open Core to let Sun, or the JCP, dual-licence the result, as Sun does with OpenOffice.org and StarOffice. If a vendor does not wish to disclose its modifications then the vendor could pay for a closed source licence scheme. The payment could then be split up between Sun, the JCP, and the contributors.
Ask IBM and HP what their customers are demanding and you will find out, more often than not, that it's vendor-neutral/independent solutions. Customers don't want lock-in slavery anymore. That is why Linux is such a success and why there is more demand for Java skills than any other programming language.
It should not be necessary to open source Sun's JVMs. In the long run it could greatly benefit Sun to develop the JVM under a dual licence as it is doing with OpenOffice.org and selling StarOffice.
Releasing a fork of the Solaris kernel makes even more sense when you consider Sun's move towards commodity based hardware, like AMD's Opteron, and enterprise desktop systems. Sun is going to need drivers to interoperate with x86 hardware and common peripherals. In comparison to Linux, the range and quality of hardware drivers available to Solaris is pitiful.
If Sun can manage to get out from under the SCO Group's claims over the old AT&T code base by dealing directly with Novell, which still appears to hold the rights and copyrights, then Sun would be free to release a fork of the Solaris kernel under the GPL licence. It could then take any source code from the Linux kernel and incorporate it into the GPL'ed Solaris kernel fork, and deploy that kernel in desktop and clustered systems markets, where Linux currently does have a lead over Sun.