Why IBM Open Sourced Cloudscape 108
An anonymous reader writes "A common and a consistent framework for accessing information enables developers to do more things with more people more often. This article shares how Derby fits into IBM's developer strategy, the Java application stack, its intention to drive more innovation around Java on Linux, and why they want to make the Derby database become as ubiquitous as the Apache HTTP server." (Derby is the new name for the project based on the formerly commercial Cloudscape database.)
All this talk... (Score:4, Interesting)
Re: (Score:2)
Re:All this talk... (Score:3, Insightful)
-They have mutual lisencing treaties with Microsoft as the first versions were developped together with microsoft, thus they canot give away other peoples' code -They probably have other parts of the system lisenced from other parties and thus again cannot give away other peoples' code. -Last but not least: by now they probably just want the thing to silently die as it is not a commercially viable product, but they are encumbered by it.
Re:All this talk... (Score:3, Informative)
ATMs (Score:2)
Re:All this talk... (Score:5, Informative)
OS/2 is still available and developed as eComStation http://www.ecomstation.com/ [ecomstation.com]. I have to say that I think that it is very expensive, on the other hand it is far from dead.
Re:All this talk... (Score:2)
I can't say this with any authority, but for a project of that size I would be surprised if every single line of code was written by IBM internally and there are no patent licenses involved.
Open OS/2 (Score:5, Interesting)
It's time once again boys and girls for my Patented Bullshit Theory of the Day!! All BToD opinions are copyright and drug induced from the unraveling mind of me. They are to be taken lightly and humorously.
Open Sourcing OS/2? Could be promising, except I seem to remember that OS/2 was a collaborative effort between IBM and our beloved Microsoft. (Note, Sarcasm mistranslate netwise). Due to this much of OS/2 is in NT and much of NT is in OS/2, which is why OS/2 could run Windows 3.1 apps natively without and user intervention. OS/2 had a Win3.1 VM that worked so well Microsoft had to implement Win95/NT 4.0 style API's to break the compatibility.
So, if my memory is correct (and with this many holes how could it be wrong) IBM simply can't Open the code to OS/2 because they don't own 100% of it. Too much of it has Uncle Bill's own stamp on it and Opening the IBM only code would not produce a working system.
Also if memory serves, there may be some major HIPAA style agreements in place that would keep it from happening even if the ol' Softie claimed that no code from OS/2 was in the 2K version of the NT kernel and that they didn't care if the whold world saw it. Because OS/2 is used in a lot of back office banking, telecom, and medical solutions. I know that one is true, I worked for a CLEC for a while and a good 80% of the boxes were running OS/2 Warp 4.0.
So, even though it might be kind of interesting to take a long hard look at the OS/2 code. I don't personally think it would ever happen. Too many legalities and too much legacy in place that still just works to hand keys to something that might enable those less fortunate of us (humans) that feel it's ok to commit grand theft to circumvent the already cheese cloth security surrounding very personal data.
But all this is circumstantial and delusional and part of my deranged mind, and as such this has been another Bullshit Theory of the Day
We now return you to your regularly scheduled rant.
Hmm (Score:4, Informative)
No, none of NT was in OS2. Nor is any OS2 in NT. That was one of the reasons for creating NT. There is Win 3.1 in OS2 in the VM that you mentioned, but I hardly think that played much of a decision in creating a 32 bit WIN API. After the success of win 3.1, Microsoft realised that it could succed with out OS2 or IBM. So it made win 3.1 32 bit and created win 95 until NT was ready for mainstream use.
Re:Hmm (Score:5, Informative)
Microsoft largely wrote OS/2 1.x according to IBM's specifications. IBM was responsible for writing OS/2 2.x. This includes the entire Workplace Shell, which started life as a shell for 1.x, actually: I saw it demo'ed in 1998 running on 1.x. This was an early demo: even window resize did not work! :)
While IBM was working on 2.x (mainly WPS stuff), Microsoft was tasked with writing the next version of OS/2: 3.0. It was about this time that Windows 3.0 became such a success. Microsoft then too their OS/2 3.0 code and decided to make Windows NT.
That is why an *amazing* number of Win32 (as in NT, *not* 95) calls are merely renamed OS/2 calls. In fact, IBM ported Lotus SmartSuite to OS/2 by creating a Win32 (again, NT, not 95) to OS/2 translation layer that allowed them to port like 85% of the SmartSuite code without rewriting.
Windows 95 was not even *thought* of at this time. We're talking 1991 timeframe. Windows 95 was never supposed to exist: NT (NT 3.1, that is) was supposed to be the 32-bit OS that the world moved to. But it was too big, too bloated, too unusable.
However, there is (well, was) a *ton* of code that started life as OS/2 3.0 in Windows NT. That's why during the divorce, IBM was given the rights to a source license of Windows 3.1. Which is also why shortly afterward Microsoft release Windows 3.11! :) In IBM's "Blue Spine" version of OS/2 (the one that included Windows 3.1), IBM's copy of Windows 3.1 ran 10% faster than Microsoft's. Why? They recompiled Windows with the Watcom C compiler instead of MSVC! :)
However, it's all kind of moot, anyway. The Win32 API is now quite a bit different (Windows 95's 'Win32' API was quite a bit different from NT 3.5's, and Windows 2000 and XP have moved in new directions, too), and OS/2 isn't going anywhere.
Re:Hmm (Score:2)
The Other way around: IBM licenced the Win16 API from Microsoft for use in OS/2 PM. MS wanted the OS/2 API to be "just a recompile" for Windows programmers, but IBM balked and changed/renamed some things. Eventually MS introduced Win32 which was much closer to Win16 than PM was.
Win32 and PM look similar because they both derive from the same base (Win16).
Re:Open OS/2 (Score:2)
You've basically got a good point, but... there were copies of OS/2 sold which could NOT run Windows 3.1 programs unless you provided your own copy of windows. These copies were sold at a lower cost. There's probably plenty of code in OS/2 that Microsoft has rights to, but that's not necessarily related to windows at all.
Re:Open OS/2 (Score:2)
Re:Open OS/2 (Score:2)
I suspect it's another ABI built into the kernel rather than a userland service like WINE. This isn't at all unusual. NT, BSD, and Linux all support binaries for other operating systems in the kernel, to one degree or another.
It would be analagous to WINE if WINE were a Linux kernel module. Which has been discussed, but never done.
Re:Open OS/2 (Score:2)
Re:All this talk... (Score:2, Interesting)
Re:All this talk... (Score:1, Interesting)
Re:All this talk... (Score:2, Interesting)
Re:All this talk... (Score:1, Funny)
If OS/2 gets open-sourced, it might encourage people to install it. Think of the suffering!
I sort through my snail mail and crack open the BOFH Monthly Newsletter, "kill -9" and check out the articles therein. There's a nice peice on making OS/2 slow, boring and painful, but it looks exactly like the OS/2 installation instructions to me...
BOFH 11 [uni-kl.de]Re:All this talk... (Score:1, Insightful)
Re:All this talk... (Score:4, Informative)
OS/2 2.0 was a fully 32-bit, reentrant, fully preemptive multitasking kernel in 1992. Linux still has issues with a preemptive kernel! The graphics interface went 32-bit in OS/2 2.1. It is a single-user system, so there is (or was, anyway) little focus on multi-user style security, at least for local users (the HPFS, and especially HPFS386 filesystems were excellent for multi-user security, including full support for extended attributes).
As for the single program locking up the entire system, that was a design decision in the Presentation Manager (the GUI API and program). It had a single input queue: all window messages went through a single queue. This has performance and usability advantages, especially when one window must modify or handle the messages for another.
However, yes, a single program that did not respond to messages could lock the GUI. The computer would run, but the GUI would be locked until you killed it.
That was changed in Warp 4.0. There were a number of user selectable ways that this could be addressed, depending on how much you might need the features of SIQ.
I'm not saying that OS/2 is perfect, or even valuable in the year 2004, but give me a break. You're talking about issues that were addressed between 6 and *12* years ago!
And the Workplace Shell features a level of object orientedness I have never experienced anyplace else, one that worked *extremely* well. The GUI was not pretty, but it was extremely robust, with a collection of very powerful features.
Re:All this talk... (Score:1)
I still would take Unix any day of the week over OS/2.
Re:All this talk... (Score:5, Insightful)
We hear this every couple of months, but let's look at the bigger picture for a minute. Why bother? OS/2 code might have been useful to the open source community half a decade ago, but by now we've made significant advances in every major area of operating system and user interface design -- there's simply nothing left in OS/2 that we can make any use of, because at this stage of the game we've already re-implemented it all.
IBM has, in fact, checked a bunch of stuff into the Linux kernel that the did own -- things like zero copy, etc. that may have been (among other places) in OS/2. So we actually did get the things which IBM owned and felt we could make use of. But if the whole OS/2 code base were opened tomorrow, I don't really think it would have much of an impact on anything. Maybe an SCO-style lawsuit from Microsoft, but not much in the technology realm.
Re:All this talk... (Score:2)
I don't know about that - I still haven't seen anything that remotely compares to the Workplace Shell in OS/2. It was so completely and elegantly object oriented. I'm pretty sure this is all IBM code too.
Re:All this talk... (Score:2)
Maybe in another decade.
(In practice, WPS will have to be redesigned/reimplemented, since the OS/2 version ran all in one process which is a showstopper these days, unless you use Java which brings it's own problems)
Re:All this talk... (Score:1)
Cloudscape, er Derby, is good stuff (Score:5, Interesting)
I've played with Cloudscape before and it's not as speedy as MySQL or as rugged as Oracle, but it does get the job done. And having a relational database right in the set top box or PDA means independence from a more heavy duty machine on the LAN, WiFi, etc.
Open source is just icing on the cake.
Re:Cloudscape, er Derby, is good stuff (Score:4, Interesting)
from IBM's site:
"it's just a 2-MB
from SQLite site:
"less than 250KB code space (gcc on i486)"
Re:Cloudscape, er Derby, is good stuff (Score:5, Interesting)
The cool thing about newer versions of Cloudscape is that it uses the exact same libraries as DB2 and I believe now supports all of the same datatypes.
So you can develop some small-scale application and run it on cloudscape, and then migrate it up to a DB2 system as your needs grow with minimal effort.
Re:Cloudscape, er Derby, is good stuff (Score:1)
note: im not saying thats a bad thing!
Re:Cloudscape, er Derby, is good stuff (Score:2, Interesting)
Obviously this was not only to be nice and get goodwill from the open source community, but to have a product that can get a foothold with small businesses instead of e.g. MySQL. The difference is of course that the direct upgrade path from Cloudscape is IBM's own DB2.
Re:Cloudscape, er Derby, is good stuff (Score:5, Informative)
A fairer comparison would be Hypersonic SQL [sourceforge.net], a free, open-source small (~100K) database server.
Re:Cloudscape, er Derby, is good stuff (Score:2)
"Fair" comparison (Score:4, Informative)
So, no, the comparison isn't fair at all.
Re:"Fair" comparison (Score:3, Interesting)
HSQL isn't entirely in memory. It writes a log file, and can have cached tables that are persisted to disk. Even non-cached tables are persisted through the log file. It does however keep all the indices in memory, and is limited to 2 GB of data, those are the real limitations.
It is faster though, primarily due to in memory indices, etc... For JDBC databases, I'm not sure anything is faster on small datasets, it certainly blew postgres out of the water by at least a full order of magnitude last time I use
Re:"Fair" comparison (Score:3)
Not entirely true. Yes, HSQL can run as a non-persistent, entirely-in-memory database. However, this is not the only mode, nor is it even the default mode. The standalone server and servlet modes both connect to physical database files, and therefore do perform real disk I/O.
Re:"Fair" comparison (Score:1, Informative)
Permanent storage... (Score:1)
Sun addresses the problem of slow disk writes (safe writes) under NFS with a NVRAM cache for writes and utility to 'clena up the mess'. It is possible to pull the power on one of these systems, bring it back up, and complete the transfer to disk. EMC and others do the same thing with their storage units. Lots of battery backed RAM for fast 'safe' transaction committal. It's not a software problem!
Re:Cloudscape, er Derby, is good stuff (Score:2)
Re:Cloudscape, er Derby, is good stuff (Score:3, Insightful)
Re:Cloudscape, er Derby, is good stuff (Score:3, Interesting)
Not sure Cloudscape actually qualifies as "lightweight". Current Java-capable handheld devices are still pretty limited in what they can run, in many cases you still have to go down to the C/C++ level to truly get the "pedal to the metal". Over time this will correct itself, of course, but for now the choices for data persistence in Java are very limited. Most people I know end up rolling their own solution on top of MIDP's Record Management System.
Eric
Eric's J2ME Pages [ericgiguere.com]
compatibility? (Score:5, Interesting)
Re:compatibility? (Score:2, Interesting)
Actually, he's got a reasonable point. Yes, in theory it *should* run on any Java, but GCJ isn't 100% there yet. It's coming along, though.
Re:compatibility? (Score:1, Interesting)
For people who believe in Java as a portability silver bullet, it may be suprising how much code in practice does such unportbale things. Otoh, coming from a C background, it's not that suprising: people will write their code to whatever they work with, in general. If it works for them, they'll ship it and ignore the l
Re:compatibility? (Score:2)
So, just for the hell of it, I try the timesheet app in FF - after installing the Sun JVM as you have to - and it works.. after a fashion. The combo boxes continually flick open and closed (so its keybopard only guys), and the edit fields won;t tab between them and so on - enough problems that I went back to doing my timesheets in IE... except
Re:compatibility? (Score:2)
Tools -> Internet Options -> Advanced -> (scroll, scroll, scroll) -> Java (Sun) -> uncheck "use Sun JVM 1.4.xxx blahdiblah"
I have the same problem to use Netegrity's Siteminder admin tool. Most annoying.
Re:compatibility? (Score:2)
No - this is a myth that strangely won't die.
Many Java implementations are 'clean room' and do not derive from Sun's code. HPs Java is an example.
Re:compatibility? (Score:3, Informative)
Whether you use a free implementation or a proprietary, it's your problem. There could be trouble finding a complete free Java implementation, but the GCJ team is working on it.
Gump is starting to do nightly builds on OSS (Score:4, Informative)
Cloudscape is a long way down the dependency graph, and you shouldnt expect it for a while. We need to get ant to boot first, which is seemingly a compiler problem.
Re:compatibility? (Score:2)
2. I am not a native english speaker, I tried to say about the opposite: that the "openness" of the spec is real right now, but not assured in the future.
3. I didn't say I knew the answer to the question, I was just stating in a subtle way that the question was stated in a wrong way. From the question it could be implied that incompatibilities could be the fault of the app developer, when they couldn't, if they
Re:compatibility? (Score:2, Informative)
vs HSQL? (Score:5, Interesting)
Re:vs HSQL? (Score:2, Informative)
How does Cloudscape/Derby compare with the other open source Java database engine, HSQL?
The big feature that Cloudscape has that I don't see on the HSQL page is XA support. Uninteresting unless you are working with a TM, but when you are XA can be the difference between "this could be made to work" and "this is a non-starter"
Re:vs HSQL? (Score:5, Informative)
Derby seems to be more of a traditional database, in comparison.
Re:vs HSQL? (Score:1)
But Derby's advantages are only really advantages if it's clusterable (I assume it is) and if you need to cluster (now or later).
Comments?
Sean
Re:vs HSQL? (Score:3, Interesting)
For large projects I use Oracle or PostgreSQL, but HSQL is ideal for inclusion with programs that need a database distributed with the program. It's easy to use, easy to add to a java program, and works like a charm.
Re:vs HSQL? (Score:5, Informative)
Here are some preliminary notes one of our engineers compiled while investigating adding Derby to our project. They were just preliminary notes so I make no guarantees as to accuracy but they might be helpful...
CHAR/VARCHAR/LONG VARCHAR
Derby strictly enforces the size specification in CHAR and VARCHAR fields. CHAR fields are space extended; non-space data the does not fit in the field raises an exception on insert or update. LONG VARCHAR data cannot be ordered, grouped, or indexed. (Really!) I believe that SQLServer (and possibly MySQL) has these stupid limitations, too. It may go all the back to the SQL-92 spec. HSQLDB, on the otherhand, ignores all size specifications, treating CHAR/VARCHAR/LONG VARCHAR as synonymns for java.lang.String.
TOP/LIMIT
Derby does not support the TOP or LIMIT syntax. There appears to be a "FIRST n ROWS ONLY" syntax that was added to DB2 that never found its way into Cloudscape.
Case sensitivity
Derby appears to treat all columns as case sensitive; and there appears no way to change this. HSQLDB, on the otherhand, can be configured on a field-by-field basis. (SET IGNORECASE is used for the database default; and VARCHAR_IGNORECASE is used as the data declaration.)
IDENTITY fields
Derby uses the bizarre syntax GENERATE ALWAYS AS IDENTITY. This also does not imply that the field is a primary key. So, "IDENTITY" in HSQLDB becomes "GENERATED ALWAYS AS IDENTITY PRIMARY KEY". Derby allows specification of initial value and increment.
GENERATE ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 2)
Performance
Derby is nearly instantaneous for COUNT(*) queries on databases with large number of rows. HSQLDB appears to count the rows, resulting in very poor performance. Derby appears to have a better architecture for large databases. Queries seem to run in time proportional to the size of the result set. Many simple HSQLDB queries run in time proportional to the size of the database.
CHECK constraints
Derby supports CHECK constraints, e.g.,
size INTEGER DEFAULT 0 NOT NULL CHECK (size >= 0)
disposition CHAR(1) DEFAULT '+' NOT NULL CHECK (disposition IN ('+', '-', 'B', 'M', 'Q'))
FOREIGN KEY constraints
Derby supports inline foreign key declarations with implied column matching, e.g.,
smtpID CHAR(17) NOT NULL REFERENCES InboxEvents ON DELETE CASCADE
HSQLDB requires table-level contraints with explicit column matching:
FOREIGN KEY (smtpID) REFERENCES InboxEvents (smtpID) ON DELETE CASCADE
Cheers,
Brien Voorhees
Red Condor
Corporate anti-spam gateway service for less than $2/user/month
I Know One Bug That Needs Fixed ASAP... (Score:5, Informative)
You have to blow away the dbcache directory to get it to start-up. It doesn't occur frequently, but it has happened more than once in an otherwise stable environment.
Re:I Know One Bug That Needs Fixed ASAP... (Score:2, Informative)
Re:I Know One Bug That Needs Fixed ASAP... (Score:1)
Is it a trend? I haven't seen it before. I just assumed the writer made a mistake. But if it is a trend it sure needs fixed.
This makes sense... (Score:4, Interesting)
Re:This makes sense... (Score:3, Informative)
IBM picked this up when they grabbed informix.
It is used extensively within IBM java based projects. (WSAD - the websphere IDE come with Cloudscape and works with cloudscape by default).
But its quite difficult to sell for two reasons.
One IBMs database brand is DB2, which these days scales down to small hardware.
Two cloudscapes biggest plus is that it is implemented as a single jar file, but, how do you collect license fees when anyone can copy and use your jar file?
The real reason they did it (Score:4, Informative)
The technology we are talking about is called App Play and guess what it uses for data syncronization?
It does not matter if they open sourced it since they where going to be puttting it on tons of clients anyhow.
Re:The real reason they did it (Score:2)
I wish Microsoft saw the world the way you do.
Differences with HSQL? (Score:1)
Re:Real Reasons (Score:1)
My reason for not using it (Score:1, Insightful)
If they had no cost for commercial prod usage, I would be very happy to give it a spin!
Re:My reason for not using it (Score:3, Informative)
explanation in plain english [apache.org]
A common and a consistent framework (Score:2, Funny)
Why? Because it's not DB2 (Score:4, Insightful)
IBM acquired Cloudscape as a by-product of acquiring Informix. Open sourcing it is one way to get rid of YADB (yet another database) to focus on their bread and butter, DB2. Probably not a bad deal for them in the sense that it generates lots of goodwill in the community at the same time. Not that I'm cynical or anything.
Eric
How to masquerade your browser [ericgiguere.com]
It is DB2 -- in a way ... (Score:1)
Re:It is DB2 -- in a way ... (Score:2, Informative)
Eric
McKoi, HypersonicSQL (Score:4, Insightful)
Personally I wish instead of more databases, the various vendors/projects would just decide on common freakin SQL syntax...it's embarassing and stupid to have to many copies of the same SQL scripts that are only slightly different because the keywords are slightly difference.
Great Business Move (Score:3)
Derby seemed like a step back... (Score:4, Informative)
...compared to hsqldb for my purposes. Hsql supports persisting Java objects directly into an Object-type column [preparedStatement.setObject(obj)]. Derby requires that you persist your object manually and stuff it into a (statically-sized) BLOB by manipulating streams - ick!
Also, hsql allowed ps.setObject(1, null) as a shortcut to ps.setNull(1, Types.). This was really handy.
It _looks_ like derby 10 claims JDBC 2.0 support; shouldn't it have the OBJECT data type?
Re:Derby seemed like a step back... (Score:4, Insightful)
The relational model supports any kind of data structure ever invented, but it's true that you sometimes want to store opaque blobs of data such as graphics or XML documents, and most databases, including Derby, supports that as well.
Perhaps you should look at an object-relational mapping layer such as Hibernate [hibernate.org] or TJDO [sourceforge.net], or a hybrid object database such as Prevayler [prevayler.org].
Real Database they should have open sourced (Score:3, Interesting)
Postgres/Java (Score:3, Interesting)
Re:Postgres/Java (Score:2)
Executive Summary (Score:2)
Sleepycat Berkeley DB Java Edition (Score:2)
I applaud IBM for OpenSourcing Cloudscape. Always good to have different alternatives, that are suited to different needs. For people looking for Java embedded databases, in addition to Derby, there is also Berkeley DB Java Edition [sleepycat.com] that you might be interested in. I don't work for sleepycat or have any connection to them or their products. I just remembered seeing this product awhile back and it came to mind when I read about Derby.
Anyhow, thought I'd toss that out for those interested in emb
Re:Because "Open Source" is the new buzz word? (Score:1)