Squeezing a Wikipedia Snapshot Onto an 8GB iPhone 169
blackbearnh writes with this excerpt from O'Reilly Radar "Think about Wikipedia, what some consider the most complete general survey of human knowledge we have at the moment. Now imagine squeezing it down to fit comfortably on an 8GB iPhone. Sound daunting? Well, that's just what Patrick Collison's Encyclopedia iPhone application does. App Store purchasers of Collison's open source application can browse and search the full text of Wikipedia when stuck in a plane, or trapped in the middle of nowhere (or, as defined by AT&T coverage...)"
Re:iPhone apps for computers (Score:4, Interesting)
FTFA:
Already done.
However, I'm not sure that I want precisely what this iPhone app is. It strips out references, and from the sound of things also the discussion pages. I'd say about 1/2 of articles I check the discussion pages to see what's really going on. Also he says he strips a lot of the metadata, and obviously images, none of which are things I"d want to give up (some of the metadata might be superfluous, but if I'm copying Wikipedia onto my computer, I want to copy Wikipedia onto my computer.)
I understand there are licensing issues with images, but even so, the SVG ought to be safe. And that wouldn't add as much of a disk space hit as the gifs, etc.
One of the other issues is the timing of Wikipedia dumps. They only do text-only dumps, and according to the article they only happen once every few months. It would be nice to implement an image review policy, and figure out a way to allow for mirrors (or just some increased bandwidth at Wikipedia HQ) so that we can actually have the entire English Wikipedia, regularly snapshotted and compressed, available for download. And really, for that kind of thing a 3-month or even yearly turnaround would be well worth the wait.
Re:Survey of Human Knowledge? (Score:4, Interesting)
In all seriousness, I'm starting to get extremely annoyed by what is IMHO flagrant abuse of the [citation needed] tag on Wikipedia, I don't know how many times I've seen it used in situations where it just wasn't needed. And I don't mean in "But anyone who spends all day working on FOO knows that BAR!" situations but more along the lines of "The earth orbits the sun[citation needed]." or even better "Sir NameOfArticle was in his day frequently regarded as a national hero in $COUNTRY.[citation needed]. <Six paragraphs that detail, with plenty of sources, exactly how famous Sir NameOfArticle was.>".
I've actually begun wondering if maybe there are certain individuals who are deliberately trolling Wikipedia by adding [citation needed] in places where it just doesn't belong and then sit around giggling as they read the discussion pages of various articles they've messed with.
/Mikael
Re:What a total geek.... (Score:5, Interesting)
You're right that this guy has flown the geek flag pretty high here; however, at least it's to some useful purpose. There are all kinds of facts about a country that are quite hard to discover just wandering about in it, and Wikipedia would be the ideal candidate to answer them.
Last time I went on holiday (to Australia) I came back with a dozen questions I wanted answering, just because I didn't have internet access while I was out there; Wikipedia access would answer many of these questions. Examples:
I could answer these questions by going into an internet cafe, but this isn't always possible. A portable Wikipedia sounds like a great idea.
XML Compression (Score:4, Interesting)
So, I'm reading here that they convert the XML into proprietary metadata and compress that.
Why not use EXI (Efficent XML Interchange) http://www.w3.org/XML/EXI/ [w3.org] which has been tested as more efficient that gzip and requires less memory to parse? Especially since the XML processing can remain the same, since the nodeset is the same.
EXCELLENT app, but limited (Score:3, Interesting)
I've been using this app for quite a while on my 1st gen iPod Touch, and it works and works well. It's amazing just how many articles it has. Other than some cosmetic and minor feature issues, the only real limitation is that Apple limits data file size to 2GB, so there is an obvious limit as to how much can go into the file. But it is amazingly complete. No images, no fancy tables--just text articles at your fingertips.
If you Jailbreak your iPhone/iPod Touch, then an excellent alternative is the Wiki2Touch app. Unfortunately, it seems that it's been pretty much abandoned in development, so it may be hit-or-miss if it works on OS v3.x. This implementation was REALLY slick. It provided a 4GB data file (that was much more complete) and a small Web server. You enabled the Web server, fired up Safari, and pointed it to a local URL. The app presented quick and very readable articles. And if you went to the trouble to download and process, you could also add about 4GB of image files to make things more complete (on a larger-capacity device, of course.)
Here's a review that I posted for both apps just over a year ago on my iPod Touch Tips site:
http://jimstips.com/ipod-touch-tips/ipod-touch-review-wikpedia-on-your-ipod-touch.html [jimstips.com]
In both cases, the main complaint is updating. In order to update the data file, you have to re-download the data, and depending on the app, you are typically at the mercy of the developer to provide an update. Otherwise, you had to download, index, and install the HUGE files yourself.
If you absolutely HAVE to have updated, offline data, check out the Wikipanion app. It's a nice compromise.
Re:Profits (Score:3, Interesting)
I hate when companies don't just come out and say it explicitly. It makes me think they might just be paying a penny on the dollar so they can play the "philanthropy" card. I like that Target Corp clearly states that "5% of our profits" go to charity (admittedly, much of this may be in the form of product donations, but still.)
http://en.wikipedia.org/wiki/Target_Corporation#Philanthropy [wikipedia.org]