Amazon Exposes 1 Terrabyte of Public Data

Hmm, never did anything with this when I saw it and amn't going to. It's a cool development, though: lots of new datasets through Amazon from (mainly) US sources, from DNA sequences to census and traffic data. It's more evidence that Amazon is shaping up as Google's real competition in the coming years, not Microsoft or Yahoo! (put that one in for those asleep at the back).

What can we offer from museums?

Anyway, here's the link to the RWW story: Amazon Exposes 1 Terrabyte of Public Data - ReadWriteWeb

Google kills Lively, advises on digital preservation

Google's well-reported putting-out-of-misery of Lively was announced here, where pragmatically and somewhat amusingly they advise on the digital preservation methods available to those poor suckers who took a chance on a Google Labs experiment. It's pretty simple, really:
"We'd encourage all Lively users to capture your hard work by taking videos and screenshots of your rooms"
Fair enough. I think plenty of DP types would accept that at the moment there aren't that many options for virtual worlds. It's certainly another argument for improving the transportability of material between VWs environments, but presumably this would mean compromising on innovation to some extent. Hey ho, there's plenty more VWs to go around (and I never go to any of them. Not exactly an early adopter, me)

International projects in the international legal minefield

This is but a (delayed) note to remind myself about the challenges that projects like Europeana face in integrating assets from, and providing services to, several countries with only partially-harmonised legal systems. This was brought home to me by a recent German ruling on use of thumbnails by search engines. Amalyah Keshet (on the MCN mailing list) cited the following snippet from Arc Technica's post on the ruling:

"As much as people complain about the challenges of balancing copyrights and fair use in the US, overseas courts have been happy to provide examples that remind us that some aspects of US copyright law are actually fairly liberal. The latest such reminder comes courtesy of a case in Germany that revisits an issue that appears settled in the US: the right of image search services to create thumbnails from copyrighted works to display with the search results. The German courts have now determined that this is not OK in Germany, where Google has just lost two copyright suits over image thumbnails..."

To Leuven (expenses paid), but who will pay for EDL?

Tomorrow sees EDL's working group 1 meeting at the Katholieke Universiteit in Leuven, not far from Brussels. Quite exciting to be visiting, albeit fleetingly, a place that played host to Matsys, Bouts, Erasmus and Vesalius, amongst others (not to mention, apparently, the infamous AQ Khan). I'm looking forward to attending, though with no expectation of being able to contribute a lot since this group covers different ground to the one I've worked with up till now. I'm not even sure if I'm part of the group or simply in attendance. Anyway, the meeting will look at progress with Europeana so far, and consider the business issues facing it, particularly how to move to phase 2 (i.e. following the prototype, to be launched in November) and how to build a sustainable future. Working my way through 100-odd pages of reading matter in preparation for the meeting, I'm struck by how big a challenge it will be to find the necessary ongoing resources, but also the fact that they are tackling the problem head-on and examining a wide variety of options, from direct subsidy, through subscription by contributors or users, to corporate partnership or sponsorship.

A significant factor in the search for revenue-raising avenues is the fact that Europeana is not going to be a content owner in any significant way, but rather a broker/facilitator for accessing content owned by others. One possibility that I believe it could be worth exploring for two reasons is some form of partnership with a search provider. Yahoo! may be a bit too distracted to talk at the moment, but along with Google could be productive partners. Both sides could benefit by working on an interface and aligning their data structures, and EDL could perhaps offer quite a bit to such a partner in terms of preferential access to the semantically enriched data it will hold. This might be directly to do with searching the resources in EDL, or it might be, say, helping to clean up datasets of people and places. In exchange, maybe either some cash or technological assistance? Perhaps some of the semantic-y startups currently taking wing could also be interesting to work with, but they won't be as well resourced. Cultural heritage organisations have a lot of knowledge and context to offer here so maybe there's a business model to be had.

KML to go

Well I've finally bridged the gap between our site summaries (which have long been available online) and GMaps (likewise). I told part of the story earlier - the summaries that our archaeology service write are compiled into an XML document (processed out of Word via a macro...), and transformed into HTML with XSLT. But because the location data is always in OS grid references it's no good for online mapping apps (which all like latitude and longitude). So I've been trying to find a way to get lat/long for the sites (which number many thousands) in order to let us plot the data for World And Dog, if they're not otherwise engaged.

Step one was to get a way to clean up the TQ-style OSGB data. Step two, adapt code/write a web service to enable me to pass that lot in and get back latitudes and longitudes. From there I needed to combine the resultant XML with the original site summaries. I tried doing stuff with Yahoo! Pipes but it wasn't too keen on my XML, or at least it wouldn't show me the items. Anyway, instead of that I thought I'd draw both datasets into one XSLT transformation and output KML, which is what I've done today (thanks in part to the inspiration of Raymond Yee's great "Pro Web 2.0 Mashups" book from Apress). I would have liked to just pass in a single variable (year) and go through all these steps automatically but it wasn't worth the hassle since every step needed a touch of hand-massaging on the data.

The KML includes all the site summary descriptive content. Looking at the resultant Google Maps I see there are some glitches, like things in the wrong place, and things without coordinates, and actually I need to check out 1999 which has a fatal error somewhere. I don't have time to fix these right now, but overall I'm pretty excited: at long last, we have a nice mapping interface for the public to look at all those thousands of excavations, desktop assessments, surveys etc. that MoLAS has conducted since 1992 (but not 2007 yet). Well I say all, in fact some of those from outside the London area are not included.

Now I'm hoping that someone will come and do something cool with the KML. In due course I'll have a go myself, but if you come up with anything please let me know!

So here are links to the functioning maps. Save them to My Maps and do something with the result!

[Edit: you can also see these embedded into our website. Here's 2006]

MoLAS site summaries 1992
MoLAS site summaries 1993
MoLAS site summaries 1994
MoLAS site summaries 1995
MoLAS site summaries 1996
MoLAS site summaries 1997
MoLAS site summaries 1998
[MoLAS site summaries 1999 - bust right now]
MoLAS site summaries 2000
MoLAS site summaries 2001
MoLAS site summaries 2002
MoLAS site summaries 2003
MoLAS site summaries 2004
MoLAS site summaries 2005
MoLAS site summaries 2006

Bookmarklet update

OK, there are problems with with prompting bookmarklet in IE. It's all to do with the prompt. Yesterday it worked, but only on the first use, then it would stop prompting for the language-pair value (and in fact ignore the default value I'd put in there) and just skip straight to the translation page, which, without a language pair, can't do much. I think it may be to do with security, since it occasionally shows that "website trying to show active content" warning for a moment before scooting straight off to Google without a by-your-leave, let alone that prompt.

So for IE, for now, I'm just using a straightforward Italian-English bookmarklet
For Mozilla, the prompting version is now also Google-based and goes there. Here it is: translate

KML goes open

ReadWriteWeb comments on Google's announcement that KML is being handed over to the Open Geospatial Consortium. As RWW says: "For something as boring and painful as it is - standards work is very sexy". This gives us all more confidence that it is a format that's going somewhere and should be reliable for a good while to come. No more hangups about being too tied to Google's proprietary format. Cool!

If size doesn't matter, perhaps you're just not measuring it right

Oh thank you Google!
http://www.theregister.co.uk/2007/09/25/mesopotamian_nil_cubit/

Google's stab at the online/offline thing

To join the various announcements from Adobe, MS, Dojo et al, here is Google Gears, a browser plugin and API (well, I believe 3 APIs) to enable online applications to work offline too. They are apparently working with Adobe to integrate with Flex but we need to see how all this shakes out in terms of standardisation. The plugin and SDK are released under BSD.
As always, what's the relevance to my project? Well, for one thing there's the changing nature of applications - where they reside, who has power over them, what they can do, how we maintain our identity in them. There are questions, as I say, related to standards or the need for them, and what their absence can mean for the long-term viability of applications built without them. And of course it's pretty cool tech.