I'm a Dapper Dan man

Well, that's definitely overstating the case - I'm no whizz with Dapper, and not exactly the spit of Ulysses Everett McGill. However I am becoming rather a fan, somewhat belatedly (I think I first heard about it from Mike Ellis some time ago). Anyway, before my SKOS experiments I hade my first proper play with Dapper to give an API to the great geophys survey data that English Heritage have made available. One nice thing was that the results aren't paginated (I just twigged how lucky that was), so without much trouble (given I was learning the tool) I came up with a quick dapp for the search. You can search the DB via this dapp using any of the parameters on the original form, though I only built four into the dapp, which you see on the test box - the reason being I wanted to search geographically to integrate the XML that's returned with some MOLA site data. But that only gives the barest info, and I wanted the details too really, so I made another dapp for the details, which takes the ID, and hooked them together for seach results with details. Very gratifying!

Next step is KML for this lot, but I will talk to the content owners before doing anything like that, which might put a load on their servers were I to layer their content onto our maps. One helpful thing, though, is that their app takes a parameter for the size of square to return results for, which means you could write a map application that would only query for the area you're showing at the time.

Shakespeare's first theatre

Here's the BBC story and video about a bit of news we'll probably see in a few other places soo, the discovery of what's thought to be the first theatre to show the work of William Shakespeare (and where he also worked as an actor). Though Museum of London Archaeology is going through a tough time at the moment they are, as ever, involved in some very exciting work. Maybe a little less romantic that Romeo and Juliet itself, but thrilling enough.

Museum of London API enhancements

I've been extending the MOLA publications database lately so that it now includes archaeological sites to which the publications relate. For pretty much all of these we have decent geographical point data (as we should, having dug most of them) and of course this is great for playing around with the data. So I've got a KML output going for that web API, which returns placemarks for sites corresponding to publications returned by your query. In the placemark are links to all the publications related to the site. You can also go the other way round: either query by the publication's ID to get its details and (in KML mode) related sites; or query by site, which returns simple XML for each related publication or, if you choose KML, returns all points related to books linked to that site (potentially lots of points, not just the one in your query). What it all means is that you can see your search results in Google Earth or on a map if you like by pasting in the URL for the KML e.g. http://www.museumoflondon.org.uk/museumoflondon/food/rest.aspx?source=pubs&mode=kml&period=roman
So far I've not put this into the public interface because I think to do so requires some consideration, but this will come. I'm pleased in part because I can start to use this API in my own behind-the-scenes integration. For example I've also just done a load of work on the site summaries that we publish for all the work that MOLA does. The KML for these is cleaned up and the old ASP/XSLT thing I did to search across these (and other reports) by borough has been refreshed. It now lets you search by site code, and now that I can get at publications via site code it's a pretty small step to get it also to return related publications, which should come very soon (perhaps before going-home time). Because they're in very different data sources (XML files and SQL Server) it wasn't so straight forward before, now it is. The publications API is mainly for internal consumption like this (it also runs the user-facing publications page itself, at a lower level). I'd be really pleased, though, if others find a use for this, and any thoughts about the rights and wrongs of how I've done it would be gratefully received.

KML to go

Well I've finally bridged the gap between our site summaries (which have long been available online) and GMaps (likewise). I told part of the story earlier - the summaries that our archaeology service write are compiled into an XML document (processed out of Word via a macro...), and transformed into HTML with XSLT. But because the location data is always in OS grid references it's no good for online mapping apps (which all like latitude and longitude). So I've been trying to find a way to get lat/long for the sites (which number many thousands) in order to let us plot the data for World And Dog, if they're not otherwise engaged.

Step one was to get a way to clean up the TQ-style OSGB data. Step two, adapt code/write a web service to enable me to pass that lot in and get back latitudes and longitudes. From there I needed to combine the resultant XML with the original site summaries. I tried doing stuff with Yahoo! Pipes but it wasn't too keen on my XML, or at least it wouldn't show me the items. Anyway, instead of that I thought I'd draw both datasets into one XSLT transformation and output KML, which is what I've done today (thanks in part to the inspiration of Raymond Yee's great "Pro Web 2.0 Mashups" book from Apress). I would have liked to just pass in a single variable (year) and go through all these steps automatically but it wasn't worth the hassle since every step needed a touch of hand-massaging on the data.

The KML includes all the site summary descriptive content. Looking at the resultant Google Maps I see there are some glitches, like things in the wrong place, and things without coordinates, and actually I need to check out 1999 which has a fatal error somewhere. I don't have time to fix these right now, but overall I'm pretty excited: at long last, we have a nice mapping interface for the public to look at all those thousands of excavations, desktop assessments, surveys etc. that MoLAS has conducted since 1992 (but not 2007 yet). Well I say all, in fact some of those from outside the London area are not included.

Now I'm hoping that someone will come and do something cool with the KML. In due course I'll have a go myself, but if you come up with anything please let me know!

So here are links to the functioning maps. Save them to My Maps and do something with the result!

[Edit: you can also see these embedded into our website. Here's 2006]

MoLAS site summaries 1992
MoLAS site summaries 1993
MoLAS site summaries 1994
MoLAS site summaries 1995
MoLAS site summaries 1996
MoLAS site summaries 1997
MoLAS site summaries 1998
[MoLAS site summaries 1999 - bust right now]
MoLAS site summaries 2000
MoLAS site summaries 2001
MoLAS site summaries 2002
MoLAS site summaries 2003
MoLAS site summaries 2004
MoLAS site summaries 2005
MoLAS site summaries 2006