Rollin' along with the tumblin' tumbleweed

Well I know it's been pretty quiet on this blog of late, something to do with too much going on at work to think much, and too exhausted in the evenings to do much (aside from fill up the new iPod). And now it's going to get a level quieter still as I'm on leave for a few weeks. Just watch that tumbleweed roll by. Adios for now.

Why Gnip caught my eye: a bit more depth (just a bit)

Eric Marcoullier commented on my last post on Gnip and I wrote him the following e-mail because, as I say, it's about time I worked through a little bit the reason why his baby caught my attention (not that it's a particularly worked through working through, but hey, it's a start). It was a bit much for a comment but enough for a post, so here you go.



*******************************************************************

Many thanks for taking the time to look at my brief notes, you must be a busy man so I really appreciate it. It's definitely time I tried to put some flesh on the bones because it's true, I've barely sketched the link between Gnip and my own preoccupations.

My research is looking at how museums keep their digital stuff useful; in other words, how and when we keep on trying to squeeze value out of the digital stuff we've invested in. I'm trying to put a particularly museum-y spin on it because it would be all too easy to look, for example, at general questions related to digital preservation (yawn). Hence I'm exploring the specific conditions and challenges that museums have to face, as well as the way in which they value what they hold - as a "memory institution" with a remit to preserve and to serve the public, a museum has potentially got a slightly different way of valuing what it holds, though arguably this won't really apply to digital material except in special cases (like digital art). So that's the basic thread of my research: looking at how museums can and do decide a strategy for maximising value from their digital assets, and for planning new ones.

Of course, no museum is an island (that's kind of the point of the 'net, right?) and I'm inevitably thinking a lot about the relationships between museums and other parties that might provide or use services and data to/from them - this is key to extracting value, but it's also a dependency for which we need to understand the risks. In the museum community, a lot of the talk (for a couple of decades or more, now) is about how we share our most obvious USP: our collections data. Loads of work has been done on this and yet we still seem to be a long way from the dream of a way of effectively integrating the collections of more than a few institutions. This is why I've been working with the European Digital Library/Europeana project. The reason that Gnip caught my eye was because it suggests another model for data interchange. It may be not be appropriate for the scenario of sharing collections data, and one could argue that in some ways other museum initiatives share some of its characteristics (federated search, metadata harvesters etc.), but I was interested in whether we could learn from the model of a neutral mediating agent as rather than a central pool of data. We're not short of standards but we are short of co-ordinating mechanisms that we can all trust and feel we leave us with some control over "our" data.

The actual purpose of Gnip as an exchange for social data was probably of secondary interest to me, but of interest all the same - it's just an area I don't know much about. I think that on the whole museums won't need to concerns themselves directly about how whatever it is they do will relate to Gnip: I presume that if they incorporate a third party service in their site, or perhaps have an installation of WordPress, then a lot of the mechanics may be dealt with already (or will be in due course). But concern about interoperability and data portability may well be a reason why many museums (my own included) haven't yet done an awful lot with social software - although there are some notable exceptions. If Gnip helps to address these concerns then all that will still be lacking is our imagination!

One other possibility is that museum applications could indeed work with Gnip to integrate individuals' public information with their own services - say, by drawing links between a person's list of interests or music preferences, and what's in a museum's (or a library's)collection; or by suggesting events to attend based on user location, age and interests. I don't understand Gnip well enough to know if this is plausible, though, but it's an intriguing prospect.

*******************************************************************



Thanks again to Eric for taking the time to contact me, I think it speaks well of new ventures like this (OpenCalais was another) when the key people go out out of their way to make contact with the people that are talking about them.

IMLS looks to the future

Once again, kudos to Nina Simon for her latest post, Notes from the Future: Reflections on the IMLS Meeting on Museums and Libraries in the 21st Century. I have to admit I've not read it thoroughly yet but (a) there's lots going on in the IMLS study that overlaps with my research interests and (b) as usual, she has a pretty strong take on it and interesting things to say, and it's productive perhaps to triangulate between her strong perspective and the equally strong conservative perspective she cites. Have a look. If I have time I'll try to digest it properly and give a proper response - both here and on her blog since they want our thoughts (perhaps even non-US thoughts...)

O'Reilly Radar on Gnip

As I mentioned last week, Gnip looks to be at least of tangential interest to use in museum tech, whether because we may be involved in social software, or as a model for addressing the sort of data interchange problems we face (though of course there are alternative approaches). Jim Stodgill has written an article on O'Reilly Radar of which I understand about 30%, but which looks at how Gnip compares to enterprise service bus, in the problems it's tackling and the solution it offers. It's worth a read.

Delayed post #1: aspects of [web] preservation

[[this post never got finished but I'm having a clear-out of my drafts and they're getting published or deleted, ready or not]]

Brian Kelly just blogged [[hmm, well, back in July, I think....]] on the JISC-PoWR site about "three key aspects of web preservation": experience, information, and access. I have a fourth, but I've been using it in the context of "sustainability" (the subject of my thesis), and so first I want to say a couple of words about this vs preservation, since although I've been writing papers for Ross for a couple of years arguing about the distinction between the concepts, I've not really rehearsed this in public before.


For the last couple of years I've been arguing that the problem of sustainability (S) is distinct from that of archive-style preservation (P). I won't go into the details of the distinction here but in essence I see P as concerning the persistence of a state, and S as the persistence of a process or activity. Recent work by Chris Rusbridge and others has been blurring the boundary ever more, although in a useful way: by questioning the purpose of preservation and weighing up what are the important aspects ("significant properties") of resources, they have been starting to argue for an approach to preservation that looks a lot more like what I was previously describing as sustaining. I still sort of believe that it is useful to distinnguish between the two concepts, but there's a lot of overlap.


The aspect that I think is especially pertinent to sustaining is "purpose". I don't think this is the same as the "experience" that Brian cites Kevin Ashley on (although experience and information may feed into purpose). It's focused on the objectives of the resource, which may be attainable through radically different experiences; for example (in the case of the environment in which KA operates), the learning objectives that an educational resource was created to serve. For a museum, perhaps a resource was prepared for use in a temporary exhibition, with the objective of enriching the experience visitors to that physical space by illustrating relationships betweeen objects, and providing media resources to bring them to life. When that exhibition closes, the original objective is partially voided - there is no physical visit to enrich - but aspects of it may still be viable - the objects probably still exist and the tales about them are still worth telling, perhaps more so than ever since we've stuck them back in the store-room out of easy access. Brian was talking about web preservation, of course, and I've taken a non-web resource as an example, but my interest in the question of sustainability extends beyond the web and the point applies regardless.

In any given digital resource that we're talking about preserving/sustaining, experience and information at least (perhaps access, too, sometimes) will have contributed to the original purpose to varying degrees - sometimes the experience is the whole purpose, sometimes it's an unimportant side effect of providing access to information. And sometimes it's important to the preserver regardless of its significance to the original purpose.

So if the original purpose is no longer served by a resource, what then for our "preservation" plans? There are still often reasons to preserve (freeze) or sustain (keep alive) an application, or aspects of it - in other words, some sources of value, ranging from historical interest to new uses, which may let a resource adapt and survive. The "significant properties" idea fits in to this. For me, if you are trying to maintain some quality of the original it's more of a preservation activity; whereas if you are more fundamentally interested in continuing to extract value through maintaining utility of whatever sort, we're talking sustaining.

Short cuts

Little more than bookmarks from today:

Short URL/snapshot/citation/bookmarking apps:

• bit.ly http://bit.ly/go
• WebCite http://www.webcitation.org/ (courtesy of Seb Chan)

Google's new VW lively. Is it cool or does it suck? Some debate. RWW techcrunch Raph Koster

I need to find out more about Lexara and Project SILVER: http://www.lexara.com/lexara/project-silver/ and http://www.silvereducation.org/

EDLocal >> EuropeanaLocal

Fleur says: EDLocal becomes EuropeanaLocal. I'm looking forward to hearing what else came out of the recent launch of the project, which will hopefully help to smooth the process of participating in Europeana for institutions of all sizes. I'll keep you posted (but I expect Fleur will do a better job so keep an eye on her blog!)

Two interesting ReadWriteWeb stories

Just a quickie. Two things caught my eye on RWW:
Confirmed: Microsoft Acquires Powerset. So MS moves into semantic search with the acquisition of a promising startup. Yahoo! will have to fight ever harder for survival if MS is really determined to do this. Anyway, it may be time to start talking to MS about how to work with all the yummy structured data we have in MLAs (yes, that means you EDL)
Gnip: Grand Central Station for the Social Web. This is all about tackling the myriad interfaces and data formats of social software APIs. It's a comparable problem to that faced in our sector and the solution of a mediator is interesting mainly for that reason. It's also potentially directly relevant where we're working with data in social sites.

Showing us the way

I presume it's uncontroversial to say that it would be useful to have terminologies available as web services. Right now, you can browse various sources of reference terms that are useful to museums (amongst others): sources like the british & irish archaeological bibliography (including its approved term lists); the National Monument Record Thesauri; and the museum codes and SPECTRUM terminology termbank maintained by the Collections Trust (to highlight some UK examples). I'm certain it would be useful to have these available as web services (some more so than others); likewise other thesauri that AFAIK aren't available to programme to: ULAN and AAT, for example, which are collected under the CCO initiative.

There's every chance that I misunderstand some or all of these "services" in terms of how they're used and by whom (I'm very shaky on the status of CCO and its relationship to AAT, for a start). But I'm sure that there are many ways in which a programmatic interface to their contents could be used. Which is why (to get to the point of this post) the service that OCLC's top geeks have come up with here is a great example for us in the museum world to look at (blogged here on Hanging Together). This is a collection of esssentially library-related thesauri onto which they have created web services. I like the look of FAST best of all; it could be really useful for us in the dusty bones world too.

Lorcan Dempsey also blogs today about the WorldCat identities API and other cool services. I fancy the name look-up service: aside from anything else it gives us a URL to refer to for those individuals in the WorldCat database. Here's a not-so-random entry.


So once again we can learn a lot from libraries leading the way. One of the cool things, though, is that OCLC are a cross-domain organisation, and people like Dempsey think constantly about breaking down the barriers between libraries, archives and museums. If they've sprinkled their magic onto library terminologies, I'm sure they'll be only too happy to help the museum world to take similar steps.

Mike Dunn on "technical due dilligence"

Extracted from an interview with Mike Dunn of Hearst, published on ReadWrite Web, these are the top five aspects of "technical due dilligence" that Dunn looks for when considering a startup for acquisition.

1. The primary things I look for are a thorough understanding of a company's current technology state and a roadmap of their future. I then fill in the building blocks to paint a picture of the company and its structure via the next 4 areas.
   
2. Staffing: The company should have a proper ratio of dedicated to outsourced staff. The focus for in-house staff should be on owning and extending the company's value-add. The focus of the outsourced staff / service should be on areas where technology is available at a reasonable price.
   
3. Infrastructure and Architectural: I look for alignment between the infrastructure in place and their roadmap. I try to understand their architecture, i.e., have they designed something that will be stable, yet scale and grow as their business requires? Have they over or under built, are their investments proper for current state and extensible as their growth requires?
   
4. Workflow and Processes: This is usually the hardest part of my interviews with startups because while most have ways they do things, they often aren't comfortable expressing them. They also aren't normally done in a way that's repeatable to the point where they could be called a workflow. This is OK. As they mature, standardized workflows and processes will be established, normally out of a necessity to ensure they're providing a stable environment that doesn't get negatively affected as they introduce change.
   
5. Costs: This is the spreadsheet part of the conversations. What has been spent to get them to the point they're at, what do they need to spend near term, possibly with funding from my company, and what do they envision they'll need to spend? I look for a grounded approach to spending.
   

With a little flipping around these consideration probably apply just as well to museums.

Conference ketchup

Well it's been a pretty busy time. After many years of avoiding presenting at conferences, following a number of crappy performances in '99, I bit the bullets kindly shot at me by Ross and Jill and opened my cakehole to several hundred unfortunate captives, first at the UK Museums on the Web conference in Leicester, and then at the EDL plenary conference in the Hague. And I'm truly grateful to both Ross and Jill for the opportunity to do this: it's very flattering, humbling, really, that they felt I'd have something worth saying to such informed and inquisitive audiences.

In the end, nervous anticipation gave way to the onrush of time and once I was up there in front of faces familiar and not I felt a more at ease than I would have expected. Having listened to the recordings, well, there were a lot more "ums" and "errs" than ideal, but hey, I didn't forget too many things and I kept pretty close to time, which is a big improvement on my earlier debacles.

So what was I talking about? In Leicester, I talked about Europeana. It was not meant to be an overview as such (that's not really my role), but an account of my involvement and interest, focussing on my hopes for the project and, of course, the role that APIs play in that. During Q&As and coffee breaks I had a lot of really useful feedback to my question: what is stopping many more UK museums from getting involved in the project? On the whole these revolved around the burden and mechanics of providing data, which was pretty much as I suspected. It's made me more determined to do what I can to simplify these processes, but also to ensure that the pay-off to partners is as high as it can be and as well understood as possible. Perhaps we have the furthest to go to achieve the latter.

At the Koninklijke Bibliotheek in the Hague I had an even shorter slot, which was fine by me, as part of a panel whose other members were intimidatingly illustrious. The subject of the conference was "Users expect the interoperable", and this particular session had two panels discussing interoperability in relation to archives and museums, respectively. I took part in the latter panel. I still don't know if I actually said anything, really, because I had little in the way of conclusions to offer: I just teased out some ways in which I thought "interoperability" questions pertained to APIs in a museum context. I also looked at a few examples from the world of semantic enrichment - a strange choice, perhaps, but made because there are really no proper museum APIs to compare to, and in order to show that a lack of standardisation in that area is no barrier to those APIs (Calais, Hakia, and Yahoo! Term Extractor) being useful. Simplicity gets you a long way, as does the use of existing data formats (e.g. DC or microformats). These also fit well with the other drum I was banging, the services that EDL could offer to contributors and third parties for enriching content. So, a kind of bitty talk but at least it was brief!

On Tuesday the conference wrapped up (and I do want to talk a lot more about it ASAP, because apart from anything else the first prototype was shown off and it's COOL!). I attended a hurried meeting of WP1 and Harry Verweyen presented his paper on the business model. I think he's done a great job, although this is so far outside my area of comptence I scarcely dare comment. He'd also done a lot of work integrating some of my suggestions into the plan, and it became still clearer to me how much of this hangs off the success of the semantic web tech part of the project.

Both conferences were really rewarding in their own ways and I'll try to offer some proper notes from them as soon as I find my feet again.

Hakia, semantic enrichment, and EDL

Moving in the same direction as Reuters, with its OpenCalais service, Hakia has started offering two new APIs, one related to search and the other to content summarising and enhancement (see RWW's story). Perhaps it has some way to go before this is a really useful service in terms of the quality of its output (if RWW's experience is any guide) but it's early days. In any case, putting this alongside Calais and Yahoo!'s Term Extractor (not to mention other semantic enhancement services extracting, for example, location data), this shows at least that there are quite a few people out there that think there's a market for this sort of service.

Semantic enhancement (as well as data validation) is a service that I've mooted as a possibility for Europeana. With a specialist and very authoritative data set, it could appeal to those needing to enrich cultural heritage content. There may not be a lot of money in it, but as Harry Verwayen pointed out to me, that's not necessarily the only benefit to the service provider (or I doubt Reuters would be in this game). Building traffic around the site and strengthening the brand is a benefit. Similarly, increasing the use of the ontology/thesauri used by EDL increases its influence.

Harry is putting together a presentation for next week's WG1 meeting after the EDL plenary, and he's generously put my name on the front too although I have little to contribute beyond these slightly flaky suggestions. We'll be throwing these ideas into the mix in a discussion of business models for EDL when it goes live.

Clarification

Following my off-the-cuff post about the Museum of London's strike (in which I did not participate, not being a member of a union), it's become apparent that there is some confusion as to the status of this blog. It's important, therefore, that I make it very clear that the contents of The Doofer Call are a personal expression only. This is a research diary for my PhD that I've chosen to make public in order to engage in debate and help people find things that interest me; my profile states this. However, because I talk regularly about things I'm doing at the Museum I can see that casual readers might mistakenly believe that I am writing in an official capacity, or with official sanction, and any confusion is my fault and for me to address. Hence this post.

Friday's missive may turn out to be factually incorrect (one might argue that the pay at MoL is not rubbish, or that the pay rise was not over a year late) or inaccurate in its representation of the reasons for the disgruntlement that led to the unions striking, but that's not really the point. What's important is that it's clear that what I wrote was not an official press release, but the completely unofficial, independent expression of someone who is also employee, written at home, out of office hours (like this).

So for clarity, I am posting a disclaimer that can also be read as a declaration of interests. Please read it here.

My apologies to anyone who previously mistook this blog to be anything other than my thoughts in my words, I hope this has made things clearer.

Note: all of the organisations with which I have a formal relationship are included as tags on this post in the hope that people searching for the organisations and stumbling across this blog will also see this clarification.

Disclaimer and declaration of interests

The following are organisations with which I have a formal relationship (most of which I've written about at some point). The opinions expressed on this blog are mine alone, and are not to be attributed on the basis of this blog to any of these organisations:

1. Museum Of London, my employer. This includes Museum in Docklands and MoLAS. The Museum is also a contracted partner in my PhD


2. the City of London Corporation write the contracts and the pay cheques, and the Greater London Authority are 50% partners in the Museum


3. University of Leicester, Department of Museum Studies, at which I am studying for a PhD


4. Lexara (and previously Simulacra and MWR), the non-academic partner in my PhD. Lexara also supply some technology to the Museum of London.


5. Richard de Clare Primary School, where I am a parent governor with oversight over ICT across the curriculum


6. EDLNet, the EC project for which I am the official representative of the Museum of London

None of the above sanctions or previews anything that I write here. Equally, you should read my remarks in the light of my committments to them all.

If anything written here is inaccurate or unclear, please contact me directly or through comments about changing it.

Victorian photographers API now also alphatastic

Didn't manage (didn't dare) to upload this before going-home time on Friday. There is now another string to the bow of the nascent REST alpharama here. You can query the people in the database of 19th Century photographic London (pardon the contraction) at the PhotoLondon website we launched recently. Right now, it only sends you back a list of people matching your criteria (100 records per page) with no details. I will soon produce a detailed machine-friendly record page for each person (an example of a human-friendly one is here)

The querystring takes most of parameters you can see on the form including surname, forename, search text (but not multiple words like the form), gender, year of birth/death, "alive in..." year, place of origin, photographic occupation, non-photographic occupation, and presence of attached images. Here's an example search so you can see what you'll get back right now. Here's a wider one.

There are bugs, no doubt, and it will be of more use when I can send you the person's details, as well as a list of countries and occupations to build queries from. I realise I also need to write some proper documentation. Still, it's a start. Please tell me if it might be useful to you, and what you'd like to see it doing/do with it.

Guardian bigs up the 24 Hour Museum

Jack Schofield today writes up 24HM here. It's funny timing - as he himself points out, it's looking at a relaunch pretty soon - but perhaps there are clues in that as to the timing of this review. It's a pretty positive piece on the whole but suggests that "Culture 24 could be so much better" (after its relaunch). I'm as intrigued as JS to know what's planned - and whether, for example, it will be able to consume/aggregate RSS feeds as well as publish them!

Small API update

A couple of small advances on the API front (again, see here)

• fixed a bug on the geo thing. For some reason an imbecilic code error wasn't breaking the script on my machine, but did on the web server. Now fixed.


• a CDWALite-lite output for individual object records (example). There's more to add, glitches to fix, and ideally a better solution to the URL, but it's a start. Next thing is a search interface but that depends upon agreement within the Museum. A good solution may be to combine CDWALite and OpenSearch-style RSS, with the records enabling users to find the data end-point, as well as the HTML rendering. In due course I'll probably add tags to HTML record pages to point at data like this, or I may do it with some POSH.


• the photoLondon website data now has a basic API, which I'll put on the live site next week. It returns basic person details and search parameters include: surname, forename, keyword, birth year, death year, "alive in" year, gender, country of origin, photographic occupation and non-photographic occupation. I'll work on the search result format soon, as well as the person details.

MoL APIs live (but very alpha)

Well there's more to do, but an alpha version of three services is now there if you want to play. All of them currently put out only XML, no JSON or raw text, GEDCOM or whatever else, but this may change and new services may be added. Have a read here. There is an events database, publications from our Archaeology Service, and a sort of geocoding tool. I've rejigged the code so that adding a different (XML) output format to these involves just writing XSLT and putting in a new value for the "mode" parameter, rather than fiddling around with C#, recompiling and all that. All feedback most welcome. But don't bug me about a collections API!

To start you off, here are links to one request from each service.

• events API. This won't work forever as the events will expire. Uses the format that Upcoming outputs with xCal, DC and geo extensions
• publications API
• the geothingy converter whatsit

The browser on the server

I've been wondering about the possibility of running JavaScript outside the browser, in fact as part of the web server. Of course, old-style ASP could be written with that language but that's not really the same - what I want is the ability to take and manipulate objects on the server that you'd otherwise be expected to work with in a browser context. I was thinking initially about JSON - I know that it's perfectly possible to use JSON from other languages (libraries exist for PHP, Python, .Net etc) but wouldn't it be better just to do it with a JavaScript engine? And then I thought, well, Mozilla's engine would be the obvious thing, I presume they've built it in such a way it can run in other contexts. So I started googling and obviously this is a very well-trodden trail that I'm just late in exploring: there's loads of stuff out there. I've not checked out any of it properly but thought I'd put some links here in case anyone else is toying with the same idea, or is prompted to now.

Incidentally, one of the things that's nice about JavaScript is that it's all focussed around the DOM, unlike many other languages, and I'd think it's pretty forgiving too (depending on the engine). All of which makes it a good candidate for screen scraping. And one thing I've not yet found is a way to tie this into .Net nicely

• http://en.wikipedia.org/wiki/Server-side_JavaScript This lists loads of products for various environments that will run JS server-side. Looks like Mozilla's Rhino and SpiderMonkey products are the basis for most of them.
• http://ejohn.org/blog/server-side-javascript-with-jaxer/. This must be the first stop for me. How cool is that? Here's jaxer itself: http://www.aptana.com/jaxer/. Yes, it is based on Mozilla. Damn, it's for Apache, but it's not a deal-breaker.
• from the same source but older: http://ejohn.org/blog/bringing-the-browser-to-the-server/ Look in the comments too, plenty of starting points there
• I also turned up some of what I take to be the original Netscape JavaScript documentation, which talks a lot about server-side javascript but in relation to the old Netscape Enterprise Server

Made an API

No, not a collections database interface, Mia's onto that one. But I'm scheduled to work on the events database about now, and with the mashup day (now full up, I think) coming it seemed a good idea to grab a day and finish off this job. For ages we've had an RSS feed which gets a lot of use, including consumption by sites like dockland.co.uk, and of course search functionality onto the events web pages, but the feed is (a) a bit basic and (b) just shows everything for the next 14 days.
I wanted to turn the static feed into a properly searchable REST interface, as well as load it with more good data. So I plugged a hole whereby only a start date could be specified, put in filters for audience, event type, and keyword (like the web pages), and neatened up the separation into XML. Now it can be churned out in raw straight-from-the-database XML or in RSS 2.0 with extra hCalendar. I've got a couple of tweaks to make (adding in addresses and geo data) and then will invite the world. The plan is that you can set the format in the query string, so RSS2 will be one option but so will xCal (as used in Upcoming's XML) and whatever else - it'll just pick up a stylesheet and apply it on the fly. There are more filters I could apply, and documentation to do. Then thinking about pagination with OpenSearch extensions... and then who knows. Most of it will wait, so I'll post soon with the details on the where and how.