About Me

My photo
Web person at the Imperial War Museum, just completed PhD about digital sustainability in museums (the original motivation for this blog was as my research diary). Posting occasionally, and usually museum tech stuff but prone to stray. I welcome comments if you want to take anything further. These are my opinions and should not be attributed to my employer or anyone else (unless they thought of them too). Twitter: @jottevanger

Friday, April 29, 2011

What are they building in there?

I started this weeks ago and have decided I should just get it out there rather than finish it! That's the main lesson I've mis-learned from agile, although I've mis-learned lots more besides.

The work of the New Media department at the IWM is very complex at the moment, as it has been since I started almost a year ago. We're now a pretty large department and I'm not deeply involved in all aspects of what we do, but we're also a tight team and I think we all keep an eye on what's happening in other areas - which is great and necessary, but sometimes it all gets pretty confusing! Part of our team needs to focus on keeping our existing sites going a little longer, but they too are also working on the new developments. So with so much going on I thought I'd step back for a moment and try to lay it out for my own benefit as much as anyone's - it's very easy to get lost in the many strands of activity.

We're working on a number of overlapping projects under our direct control, but we also have crucial dependencies upon projects that belong to other departments. For us the flagship project - the one where the whole organisation is watching us - is the redevelopment of our core websites, which will launch later this year. This is also the mothership - the project that most other activities rely on or feed into. These include

  • a new mechanism (and UI) for aggregating, preparing and delivering collections content
  • the ongoing effort to improve our collections metadata (not our work, of course)
  • a wide-ranging e-commerce programme (which runs for well over a year after we relaunch the sites)
  • a realignment of IWM's identity
  • a refresh of multimedia throughout our branches

  • overhauling our web hosting

  • and finally, sorting out our development environment and re-thinking how we work (not a project but a big effort)

Some other departments have had to make considerable efforts so that the road-maps for the projects they own align with our own, and thanks to those efforts I think we're going to be able to deliver most of what we hoped to and maybe some extras when we launch, but of course that will just be the start of a rolling programme of improvement. We will start, for instance, with temporary solutions to some of our e-commerce provision. It's an intricate dance where we try to avoid doing things twice or, still worse, do things badly, but we also try to avoid being so dependent upon doing something perfectly (or that's out of our control) that we risk failing to deliver anything. For this reason we're limiting our ambitions concerning e-commerce to improving integration at a superficial level but postponing deeper integration such as unified shopping baskets and the like.

For me there are quite a few learning curves involved in all of this. I've got thin experience with the LAMP stack (we're using most or all of this, at least in our development environment - live O/S TBC), and nil with Drupal (we're using D7 for the core sites). My experience of working as part of a team of developers is also limited and has taken place in purely Microsoft environments, whereas we're adopting agile techniques and using tools like Jira/Greenhopper and SVN (no more Visual Source Safe) to help in this. The infrastructure at IWM is quite different to my old haunt, too, and finding my way round that, finding out even what I needed to know and how to get it done has taken pretty much until now. We do now have a development environment we can work with, though, both internally and with third parties, and in due course I hope to understand it! Besides that, there's the ongoing activity of researching and testing new solutions and practices, which is always fascinating, sometimes frustrating, and generally time-consuming.

So the new website will be in Drupal 7, although we may continue to use WordPress for blogs (WP3 is another new one for me). We were treading a fine line by choosing to use version 7, a decision we made a few weeks before the betas and, finally, RC1 came out, but we're feeling vindicated. Because some modules still have quite a few wrinkles there are some blockages (Media Module anyone?), but on the whole we feel it was a risk worth taking - there are major benefits in this version, and D6 will be retired well before D7. So when bugs have come up, Monique and Toby (the two Drupal devs we're so lucky to have working on this) have been balancing their efforts in working around problems, contributing fixes, and putting things on hold until patches are available. They've undertaken a fair amount of custom work where essential and we have a lot of functionality in place ready for theming. Design and theme development are taking place out-of-house, though, so the next few weeks should be interesting!
[June note: and so it has proved]

Lots of our thinking is around integration, of course, both to do now and in readiness for future developments. There's a lot around discovery and whilst I've not been anywhere near as involved with Drupal development as I'd like, I've been doing some of the work on search. We'll be using separate Solr cores to index Drupal content, e-shop products, collections metadata, stock photography, and a variety of non-Drupal/legacy IWM sites (indexed with Nutch). This will support an imperfect, loosely-coupled sort of integration, one that we can improve bit by bit.

Our strategy for delivering collections-related content builds on the software that we commissioned from Knowledge Integration when I was at the Museum of London. That conceptual approach and architecture (which I'll save for another post) was one idea I was really eager to bring with me to IWM, but I have to admit I found my advocacy skills seriously lacking when it came to explaining the case for it. All the same, we got there in the end and I believe that there's a growing understanding of the benefits of the approach, both immediately and in the long term - and now that we're making real progress with implementation confidence is growing that we can do what we promised. Phase 1 is strictly limited in any case, so that we have what we need for the website launch and that's it; Phase 2 will have the fun stuff - the enrichment, contextualisation, and nice things for developers - and we'll start building other things on top of the mechanism too, because of course the website is but one front-end.
[June note: K-Int have packaged up the software into a product which they launched at the recent OpenCulture exhibition. Very well worth checking out.]

That's where I got to. Will add more soon. Suffice to say that things remain quite busy but that we do at least now have a newly-skinned e-shop which teething problems aside looks a damn sight better than before, thanks to the efforts of Christian, Wendy, Kieran and Garry "The Bouncing Ball" Taylor.