About Me

My photo
Web person at the Imperial War Museum, just completed PhD about digital sustainability in museums (the original motivation for this blog was as my research diary). Posting occasionally, and usually museum tech stuff but prone to stray. I welcome comments if you want to take anything further. These are my opinions and should not be attributed to my employer or anyone else (unless they thought of them too). Twitter: @jottevanger

Wednesday, July 04, 2007

On the back foot with SMTP nightmares

Lots of catching up to do. Though I didn't post much before holidays, with too much going on at work, there are quite a few things I need to note down for my own benefit if no-one elses.
Firstly, though, my current trials.The last week or so have been a battle with e-mail. Mail from the web server has failed to get through and we have lost key capacity to diagnose and rectify the problem. The trouble is that it is a function that bridges several areas of comptetence and when we lost Rich May, network/helpdesk manager and good friend, back in April we lost vital knowledge, not to mention simple capacity. One of the great things about our team's structure is that we work side-by-side, pitching in as appropriate, whether we're developers, managers or HDEs, and I know it gives us an advantage over larger organisations with more differentiation/compartmentalisation because it we can have rapid, informal communication and the flexibility that comes from being all the same department. On the down side, the loss of one central member, not to mention the pitifully slow process of replacing him, leaves us badly holed. By the time a replacement is in position it will be three, perhaps four months since he left, plus one for his notice period. Of course, no-one will have exactly the same patchwork of skills nor the case-specific knowledge of the person they're replacing, but this is a study in how not to manage knowledge - far from allowing for a cross-over between outgoing and incoming employees the organisation has ensured that we have a 3 month gap between them. Of course, as much as possible was handed over to our excellent HDEs and to the rest of us on the team, but being short-staffed Help Desk have been unable to exercise much of its knowledge as they fight fires elsewhere. Projects have slipped and broken stuff gone unfixed - unavoidably, given the policies that left us underpowered for so long.
This brings me back to my e-mail issue. It turns out that e-mail from the web server does get through to external addresses (in fact, some, esepcially spam, gets through to our own mailboxes), and its looking like a spam filtering or probably DNS/SPF problem. I dabbled with these possibilities early on in diagnosing the problem but there were, as always, a number of overlapping or coinciding problems and red herrings and I spent a lot of time following these up. This is a very good example for me of how loss of capacity or knowledge can incapacitate our services or cost us dearly in time.