A metaphor

OK, this is for the scratchpad function of this blog, where I just want to stick down some ideas in case they can be turned from into something more interesting. This one's about the problem of splitting structure, style and content and still retaining meaning. It's the same deal wherever you look in computing, especially where there's communicative intent. Building a web page, constructing reusable learning objects, musing the semantic web, it's the same issue for everyone, and although we may try to embed semantic markup into the content there's stuff that can't really be captured like that.

It struck me that music, and especially musical performances, are like that. You lose meaning by disaggregating it; musical scores are examples of instructions for the reconstitution of a piece in the knowledge that each performer and performance will still be an interpretation of these instructions – the score is not the piece.

To split out the notes and the rhythm is many cases is relatively straightforward, although unless the former are sequenced it's no use. So, a sequence of notes, already arguably an aggregate, is the starting point, plus the timings of each and probably indication of instrument. So far so MIDI. In addition there may be notation of expression or technique (hand to use, groupings, slurs, stress and timing changes, grace notes, crescendos, descriptions of feeling). But whilst this may work for the score it will not suffice for a performance, where infinite indescribable gradations of expression cannot effectively be isolated from the rhythm, notes or tone. In other words, where content is readily captured in discrete units such as lexicon or numbers, it can be disaggregated, but not all aspects of an item's content can be captured in this way and not all elements may be split apart if they lose communicative intent by that process. And it follows that words within text that has been fully marked up with a thorough ontology are still not going to carry their full meaning as this comes not from the ontology but (at least) from the whole context.

 

Reading Heim, Chapter 6

He's discussing the need for humans to adapt to their tools and to interfaces (he distinguishes between the two, the former not adapting to the user except in the most primitive way). This has always been the case, though; as we work with new tools or systems they "govern our psychology" [p79] and do so all the more readily if the interface is good and the fit natural. It's rare that a complete and perfect solution is found to a problem - Oldowan tools helped but could be improved; iconographic writing systems developed into symbolic – and the always force a change in our behaviour. Whereas a "perfect" solution might arguably required nothing of us but fit with existing behaviour, or the behaviour we wish to indulge (at least by the logic of Heim's words). But in fact the way that we move forward is not simply by solving problems in such a way that the solution won't change our behaviour, but by finding that the answers create new questions or needs to be answered to improve our fit with the world – or at least show us the inadequacies of that solution and force us to look elsewhere.

We can see this by the physical changes in humans, too, adapting to fit behavioural solutions to problems: squatting facets indicating that behaviour rather than sitting; development of dominant hand/thumb in response to behaviour (Bimson et al.); knees and toes to kneeling to weave etc. So yes, interfaces require us to adapt, mentally and/or physically. It may be that with technological approaches we often have the option to design systems that could fit seamlessly and invisibly with our existing behaviour and thoughts, but usually this is explicitly avoided – design is intended to require explicit conscious choices from the user and this means that the controls must be sufficiently distinguished from those actions already being carried out for other reasons. It would be no food if a system read every movement of a hand through the air as a command, but touches to a screen of a mouse may be read differently.