Some Stack Overflow questions

The Mamykina paper uses a nice mixed methods approach to look at both the interaction patterns on the site and the motivation of users and community owners. Some key design aspects highlighted are that the community managers are themselves respected domain experts, and that they have taken an evolutionary, collaborative approach to the development of the platform. In terms of site activity, the authors note that a high proportion of questions receive at least one answer (92%) and that answers are received and accepted quickly (median 11 minutes to be received, 21 minutes for an answer to be accepted). Whereas some users only ask (23%) and others only answer(20%), a good number do both (21%). They go on to recognise distinct user profiles in the community: activists, shooting stars, low-profile users and lurkers/visitors. Activists and shooting stars – active for a short period – together provide a large proportion of the answers despite representing a low proportion of the community (figure).

Answer behaviours (data from manykina et al)

Mamykina et al also note how the gaming aspects to the site – reputation scores and badges – clearly add to the site’s stickiness and appeal, though are not without potential problems, including the tendency for answers to be provided quickly in a rush for the accompanying reputation points.

Truede et als’ study is qualitative, looking at the types of questions that are asked and the tags applied to them. They propose a coding of programming language; framework; environment; domain and non-functional for tag categories and question types including how-tos; discrepancies; environments; errors; decision and conceptual. They found code review questions to be the most frequently answered satisfactorily (ie with an “Accepted” answer in 92% of cases studied), proposing that this is the easiest category to answer being the most concrete and self-contained, usually containing a code snippet for answerers to critique and correct.

Kumar et al looked at the Stack Overflow platform as an example of a “two-sided market”, where there are two distinct user groups – questioners and answerers – and a network effect between them that helps to determine its attractiveness to new adopters. Using a data extract, they derived “attachment curves” for the platform, defined as the probability that a user of one type or another will join the platform. They found a strong “cross-side” network effect, where the rate of questioners joining the network was strongly influenced by the presence of answerers, but that this effect was asymmetric and answerers grew more slowly in the presence of many questioners.

These papers together provide some good insights into how this online community has grown and been a great success – there is a symbiotic and sustainable balance between questioners and providers, groups both being served by the interactions, to the huge benefit of the third group, the search engine visitor. We also see how some question types and categories work more successfully, and how rewards might encourage speed and quantity but may also compromise quality.

REFERENCES

KUMAR, R., LIFTSHITS, Y. and TOMKINS, A., 2010. Evolution of Two-Sided Markets, Third ACM International Conference on Web Search and Data Mining, February 3-6 2010
http://research.yahoo.com/pub/3015

MAMYKINA, L., MANOIM, B., MITTAL, M., HRIPCSAK, G. and HARTMANN, B., 2011. Design lessons from the fastest q&a site in the west, Proceedings of the 2011 annual conference on Human factors in computing systems, 2011, ACM pp2857-2866
http://bid.berkeley.edu/files/papers/mamykina-stackoverflow-chi2011.pdf

TREUDE, C., BARZILAY, O. and STOREY, M.-., 2011. How do programmers ask and answer questions on the web?: NIER track, Software Engineering (ICSE), 2011 33rd International Conference on, 2011, pp804-807

PFEIL, U., SVANGSTU, K., ANG, C.S. and ZAPHIRIS, P., 2011. Social Roles in an Online Support Community for Older People. International Journal of Human-Computer Interaction, 27(4), pp. 323-347

WELSER, H.T., GLEAVE, E., FISCHER, D. and SMITH, M., 2007. Visualizing the signatures of social roles in online discussion groups. Journal of Social Structure, 8

This set of studies looked at data from online support and discussion communities with the aim of understanding network structures and the parts played by types of community members. Cobb et al used network analysis alone in their study of a community of smokers looking to give up (QuitNet). Pfeil et al attempted to integrate content and structural analysis in their analysis of the SeniorNet community for older people to discuss health issues. Welser et al sought to correlate structural characteristics with predominant interaction behaviours in three sets of Usenet data (two technical discussion groups and one on kites!).

Image: Flickr/barabeke

Cobb et al. start out by comparing weakly connected with strongly/densely connected members – on the basis of blog post and private message connections. They find more of the strong or densely connected core are women, and that they are older than the weakly connected core. They are also less likely to have lapsed in their attempts to give up. The study goes on to identify three significant groups: Key Players (50 users of the 800,000 registered and of the 7500 or so active in the study period), Newcomers (800) and Integrators (750). Key players were 80% female, with an average age of 49 and a high level of activity in terms of posts, connections and reciprocal messaging. Newcomers were all smokers (avg age 39) who joined the site during the study. Integrators were those who formed ties with the newcomers by initiating contact. The authors note a healthy heterogeneity to the network, with abstainers mixed with smokers – thereby providing the normative influence needed for quitters to keep it up.

Pfeil et al used a coding scheme to tag forum posts in a much smaller community (only 30 or so active members analysed) according to: self-disclosure, community, factual, slightly-off and support oriented content. The tagging scheme yielded seven types of user based on the predominance of these post types. They also undertook a regular equivalence analysis (REGE algorithm) of the network, which yielded six groups showing distinct interaction patterns. In their discussion they attempt to integrate the two groupings, though note that:

“At first glance, it appears that there is no overt connection between the results of the structural approach (social network analysis) and the behavioral approach (content analysis). The groups identified by the two approaches contain different members. However, at closer look, some relationships can be found.”

The groups they arrive at are:-

• Moderating supporter: Givers and receivers of support, active in welcoming new members and posting frequently at length;
• Central supporter: Frequent posters proving support and strongly connected, but less active in welcoming new members;
• Active member: Regular posters mostly chatting about themselves;
• Passive member: Less central. Did not provide support but did show self-disclosure
• Technical expert: Short messages on technical issues.
• Visitor: Remote positions, limited engagement.

Pfeil et als conclusions – that moderating supporters should be recognised and cultivated – are not especially surprising or new, though they do make an interesting observation about building trust in order to encourage self-disclosure.

Welser et al’s study focussed on the answer person “a participant in an online discussion group whose primary mode of interaction is the provision of helpful, informative responses to other group members’ questions.” Answer people are important for contributing without expectation of recompense and have high community value. For this reason the authors wanted to see what structural aspects of communities correlated well with answer people. They found a characteristic pattern of a sparse, star shaped egocentric network with connections to isolates, few of whom were connected to each other (see their example). Their analysis found that “authorlines” (thread contribution patterns), neighbour’s degree distribution and local network characteristics were good predictors of answer people.

The studies are methodologically interesting, with shortcomings of a purely structural approach noted by Cobb et al – for example, it is impossible to discern the influence of the community on more passive members. And what are the motivations of the key players? Pfeil et al tried to triangulate but found it difficult, suggesting that one-dimensional categorisation of users may be an over simplification.

.

Alvin Goldman’s 1999 book Knowledge in a Social World¹ provides a framework for the evaluation of social knowledge practices². My interpretation of his model toward the QA context is that it comprises:

1. Allowance for personal interest in the topic : “sensitivity to relative amounts of interest should play a modest role in assessing .. epistemic credentials”;
2. Need to know: Not everybody in a community needs to know everything, instead knowledge should be available to those who need it most;
3. The mechanics of the knowledge exchange practice itself, which involves: the inference practice of the “credal agent” (user in this case); the speech practices of the speaker (answerer) and the communication-control practice (the interface and its affordances + community moderation)
4. Accomodation of different types of question. For questions where no agreed “true” answer exists, it might be treated in a Popperian manner – you cannot know what is true but you can reject specific hypotheses. For this to work, however, a full range of hypotheses need to be represented.

How does Quora look against this framework and other related thinking in credibility and the epistemology of testimony? In terms of interest and need-to-know, the site limits your view of the community to just those parts you have shown an interest in. While this may lead to a confusing experience for those coming new to the homepage, it is perhaps a deliberate social design decision to create specialist areas and multiple entry points, as venkat argues in this post.

As far as practices are concerned, some of this will be down to norms that evolve and change with the expansion of the community of users. User inference practices need to be further studied, though it is likely that personal epistemologies – a personally held understanding of what knowledge is – and a desire for quick answers or “epistemic closure” will continue to be significant, though competing factors. That is, the need for closure may hinder the development of a richer understanding of knowledge as contested and nuanced. In providing responses, the motivation of answerers may be altruistic, competitive or toward self-promotion, though the relative effect of these are perhaps less significant when the community as a whole can vote answers up or down. And this collaborative filtering, while flawed, is at the heart of the consensus model that the site hopes to implement. Notably, Quora does not use the “Accepted answer” feature that other sites use which places emphasis on a particular answer.

Other interface features seem to be geared toward enhancing the user’s ability to evaluate answers. In terms of your qualification to answer on a topic, Quora allows you to provide your experience per topic rather than overall for your profile. This could provide important – though of course manipulable – cues to credibility. Evidence has shown that the closer testimony is to first-hand experience, the more important it is to us.

Criticism of Wikipedia in terms of credible testimony³ has been that the masking of author identity makes it very difficult to assess. As this question reveals, Quora insists on you providing your real name, although you may also opt to answer particular questions anonymously.

It has been noted that the discussion pages in Wikipedia provide the real story behind the knowledge presented on the page itself, and that the user should use these to help understand the debate and range (or lack) of views presented. By having both edited summary and discussion together in the page, Quora potentially provides more transparency in this. The edited summary should serve as a useful overview when woven together from a number of answers – providing the opportunity for “braided learning”⁴.

A further spur to credibility which Quora is trying to encourage is for users to expose their reasoning and argument rather than simply providing pithy answers, thereby avoiding a “black box effect” which leaves important gaps in the explanation and requires a leap of faith on the part of the information consumer⁵.

None of this is to underplay quality issues and potential for abuse and manipulation, which exists as much in Quora as other QA systems. The question is whether Quora can develop a large and loyal enough community to enable the ongoing accretion, filtering and refinement of good quality knowledge. It’s potential positive impact is significant – as a species we are predisposed to trust others on topics that we are not well equipped or primed to understand. Friendly, tailored advice caters to this. But this credibility is not without risk: while working on this I saw Gavin and Stacey, the one where Pete asked a social QA site “Should Pete divorce Dawn?” – the answer persuades them to give their marriage another shot. The complete lack of context and detail given in the question and their blind faith in the answer was the joke, but it also makes a point about how we may rate a stranger’s advice to a life-changing extent!

Footnotes
1. Goldman, A.I. 1999, Knowledge in a Social World, Oxford.
2. More recently, Goldman has recognised several shades of social epistemology, and suggests that the internet should be approached with a systems-oriented social epistemology, where the individual components should be understood but their interactions and the emergent effect of their aggregation are also significant.
3. Magnus, P.D. 2009, “On Trusting WIKIPEDIA”, Episteme, vol. 6, no. 1, pp. 74-90.
4. Preston, C.J. 2008, “Braided learning: an emerging process observed in e-communities of practice”, International Journal of Web Based Communities, vol. 4, pp. 220-243(24).
5. Rieh, S.Y. & Danielson, D.R. 2007, “Credibility: A multidisciplinary framework”, Annual Review of Information Science and Technology, vol. 41, no. 1, pp. 307-364.

Images:
Hands Flickr/coldtaxi

In this paper, Mercier argues that an evolutionary account of reasoning that derives from social argumentation is more compelling than the classic view which frames it as a crown jewel of individual cognition. He sees reasoning as a necessary outcome of human communication, where the responder employs sophisticated coherence checking of newly received facts (tending towards conserving existing knowledge), and the sender employs techniques to select the most convincing premises to persuade the receiver (while striving to avoid an irreversible loss of trust).

Mercier draws support for this theory from the observations that:

1. Confirmation and disconfirmation bias are a common and robust experimental finding, with people finding support for their side of an issue much more readily than the other side;
2. Individual reasoning doesn’t necessarily lead to better decisions, and here Mercier cites evidence for poorer decisions when people are asked to consciously and explicitly consider their reasons first.
3. Individual reasoning tends toward options that are easier to justify, and this may lead people away from normative answers toward those that would be easier to defend if called upon to do so.

Individual reasoning is therefore “a tool of (anticipated) persuasion”. Groups can perform better, on the other hand, perhaps due to a fairer balance between evaluation and persuasion. Here Mercier cites the “truth wins” and “assembly bonus effects”, where groups converge on correct answers, even where no individual members have these answers to begin with. In ideal social argumentation, the net effect can approach unbiased reasoning, because the type of biases named above provide the means to divide labour (each person only needs to find arguments for their own position and these are judged by the rest of the group on their merits).

Mercier does not always explain the evidence that he draws on very clearly, and it is tempting to wonder if he is exhibiting confirmation bias himself. Clearly neither the shortcomings of individual reasoning nor the merits of group decision making should be over-generalised. That said, there seems intuitively (i.e before engaging reason!?) to be much value in his approach in seeking a social, relationist origin to this cognitive capability. These ideas seem to fit well with that of enactive cognition – that individual cognitive structures are a consequence of a structurally coupled social system – and social epistemology, that social evidence should be considered a valid and primary source of knowledge.

Tollefsen, D.P. (2009) WIKIPEDIA and the Epistemology of Testimony. Episteme 6 (1), 8-24.
Wray, K.B. (2009) The Epistemic Cultures of Science and WIKIPEDIA: A Comparison. Episteme 6 (1), 38-51.
Sanger, L.M. (2009) The Fate of Expertise after WIKIPEDIA. Episteme 6 (1), 52-73.
Magnus, P.D. (2009) On Trusting WIKIPEDIA. Episteme 6 (1), 74-90.

In this set of articles, philosophers considered the implications of Wikipedia on theories of testimony, taking various positions on its role and significance as a knowledge source. A common theme is what our use of a collaborative but mostly anonymous system tells us about human credulity. A further thread is how many of the cues we normally use to assess credibility are not available in Wikipedia. Although there still ways to determine how accurate its articles are, we don’t actually tend to use them in practice. In taking the philosophical tack, the authors join up our current knowledge seeking habits with traditions within philosophy, but also remind us of the huge power and reach of Wikipedia.

In the first article, Tollefsen claims that Wikipedia can count as testimony on the assurance principle – that the authors are providing some assurance about the correctness of the content. Moreover, it has many qualities of group testimony as distinct from the aggregation of individual testimonies. She notes that the more well established articles represent a balanced view of the topic (as we’ll see this is perhaps overidealistic and is not entirely corroborated by the other authors.), seen in a way as the settling of the group mind on an issue. As we don’t know the individuals or groups involved in authorship, we are also to some extent placing our trust in a system.

Tollefsen largely supports an ‘antireductionist’ approach to justification, where you you are justified in believing others as long as you have no reason to believe them insincere. In the case of Wikipedia, however, she concludes that default entitlement to trust does not hold, and that we need to treat Wikipedia more cautiously, like “talking to a child”. Later on, she notes that we monitor testimony by checking it against our background beliefs, and this is how we may choose to accept the content of articles.

Wray takes the approach of comparing the process behind wiki articles to the practices of scientists, and comes out more in favour of science as a sound knowledge making practice. He observes how hard it is to authenticate online claims, quoting Shapin that “trust is ubiquitous in knowledge societies”. He further notes that the reliability of wikipedia is based on ‘invisible hand justification’ – in that there is no one person in charge of quality assurance on an article.

Wray sees wikipedians as having less to lose than collaborating scientists and therefore less incentive to be honest and conscientious. That said, he does go on to note that the rush to publish in science is a knowledge anti-pattern and may lead to as many accuracy issues as the rush to post online, which he casts as a form of gossip.

Wray compares egoism and non-egoism as justification in testimony. An egoist approach would say that you need to know the character of the person testifying, and a non-egoist would say it is just enoughto be told someone else believes something. But he concludes that neither extreme really holds in Wikipedia – we dont know the reputation of the authors nor if they really believe what they tell us.

Sanger, a philosopher as well as one of the originators of Wikipedia discusses whether experts still needed now that non-experts can so effectively aggregate knowledge. He concludes, however, that the role of the expert has been too far downplayed in Wikipedia, as experts may be pushed out or ignored.

In conflict with Tollefsen’s views about the “balanced” article emerging over time, Sanger claims that article quality may be a function of the persistance and aggression of the article’s followers, not just the lifetime of the article. In content wars, experts may be driven off by aggressive amateurs with more time on their hands.

Magnus asks what it means to trust Wikipedia as a knowledge source. He notes that in the famous (though controversial) Nature study, Wikipedia proved more variable and slightly less accurate than Brittanica, though perhaps in Wikipedia omissions are more common than inaccuracy.

Magnus provides a useful list of cues that we may use in assessing written testimony including: authority, plausibility of style and content, calibration and sampling with usual sources. He notes that these come undone on Wikipedia. Authority is hard to determine – and not guaranteed with referencing. Style can be edited out. There is a real risk of plausible falsehoods in the content. Calibration may only occur on widely known, as opposed to more specialist portions.

Magnus offers some solutions: Follow sources, interpret the palimpsest – the evolution of the article – from the discuss pages, and link to the dated version of the page. All things that are possible, but even academics probably don’t do.

While the more cautious views on how we “should” treat Wikipedia presented in this journal are coming from experienced academics, I am fairly sure that the position of the average user – including students and teachers – is much more toward anti-egoism and anti-reductionism. Thomas Reid’s principle of credulity – that we are predisposed to trust our fellow man – certainly holds in the large part, especially as often there is not much at stake if we are taking on false beliefs. As the need for correctness increases, however, then so does the need to spend time (and cognitive resources) in assessing credibility.

One area that I felt was not well covered in these papers was the tendency for online information to be recycled. The verifiability principle mentioned by Sanger works well, though often means that large sections are lifted directly from pre-existing sources on the topic, many of which are available online, many of which are written by “experts”. I expect that there is research available on the extent to which Wikipedia articles are largely paraphrased prior texts, and will report it here if I come across it.

9 Jun 2010, UCL

A set of interesting talks from researchers, museum staff and a technology supplier on digital heritage. Common themes were enhancing interaction and engagement with collections through digital media and the crowdsourcing of transcription work with its attendant risks and benefits. Crowdsourcing is seen having particular potential at a time when it is hard to secure funding for digitisation and online projects.

1. David Arnold, University of Brighton

David Arnold presented on advances in 3D object representations for heritage collections. Not really my area this, so I only picked up a few points:

• 3D Coform Project – part of big EU project with museums.
• Phased adoption of technology – first phase just reproduces what you do manually. Subsequent phases only possible with tech.
• Every time we digitise, we create potential legacy problems with the data

Tools:

• triangulated laser scanning, KULeuvan ARC 3D tool
• Sourceforce FOSS mesh processing

Example: publicscupturesofsussex.co.uk – map + 3d pics

Metadata: 3D COFORM adopted CIDOC-CRM & extension to “Digital Provenance”

2. Andy Hudson-Smith, UCL (cf Digitalurban blog)

Andy spoke about a new project aimed to attach memories to objects: Tales of Things and electronic Memory (TOTeM). Site is: talesofthings.com

You take a photo of an object/building etc.. of personal significance to you, upload it to their website and get a QR code sticker. API under development.

The object is then tagged and you can add your memories to it via the web. Object also tweets on updates. Audioboo for the audio tagging.

Apps: talking Oxfam objects (what happened to my old trousers?), artist adds QR to art to record people’s reactions to his art.

Plan to make QR code out of mosaic tiles for Shoreditch church.

“Tagging is Connecting” paper M/C journal

Follow project at twitter.com/talesofthings

Alzheimers link – triggering memories in sufferers.

3. Melissa Terras, UCL

Melissa spoke about her “Transcribe Bentham” Project & Crowdsourcing the transcriptions.

Jeremy Bentham – large amount of papers & material on him at UCL.

Crowdsourcing roots. Often those outside institutions took lead in exploiting new tech. Old idea – e.g. metal detectors revolutionising archeology with uptake by enthusiasts.

Holley, R D-Lib Magazine. “Crowdsourcing: how and why should libraries do it?”

Crowdsourcing successes:

• Galaxy Zoo – 60 million galaxies classified
• Australian Newspapers Digitisation Program www.nla.gov.au/ndp. 9000+ members thousands of lines corrected
• V & A – crowdsourcing best crops of images from collections. 14,000 to date.
• Family search indexing of family trees www.familysearch.org

and many more..

Learning from crowdsourcing experience:

• Majority of work is done by 10% of users
• Personal interest & reward – use of cognitive surplus
• Pensioners, disabled, terminally ill particularly keen
• Builds up IT expertise, “addictive”, rewards = ranking

.

Rose Holley’s checklist (on dlib). Launching system in July.

www.ucl.ac.uk/transcribe-bentham

..system will be based on mediawiki

Issues :

• linking transcription, catalogue & image
• Undoing vandalism
• Quality control

#TranscriBentham

4. Fiona Romeo, National Maritime Museum

Fiona also presented a range of NMM digital projects including crowdsourding of transcripts also.

“Your Paintings” project with BBC. Public Catalogue Foundation.

NMM photos on Flickr. Get foreign language tags as well as semantics (content related) . Some additional annotations. Get questions, which helps guide information design. Indicommons project, make it easier to share content.

Naval-history.net Naval history community – connected with them and got them to work on a set of scans – CC sharealike licenced.

NMM have a new wing under development dedicated to digital interaction and a number of interesting ideas for social sharing and enhancement of objects in the collection.

5. Sascha Curzon, Gallery Systems

Sascha presented the eMuseum federated search system, a nonprofit project which enables museums and galleries to expose their catalogues, enabling end users to search across collections. The aggregated data will also be available as XML and RDF.

I only discovered this book recently but it contains some interesting insights into group decision making, providing a range of evidence to support — often counterintuitive — observations about the best conditions for knowledge creation. As such, it relates well to Alvin Goldman’s “veritistic selection practices” — social design aimed at maximising the truth value of aggregated opinion.

Produced at a time when prediction markets, blogs and wikis were attracting growing interest, it is the book’s analysis of the more traditional social collaborative processes that stood out for me.

A central idea early in the book is the Condorcet Jury Theorem, which states that — on binary and numerical judgements at least — a group is more likely to be correct than an individual under the conditions that:-

1. Majority rule is used
2. Each individual is more likely than not to be correct

Notably, only some of the group need to know the correct answer, as the rest will answer randomly preserving the effect. This is redolent of Millionaire’s “Ask the Audience” option, which another book The Wisdom of Crowds claimed is correct 91% of the time, more often than the phone a friend option. Some warn against calling this mathematical effect “wisdom”, however.

Sunstein goes on to provide a range of evidence supporting the observation that deliberation does not lead necessarily lead to more accurate outcomes than polled, statistical samples. This may be due to:-

1. “hidden profiles“, or accurate information available to a group which is not accessed in deliberation (perhaps because only a few members of the group possess it, or are reluctant to go against the grain and challenge the majority view);
2. The “common knowledge effect“, where shared knowledge has more of an influence on the group’s decision than information held individually;
3. Informational and reputational cascades, where individuals change their view or the willingness to speak out after hearing the views of others;
4. Group polarisation, where members of a group end up with more extreme views — in line with their initial tendencies — after debate.

What this all mean for knowledge creation on the social web? Perhaps that giving scores and weighting to reputation, while establishing authority and expertise, may also dissuade or detract from informed dissent, and that such dissent may also warrant recognition.

In another review of the book, Ethan Zuckerman noted:-

“A large number of the most interesting projects taking place on the internet use strategies to aggregate information from multiple users to create new knowledge …. analyzing these systems in terms of their effectiveness in getting people to reveal hidden knowledge is, in my opinion, an excellent framework for evaluation.”

If theory and research are meant to inform practice, recommendations need to be current, relevant, and actionable. This can be hard to achieve in a domain that develops so quickly. Checkland (1998) observed a while ago how it is very hard for theory to keep up and be informative, thereby failing to match the “ideal” dynamic:

Checkland's ideal for theory informing practice

Whilst a range of research-theoretical stances can be taken, justified and accepted by the research community, only some seem to be close enough to practice to usefully inform it. This is where the researchers are either experimenting directly through design, or are involved enough to really engage with the language, problems and possible solutions in the domain. So I would favour interpretive or action research approaches in this regard:

Levels of engagement and theoretical stance (based on Aakhus & Jackson (2005))

Whilst the more descriptive, long view can still be valuable, it is hard to realise this value unless the knowledge gained is acted on by those implementing new designs and approaches. As they are the ones reading the research, this should perhaps be the research community itself in the first instance. Design improvement can then be further communicated through demonstrators, but these need to be compelling enough to be taken seriously.

References

1. Aakhus, M. and Jackson, S., 2005. Technology, Interaction and Design. In: Fitch, K.L. and Sanders, R.E.( eds) Handbook of Language and Social Interaction. LEA, pp. 411-433
2. Checkland, P .and Holwell ,S. 1998. Information, Systems and Information Systems: Making Sense of the Field John Wiley & Sons, Inc. New York, NY, USA

Originally reviewed for the BCS Information Retrieval Specialists Group Newsletter “The Informer“

This volume contains the proceedings of the 2nd International Conference on Metadata and Semantics Research, held in Corfu in October 2007. The conference had a 50% acceptance rate, with most of the presented papers being published in the book.

Despite the fact that many papers present only interim or early research work, the volume provides a fascinating overview of a cross section of metadata and semantics research in both pure form and as applied in a variety of domains, including those of cultural heritage, education and agriculture. In addition to the advances in these individual fields and domains, one is able to determine some general trends and patterns running through the work presented. Perhaps overriding all is the sense that many researchers and archivists are struggling with the problem of balancing the complexity of describing information resources within a domain adequately, while at the same time ensuring that discrete sets of knowledge can actually be successfully linked and that systems which can guide the user in retrieval and discovery can be usefully developed on top of these structures.

Immediately apparent as one reads the volume is the dominance of semantic web approaches to metadata and knowledge modelling, influencing migrations from stand-alone formats and conventions to those based on semantic web standards such as RDF and OWL. Sections are devoted to semantic web applications and to ontology engineering. In several places one sees how such approaches can provide the missing semantic aspect to rather more well-established syntactic rules and begin to enable reasoning and knowledge discovery through the use of ontologies. In Semantic Application Profiles: A Means to Enhance Knowledge Discovery in Domain Metadata Models, for instance, Koutsomitropoulos and colleagues show how the CIDOC Conceptual Reference Model for cultural heritage can be represented in OWL and used to discover related works (using a painting example). In Capturing MPEG-7 Semantics, Dasiopoulou et al, argue that a semantic model is needed to enhance and disambiguate multimedia mark-up enabled by the MPEG-7 format. Also very much worthy of mention in this regard is good the article by Voss, Encoding changing country codes for the Semantic Web with ISO 3166 and SKOS which demonstrates with a grounded example how changes and versioning can be accommodated using the RDF based Simple Knowledge Organisation System. This is clearly paramount when considering the provision of globally applicable yet cohesive and adaptive data sets.

The complexity of and unwieldiness of ontologies has been an issue in the past and several of the papers address this by providing either automated means of deriving ontology structures or reviewing tools used (or needed) for ontology and semantic web data manipulation.

In the first category, Ensemble Learning of Economic Taxonomy Relations from Modern Greek Corpora by Kermanidis shows how a number of clustering algorithms can be applied in combination to arrive at quite a high level of accuracy in deriving relations, based only on the learning sets themselves. In A new Formal Concept Analysis-based learning approach to Ontology Building, Jia and colleagues promote the FCA approach in providing a lattice of concepts rather than the hierarchy given by other popular clustering methods, which they argue as being more “true to life”. Jia et al then show how the derived ontology can then be used to provide a query refinement/expansion method.

In terms of user tools, Cardoso provides a useful state of the art in use of software to manipulate RDF data and ontologies in The Semantic Web : A mythical story or solid reality , and Enokksson et al address the problem of the remote and in-place modification of RDF in An RDF Modification Protocol based on the needs of Editing Tools.

While there are a number of papers that demonstrate derivations and developments of specialised or extended ontologies and metadata schemes, I found the more pragmatic papers to be more valuable and progressive for the way they manage to remain in touch with the ultimate aim of their work. A nice balance between the convenience of machine cataloguing and the need for oversight of human cataloguers is struck in Whitelaw and Collins’ Pragmatic support for taxonomy-based annotation of structured digital documents. The authors describe the derivation of metadata for Open University course documents (based on IEEE Learning Object Metadata) using a series of more- or less automated approaches and then having these evaluated by library cataloguing staff. The optimal solution was found to be where terms are suggested from the vocabulary based on document content, but where the staff member can click through to the definition before confirming the use of the term as metadata. The authors also begin to look at the cost benefit ratio of collecting more detailed metadata to a slight increase in cataloguing effort. While these authors don’t get as far as exploring the efficiency of retrieval of documents, this is handled in Cervera et als’ Quality Metrics in Learning Objects which stresses the reusability objective and notes that quality metrics are crucial to the navigation of resources. Notably, Cervera et al describe the need for socially based rankings and ratings to be incorporated into the metadata scheme.

A further example of attempts to enhance the usability and comparability of metadata is provided by Nilsson et al in Formalizing Dublin Core Application Profiles: Description Set Profiles and Graph Constraints. A number of practical measures for introducing conventions for how metadata elements are used and coded are presented, including standard templates and the use of a wiki to provide for easy human editing of profiles which can then also be exported at XML, thereby bridging the gap between qualitative and more structured usage guides.

Against this general background of attempts to standardise, enable interoperability between collections and provide enhanced semantic infrastructures you have the inherent messiness of human decisions as to meaning and the complexity of the encoding act itself, something that is probably not discussed enough in this community but which is covered in Scifleet et al’s The Human Art of Encoding: Markup as Documentary Practice. The authors note that metadata cataloguing is socially situated and may often be idiosyncratic, though the extent to which this may affect quality and comparability is poorly known. They report on a programme of research to study this variety in the practice of participants in the Text Encoding Initiative (TEI). Unfortunately their full findings are not covered in this volume but are well worth following up for those interested in this important question.

In summary, this is an interesting and varied read, with enough “current state of the art” review papers to interest the non-specialist as well as some good fodder for those already wrestling with metadata, semantics and knowledge representation issues within particular domains.

To see them in the context of the conference programme, I converted the tweets and the event schedule to XML suitable for display in a Simile Timeline. Click the screenshot below to access the resulting timeline mashup (nb tweets take a few seconds to load).

What would have made this exercise easier, and the conference tweets easier to track:

• If the conference programme had been published in a more mashable format eg iCal / XML
• If there was a way to identify a particular conference track (a sub hashtag? – or is that overkill?)
• A way to distinguish critical comments, reportage, and general flim-flam. Though perhaps that is part of the fun!