Seminar on ISSG Search Filter Resource

The ISSG Search Filter Resource website

Enterprise Search

I attended the Enterprise search meetup on Tuesday 4th October 2011, and here are some brief notes from the event. We had two talks and a 'fishbowl' discussion. I'll concentrate on the talks, and bring in any issues from the 'fishbowl' when it is necessary. The talks were from Iain Fletcher and Charlie Hull.

1. Iain Fletcher - Search Technologies. Data Quality, the Missing Ingredient for Enterprise Search

The essential problem introduced in the talk was the poor data leads to poor search and dissatisfaction on the part of the users in the service they get from the deployed system. Problems increase with time, search administrators need to constantly update their systems to cope with growth in data. Failure to tackle these problems leads to increasing users dissatisfaction - evidence shows that half the time or less, users do not get what they want. Relevance ranking has come along way over the years, however many ranking algorithms need to be tuned to the collection they retrieve on, optimizing on constants such as B and k1 in the BM25 matching function. I've done some of this type of work myself, and had some success at the Web Track @ TREC.

A possible solutions is auto-personalisation, however Iain suggested that this may not work well, and there is evidence for this in the academic literature.

Iain stated the search engines tend to rely on meta-data, and he gave the example of Google who rely on well written pages from which meta-data can be extracted. Thus classification using the meta-data extracted can narrow down the search for the user providing the user with some ability to improve their search result (as always with search, this must depend on the user's ASK). When writing web pages to be retrieved it is best to remove non-relevant text from page to increase the chances of the page being retrieved - this is the process of 'cleaning' the data. An example would be to have a bio of an author on every page of their website - this would impact on search negatively. A process of normalization can be used to ensure that relevant text is put together on the same page, increasing the chance of better search results.

Iain then talked about complexity management, which can be a real problem in search. He advocated the use of TQM (Total Quality Management) for search, using a black box method to find problems. Optimizing on one variable is problematic, as one does not know the effect of doing this on other variables - a holistic approach needs to be done if this is going to work in any sensible way. I myself used a brute force approach to optimizing tuning constants on the BM25 matching function, but you could think of using machine learning to do this - Microsoft have used Gradient Descent techniques for this kind of work (I can dig up the reference on request).

Iain concluded with a number of suggestions as follows:

• Data needs be thought about properly. Focus needs to be on the data, rather than the search engine.
• A formal model of data is required, and a data model design is needed. In the discusion later, it appears that in my circumstances no formal document is available to provide this information or the requirements for search. Transparency is a very important factor.
• A process to keep search working is essential, and adapt to changes in the data, as it grows with time. Otherwise the search will break!

2. Charlie Hull - Flax Search. Just the Job - Employing Solr for Recruitment Search

Charlie gave an interesting talk on the practical application of search technologies to a real world case study, in this case Reed Recruitment. Reed recruitment has significant data problems with 3 million job seekers in their database, and around 300 end users dispersed throughout 350 offices the UK. Their search before the new system was implemented on a transactional system using Oracle, the relational database system.

To say the oracle search was clunky would be something of an understatement. The user had 20/30 fields to choose from, and had to wait a significant length of time for the results as 100's of millions of database records were processed. Data was held on salaries etc as well as unstructured information such as CV's and job specifications. Oracle is fine for data, but very poor for unstructured data IMHO.

In order to create the new search, data had to be extracted from Oracle and transformed to a format which could be used by a search engine such as Solr. Based on XML two processes are defined:

• Indexer: extract and process the data from Oracle.
• Config: builds and verifies the data for the search engine.

Charlie describe the process using a diagram, which I don't have but was illuminating and helped understanding (I won't try and replicate it here, my drawing skills are rubbish!). Reed did the interface part of the projects, as they know their users well.

Overall I found this a very useful case study of applying open source software to real world problems. Later on in the discussion, there was an interesting interaction on using open source vs. propriety software. Largely this is due to policy according to Iain Fletcher, which invariably means Microsoft. I was reminded of the old adage "nobody ever go sacked for buying IBM", these days its "nobody ever go sacked for buying Microsoft"!

The search is now live and working well - Reed are satisfied with it. On interesting fact that emerged was that there is considerable resistance from users who have got used to using the old system. This is normal, and reminds me that the only reason Dialog is around is because information scientists using it demand access to a command line interface (power users who want to retain control of their world, and prevent disintermediation). These problems to not appear to occur with new members of staff, not yet initiated into the ways of the old system.

Search Solutions 2009

Report of the days events can be found at the Information Interaction blog.

Roger K Summit and Dialog

Rather long, but see here:

Roger founded Dialog

Memory and Retrieval

One particularly aspect of IR which I find particularly interesting, is the effect memory must have on the retrieval process. If we consider the standard ASK model, which assumes several iterations of searches, it is clear that memory drives the whole process, and the users ability to use their short term memory must have a substantial impact on how effect they can be or how long it takes them to search. This has in my view particular implications for dyslexics, who are prone to short term memory problems.

In the light of this the work of Oliver Sacks is particularly interesting. I read his book 'The man who mistook his wife for a hat' as part of the reading for my masters degree, and really got into his work. A particular case study related to this was 'the lost mariner' about a seaman who only had a half minute short term memory (STM) life span. I cannot imagine what it would be like to have your life restricted to half second spans. It would be extremely difficult for somebody to search effectively, even if they could recall what it was they were search for in the first place. However, this is not most extreme case by any means. In his latest book 'Musicophilia', Oliver Sacks discusses the case of Clive Wearing, who suffered a terrible brain injury and now has approximately 7 second memory spans. He has also a severe case of amnesia and has lost a lot of information from his long term memory. Clive is a man who I admire greatly, I have a wonderful CD of him conducting the Europa Singers performing Sir John Taveners "The Liturgy of St. John Chrysostom". I found some videos on youtube of a program on his condition:

and

This is truly heart-breaking. Has someone who workings on creating IR systems which people can use, I cannot see a way of helping people with such poor short term memories - I feel completely useless! One bright spot, his talent as a musician is undiminished - I will return to this issue in a future post on my thoughts of the implications of 'Musicophilla' on music IR and relevance.

Image manipulation

Look at this image. In particular see the evidence provided for the manipulation of the image. Images may say a thousand words, but what if they are the wrong ones!

What interests me here is the implication for IR, ergo image spamming. We're not there yet with regard to indexing by image content (the semantic gap), but how would you deal with such a problem? Sadly, I have no answers.

(h/t: Charles Arthur)

Search and sources of information

Last night the BBC broadcast the latest documentary in the Horizon series entitled Professor Regan's Supermarket Secrets.

One part of the documentary had the presenter, Prof. Regan, doing a search on Google for what was clearly a research topic (I can't remember the query, it was something like 'cancer colon'). We are presented with a decision on the back of the information gathered.

Its fine for people to perhaps start off using Google to pick up some initial information on a given subject (particularly if they are new to it), but in general you'll probably want to search different type of sources in order to come to some final decision. That is, for a research topic you'll want to engage with the academic literature. This means looking at sources beyond the web.

I'm surprised that Prof. Regan let herself be used the the Beeb in this way, given that she is declares herself as a scientist. Does she use Google and only Google to search for information in her own specialist area?

I rest my case.

[Prof. Regan is a Consultant Obstetrician St Mary's @ Paddington. I found it using Google.....]

Privacy Vs. Logging

Bit of a dilemma this. In a rather interesting article, Bin Tan and Zhai talk about the problems in providing both privacy and recording information to assit personalised search. Bit like having your cake and eating it!

There was a famous (infamous) case of a New York times reporter who identified somebody using their searches from a log AOL released without thinking about the privacy issues completely t e.g. properly anonymising data. Really, really silly!

Ask have taken a different view. They allow users to choose an option not to save their searches. This is fine, but how do you use information from the user to improve searches and personalise them? I'm not sure that Ask have fully throught through the implications.

References

Bin Tan, X.S. & Zhai, C. (2007). Privacy Protection in Personalised Search, SIGIR Forum, 41(1), November, 4-17.

Smart Resizing

This is a very interesting piece of research.

I'm particularly intrested in this for a number of reasons, including the display of images when conducting image retrieval. Another application is the display of map images when doing geographical information retrieval on mobile devices. It has in my view a number of important applications in IR.

(Hat Tip: David Thompson)

IR curricula in Library and Information Science Departments

I have a particular interest in this, and the following paper looks useful:

http://informationr.net/ir/12-4/colis/colise03.html

I have been involved in workshops on this issue in the past, and with colleages I'm planning another one next year. Proceedings from this years workshop can be found on:

http://www.bcs.org/server.php?show=ConWebDoc.8958