22 October 2010
Today’s Tabbloid PERSONAL NEWS FOR b.showers@jisc.ac.uk
SUMMON 4 HN
XSL and CSS, meaning that it was much more customisable than previously. EnCLaVe allowed us to prioritise development work on the OPAC in order to test enrichment options that were previously difficult or impossible to implement.
New presentations and publications OCT 22, 2010 08:38A.M. We have been out and about talking about Summon in recent weeks You can find our latest presentations here including Internet Librarian International. Coming up in the next few months we’ll be at the UNICA seminar in Portugal and a Cataloguing and Indexing Group event in Edinburgh. Look out to for an article in SCONUL Focus too.
ENCLAVE
The rather dull ‘unenriched’ OPAC
EnCLaVE Final Blog Post
Intended outcomes
OCT 22, 2010 08:16A.M. EnCLaVE was divided into four workpackages. Workpackages 1 and 2 were about evaluating Voyager against AquaBrowser. UoE had implemented Aquabrowser mainly because it allowed us to deliver a more functional catalogue interface to our users. We felt we needed to test the ability of Voyager to replicate AquaBrowser functionality, as this could result in system savings, and reduced user confusion, by ceasing to take two systems to do, broadly speaking, the same thing. For NLS, who had concentrated more effort into developing AquaBrowser as a frontend for a range of services/content types, the issues were different, and more about focusing thought on understanding ways forward for the wider digital architecture, discovery experience and strategy.
Background EnCLaVE took place in a large research intensive institution (University of Edinburgh (UoE)), but very much within the context of a consortium of libraries (the SDLC) using the same LMS (Ex Libris Voyager). So the focus was not on any special nature of UoE users but on widely applicable benefits, or potential benefits, across SDLC sites and, therefore, of relevance to a wide community of Voyager users elsewhere. The project identified Voyager as the focal point, but looked at a range of ideas which could add value, either in terms of the richness of the experience, or by improving catalogue access and visibility. The project was an excellent opportunity to improve discovery functionality which had been neglected to some extent because of the research-intensive nature of the lead institution; priority, and project funding, had tended to focus on the research management and publication agenda through institutional repository development.
Workpackage 3 informed the project name, as it was about embedding the OPAC module within a different system, to raise its visibility and ease of use for an important community: WebCT Virtual Learning Environment (VLE) users. For technical reasons, work on this workpackage was delayed and we have been granted JISC approval for an extension to see this work through. WP3 will be reported on separately in due course (estimated as being late November 2010).
Our project partner, the National Library of Scotland (NLS), was an important contributor as joint lead institution for the SDLC, and also provided a useful perspective as a different kind of institution with a different audience and set of strategic drivers.
Workpackage 4 was very much about innovation. It aimed to build on recent work to use search data to generate wordclouds. To date, these wordclouds have been used for display purposes only, but to good effect, bringing the digital library into the new physical space that was created as part of the UoE Main Library Refurbishment Project.
The timing of the call was fortunate, as the latest version (7) of the Voyager system had recently been released, with an OPAC module running under Apache Tomcat (‘Tomcat WebVoyage’), built on XML,
1
Today’s Tabbloid PERSONAL NEWS FOR b.showers@jisc.ac.uk
22 October 2010
them (e.g. by defaulting to a particular choice of catalogue interface on a kiosk machine in a library building).
What we have done through EnCLaVE is develop our thinking on wordclouds to enable them to become search tools by associating links with the terms, and to think about other types of clouds, in addition to those created by user generated searches.
‘Choose your weapon’: users are required to pick an OPAC
The challenge
NLS were interested to know how the Voyager OPAC fitted into an environment where AquaBrowser was increasingly high profile as an entry point to the digital library.
The project sought to understand the differences between Voyager and AquaBrowser, both in terms of technical functionality and user behaviour:
In terms of access, we were keen to develop greater integration than is presently possible within WebCT.
• Do our users use AquaBrowser in preference to Voyager? Why/why not? • What are the key benefits of AquaBrowser? • What functionality is it now possible to replicate in Voyager? We conducted our own testing, and have also followed the work of the AquabrowserUX project, coincidentally also at UoE, which tested on our own users. We recognised that enrichment was only part of the challenge. A richer search tool is much more valuable if highly visible to the community, and available within an appropriate context, which is why we wanted to do more than we have been able to previously to embed the LMS into the VLE.
The rather uninspiring connectivity between Library and VLE at present
Finally, the wordcloud element of the project was trying to take advantage of what was at the outset almost a ‘throwaway’ idea to find something to make use of the new plasma screens in the refurbished main library concourse. This idea has gone down extremely well with both the user and professional community; could we tap into this and develop useful new tools on top of it? Established practice UoE wanted to move away from a situation where we effectively forced the user to make a decision (see image) or had to make that decision for
2
Today’s Tabbloid PERSONAL NEWS FOR b.showers@jisc.ac.uk
22 October 2010
Book jackets and filters: new Voyager OPAC interface (on test server) Content enrichment is available from various providers, and we looked at both Syndetic Solutions and Google Books. Syndetics offers us book covers and tables of contents; Google Books offers more content, sometimes including additional content such as book chapters, reviews, comments from the author. The implementation was fairly straightforward and the services were then evaluated by our Subject Librarians.
Mockup of Voyager embedded into WebCT Regarding our plans for word clouds, AquaBrowser already makes use of this functionality, so we wanted to look at user practice with this tool, and compare the approach it uses to ways in which Voyager generated clouds might be deployed.
We implemented in our test environment the Shelf Browse feature from the University of Wisconsin, Oshkosh using Voyager web services and compared it to the Google Books Shelf Browse. The users who reviewed the functionality really liked the way it presented in the catalogue as it recreated some experience of serendipitious browsing for users. The Google Books shelf browse is less valuable as it shows associated books, but not how they are co-located on our library shelves.
The LMS advantage Workpackage 1 found significant overlap between AquaBrowser and Voyager 7 OPAC functionality. It was recommended that we implement: • a simple search box and search refine options • clear layout of search results and full record display • spell checking functionality • a basic mobile version • add word cloud functionality • content enrichment and social networking tools • additional help
We need to do some further work on this before releasing to users, to customise the metadata elements displayed and also to seek to improve its reliability in returning results. It does provide some added value to having book covers associated with bibliographic records in the catalogue.
Full detail is available in the Workpackage 1 Report (PDF file). As a result, we have worked, in consultation with the SDLC LMS team, to upgrade Voyager sites to version 7, and have done initial testing of the additional functionality identified in WP1.
Wordcloud and VLE integration remains at prototyping stage, but we will undertake some user testing post-project. Key points for effective practice From our experience, we would highlight the following points: • Undertake user testing to test assumptions, reveal unknown issues
3
Today’s Tabbloid PERSONAL NEWS FOR b.showers@jisc.ac.uk
22 October 2010
and drive development according to user need
raise the profile of the library to internal audiences and external peer groups. However, more work is required to understand the value of word clouds as search tools. It is noted that some usability testing has demonstrated that the AquaBrowser word cloud may be helpful, but the AquabrowserUX study recorded no use of this function.
• Content enrichment: Comments on both services were broadly positive but there were some concerns raised that having two sources for this information was potentially confusing for users, and that the Google Books service often presented different book covers and editions to those in the Library Catalogue. We will investigate Google Books provision and quality a little more and have decided just to go live with Syndetics content for the moment.
Additional information Workpackage 1 Report
• Relating to embedding the OPAC into WebCT, we encountered some issues with configuring the voyager webservices for use by the project. This was further complicated by a security patch being applied for voyager just as we started working on the problem, which seems to have broken our access to the web services. We still intend to complete this work once the problem has been overcome, and this will be reported on as soon as we can, alongside any generally applicable advice for Voyager sites interested in doing similar work.
UoE commissioned Usability Testing Report (AquaBrowser slides) (PowerPoint file) AquabrowserUX commissioned Usability Test Report Voyager 7 Tomcat OPAC documentation <to follow> Wordcloud documentation <to follow>
• It is clear that word clouds generated simplistically from search terms lack value for searching. Further work is being done on this as blogged recently.
EnCLaVE presentation to IGeLU (PowerPoint file) EnCLaVE pecha kucha presentation to RLUK 2010 Conference (forthcoming, November 2010) Simon Bains Project Manager
COVENTRY UNIVERSITY PRIMO IMPLEMENTATION PROJECT
Final technical observations OCT 22, 2010 06:42A.M. Voyager OPAC with embedded wordcloud For those interested in a quick summary of the technical aspects of this project, the implementation notes which form part of the ‘one minute blog’ post may prove useful.
Conclusions and recommendations The key conclusions of the project were:
Here I thought I’d concentrate on a couple of technical areas that have eaten away at the time.
• Voyager 7 is a much more functional OPAC module, and could, if necessary, provide a satisfactory single route into the catalogue with the enrichment options until recently only available through the application of a separate third-party interface system like AquaBrowser.
Shibboleth implementation is one of the main culprits in this category. Although this formed part of the contract with our supplier, there were still significant obligations, both for library systems staff as well as IT Services staff. The university already had a Shibboleth idp service in place but on the library side we started from scratch and encountered a fairly steep learning curve. There’s a whole new vocabulary to learn when addressing issues that surround authentication - it’s wasn’t immediately obvious where to go for help. Fairly early on our PM, sporting a large grin, pronounced, “I don’t do Shibboleth!” I can see why. Added to this, the upgrade to the unversity’s idp from Shibboleth version 1.3 to version 2.0, right in the middle of our testing phase, set us back considerably.
• There are differences in how AquaBrowser and Voyager handle some functionality, and some AquaBrowser functionality that is not native to Voyager, but may be developed on top (such development work may already have been done by Voyager sites and made available to others to use – see the WP1 report for details). • Wordclouds generated from usage data have attracted much professional interest and, if nothing else, demonstrably serve to
Following on from the Shibboleth issue, we were also aware we needed to upgrade the OPAC authentication functionality which still made use of
4
Today’s Tabbloid PERSONAL NEWS FOR b.showers@jisc.ac.uk
22 October 2010
our 14 digit patron barcodes. In order to move to Shibboleth authentication, it would prove necessary to insert patron usernames into their Aleph records. Inserting this data into the relevant field (z308_rec_key) with the appropriate flag would prove time-consuming. Occasionally the unversity issues a new username to a user - dealing with updates of this type continues to create some problems.
database. 2 of these then went on to click on ‘Selected Databases’ in the Lightbox which automatically took them out of the Lightbox and back to the Article etc tab search screen with the selected database as a ‘quickset’. This was very confusing for the testers as there was no warning or indication of what had happened. To access ASSIA the testers needed to click on the title of the database and 4 did this straight away without ticking the checkbox, and 3 others clicked on the title after ticking the checkbox and having nothing happen. Recommendations: • Make the Find Databases link appear in both the Articles etc tab and the Books etc tab – Ex Libris are looking in to this. • Make the Find Databases link more visible – some testers commented that it was hidden away in the corner – we have aligned this to the lefthand side, immediately above the main search box and used a larger font. • Provide instructions for students on finding named databases on the front page of Primo. • Remove the checkboxes from the Lightbox and remove un-needed functionality (e.g. ‘Selected Databases’) – Ex Libris have removed the checkboxes and have taken out some of the functionality within the Lightbox. This needs more development work in the future.
Other time-consuming activities included establishing data export / import routines for the source systems. The library management system Aleph proved troublesome - plenty of room for improvement in the supplier’s documentation here. Having said that, the work our PM carried out on the university’s repository product, Equella, proved invaluable. Finally I’m sure my colleague who worked through many of our technical troubles (and is currently taking a well-earned holiday) would want me to mention the unavoidable overhead that is incurred when installing separate third party products which all go to form part of the complete installation. By definition any summary such as this will struggle to give a complete picture; I hope these observations have at least provided a useful snapshot.
Question 11 - ASSIA isn’t finding you the articles you want and you’re not sure what other databases you should use to find articles from the Business & Economics subject area. How would you find a list of databases recommended for this subject?
COVENTRY UNIVERSITY PRIMO IMPLEMENTATION PROJECT
Primo Usability Testing Details Post #3
In this question we were looking at whether the testers would be able to find a list of databases relevant to a specific subject area from the information available to them in the Lightbox. Many of the testers were already in the Lightbox when they attempted this question and found a list of databases relatively quickly although many clicked on several drop-down menus before realising the information they wanted was in Category – this indicates that more work needed to be done to ensure that this was more intuitive. Some testers suggested using ‘Subject’ instead of ‘Category’ to indicate more clearly. Some testers also seemed to expect the list at the bottom of the Lightbox to automatically switch to the chosen subject area and it took a few seconds for them to click on the Find Databases link instead. This expectation could be due to their experiences within Metalib where the list automatically opens. Recommendations: • Re-name ‘category’ as something more intuitive – this has been renamed as ‘subject’. • Insert text in to ‘Category’ box to indicate to students that they can choose a subject area i.e. ‘Select subject area’.
OCT 22, 2010 05:22A.M. This is the third of our 3 posts giving a detailed write-up of our usability testing. Section 3 – Finding Databases and Post-Test Discussion Question 10 - Your lecturer has told you to search the database ASSIA (Applied Social Sciences Index & Abstracts). How would you find it and access it. In this question we were attempting to find out the visibility of the Find Databases link (as it is only shown in the Articles etc. tab), and to see how testers used the functionality within the Lightbox to access a specific database. We were aware before the testing that an alternative method of finding a named database existed i.e. searching by name in the Books etc. tab and only 1 tester used this method. All 9 other testers used the Find Databases link – 3 testers needed prompting to either go in to the articles tab, or to look around the screen to see if they could see a link. 3 testers saw the Find Databases link straight away, and 3 others saw it after moving between the Books and Articles tabs. Some testers tried to use the options within the Advanced Search in the Articles tab before noticing the Find Databases link. In terms of actually accessing ASSIA, of the 9 testers who used the Lightbox, 5 ticked on the checkbox expecting that it would open up the
Post-Test Discussion / General Observations Many testers thought that once they were used to the new site they would find it easy to use, especially as it had everything in one place unlike the library catalogue and eLibrary. Some suggested that the new site was more professional than the other library search tools they had used. Facets – The majority of testers thought that the facets were best placed on the left-hand side of the screen – some testers mentioned websites
5
Today’s Tabbloid PERSONAL NEWS FOR b.showers@jisc.ac.uk
22 October 2010
such as Google and Tesco in comparison. All testers thought that the facets would be useful to them when searching library resources; however some commented that work could be done to make them more visible e.g. some testers didn’t notice them until they were pointed out. It was encouraging throughout the tests to see some testers using them on searches that didn’t explicitly require them to do so.
COVENTRY UNIVERSITY PRIMO IMPLEMENTATION PROJECT
Reviews & Tags – None of the testers knew what a Tag was. When we explained to them that they could add their own keywords to individual catalogue records and asked if they thought this was useful some of them did think it would be. However, most of them were thinking of using the Tags as a way for them to retrieve records at a later date. It would probably be easier for them to just save these records to their personal Primo space (e-Shelf or My Favourites as we renamed it). When a scenario of tagging records that could be useful for other students working on a joint project was suggested most testers thought that this was a good idea. However, the fact that most of them didn’t think of this or any other social-networking way of using tags seems to imply that this functionality may not be well used. We are planning to leave the Reviews & Tags function switched on and see how much use it gets.
This is the second of our 3 posts giving a detailed write-up of our usability testing.
Primo Usability Testing Details Post #2 OCT 21, 2010 01:10P.M.
Section 2 - Searching for known journals and journal articles Question 7 - Your lecturer has told you to read an article in the most recent issue of the journal, the Harvard Business Review. Check to see if this is available in print, electronically or in both formats. In this task we wanted to see how the testers would search for an individual journal. There are clear options for this in the library catalogue and eLibrary; however Primo does not explicitly offer an option for this at first glance. Only 2 of the 10 testers were able to correctly identify that we had both print and electronic access to the journal. The majority of testers saw ‘Online access’ and ‘Media format: EJournal’ and suggested that we only had electronic access to the journal. Only two testers clicked on the Locations tab which is where they would find details of the print access. One tester found the print access by clicking on the ‘More’ tab and noticing the ‘Check print availability’ option in the SFX menu – however they did not verbally confirm if there was electronic access. 4 of the testers tried to search for the journal in the ‘Articles etc’ tab as they assumed that this was where they would find journals. Some of these were prompted to look at the ‘etc’ part of the tab name – one tester stated that they thought this was very unprofessional terminology to use in a higher education environment. Two of the testers used the Advanced Search option in the Books etc tab and chose ‘Journal’ from the format. Other testers commented that they had previously used the Journal Search or Find e-journal options with the Library Catalogue and eLibrary. This indicates that the testers were looking for ‘somewhere’ to search for journals and wouldn’t necessarily use the Books etc tab for this purpose. Recommendations: • Re-name the ‘Books etc’ tab to include the word ‘journal’ and create a more professional-sounding name – this would indicate to users more clearly where they should search for different material types – this has been re-named ‘Books, Journals, Media & More’. • Find a way of making print journal access more visible to the user – possibly re-naming the Locations tab could help with this. • Check with Ex Libris to see if different text can be displayed in the ‘Availability Status’ line (would also help with e-books – where print availability is also hidden) – Ex Libris responded that a different text can be displayed – we are still looking in alternative wording. • Change the information in the Media Format section of the ‘Details’ tab to include both print and electronic access – if this is not possible would removing it altogether be an option? – we have removed the Media format option from this tab.
Other Actions based on the Usability Tests: • The ‘More’ tab was removed as it duplicated information found in other tabs and was felt to be irrelevant. • The main tabs were relabelled as ‘Books, Journals, Media & More’ and ‘Articles & More’ to differentiate the types of content in each tab. • Some tabs were re-ordered to give more prominence to actual availability e.g. the ‘Locations’ tab (re-named ‘Availability’) is now either the first or second in the list rather than appearing after the ‘Request’ tab. • The ‘New Search’ link needed to be made much clearer – a few testers commented that they couldn’t find a way to clear their search – we have left-justified this and made the font bigger. • The e-shelf was misleading for quite a few testers, as they thought this was an area where they could search for e-resources – we have now renamed this as My Favourites in keeping with the general trends found on other mainstream websites.
Question 7a - What years are available and from where?
6
Today’s Tabbloid PERSONAL NEWS FOR b.showers@jisc.ac.uk
22 October 2010
(dependent on Primo Central installation). Out of 9 testers who attempted the question, only 2 were able to correctly identify the years of access as from 1952 (print) and from 1922 (electronically). The majority of testers thought that we only had access to one year of the journal – 1922. This date was mostly identified in the results list, the details tab or in the sfx menu. 1 tester did not verbally conform any dates at all, despite scrolling through a list of journal issues in EBSCO. Unfortunately, due to the lack of recognition of print availability (as shown in question 7) it was not possible to effectively evaluate the information shown in the journals locations tab, although the testers who found it were confused by the amount of information on the screen and took some time to notice the Summary Holdings statement showing the years available. Recommendations: • If possible include the journal ‘from and to dates’ in the brief display and full details tabs (not possible due to cataloguing implications) • If the above is not possible, then remove the date entirely from these sections as it could be deemed to be very misleading – we have removed the date for journals. • Try to remove all of the information in the journals locations tab to make the Summary Holdings statement stand out more prominently (unfortunately this is unable to be done when using Opac via Primo (OvP) as this also removes the holdings information for books. Turning the information off, also removes the Summary Holdings statement).
Question 9 - Please try to view one of the articles you have found. This question was to see how testers would attempt to get to the full text of the article. Ex Libris had named the tab which linked to our SFX installation as ‘Services’ and we wanted to see if this was an intuitive place for the testers to click. The ‘Services’ tab was located in the same place that the ‘View online’ tab was from the Books etc search tab. Each article on the results list came up as ‘No Full Text’ even if we had access – this is related to the information coming from SFX and was incorrect due to not having implemented the most recent version of SFX. The ‘No Full Text’ information in the availability line misled many of the testers, and we were required to ask them to ignore it and try and access the articles anyway. 8 of the testers were able to find the ‘Services’ link, however many commented that this wasn’t very clear, or that they would not expect to gain access through that tab. Only 4 of the testers clicked on ‘Services’ without prompting, and 2 testers became frustrated and did not complete the question. 1 tester accessed an article through an unexpected route – the links box in the full details screen. This was something that hadn’t been considered when writing the questions for the test. In further testing by library staff, it seems that the links box does not appear for all articles and as such cannot be considered a reliable link to full-text access. Recommendations: • Rename the ‘Services’ tab to something more intuitive – one tester suggested ‘check for full-text’ or ‘availability’. Another tester suggested using ‘Find it @ CU’ – we have re-named ‘Services’ as Find it @ CU – our SFX branding. • Ensure that the availability line information is accurate – if this is not possible then turn it off completely to avoid misleading users – currently this is turned off and will be investigated again in the future. • Do further testing on the links boxes.
Question 8 - You need to find general journal articles on the subject of customer service training – how would you carry out this search? (You will need to sign in with your Coventry University username and password before starting the search) This question was included to see how easy students would find the Articles tab to locate and use. At the time of testing the Articles tab had not been set up quite in the way that was needed and so inevitably there was quite a lot of prompting to ensure testers came up with a list of results that would be useful for the follow-up question (#9). Only half of the testers found the Articles tab without prompting. Of the 5 remaining testers, 2 really struggled to find somewhere to search for articles, and 1 tester attempted to use the Journal and Article options within the Books tab advanced search. All testers found the sign-in link straight away which was very encouraging. All testers needed prompting to use the General Quickset, however some did look in this drop-down menu before prompting which indicates that if there had been subject-based quicksets there, it may have been more intuitive. 2 testers encountered a blank screen after having signed in after trying to conduct their search - behind this Primo was actually conducting the search, however there was no indication of this and caused some confusion. Recommendations: • Provide instructions on the front page of Primo to indicate what students would be able to find in the Articles tab. • For Ex Libris: ensure that a blank screen is not displayed when a search is conducted when sign in happens after search terms are entered – Ex Libris have been looking in to this, but it has yet to be fixed. • Insert text into Quickset box asking students to select a subject area – this has been done. • More usability testing once the Articles tab is set up with our quicksets
COVENTRY UNIVERSITY PRIMO IMPLEMENTATION PROJECT
Primo Usability Testing Details Post #1 OCT 21, 2010 01:04P.M. To try to keep this detailed write-up of our usability testing more manageable we’ve broken it down into 3 posts. This is the first one. Introduction / Background The initial implementation of Primo took place in mid-July 2010, and the usability testing took place within 2-3 weeks of this implementation after staff had attended a two-day Ex Libris training session. The aim of the testing was to see how intuitive the Primo interface was, and to gauge how useful students thought some of the additional functionality would be e.g. Facets, merging of records and FRBRisation, Reviews & Tags. In terms of recruiting for the testing we knew that it would be difficult to
7
Today’s Tabbloid PERSONAL NEWS FOR b.showers@jisc.ac.uk
22 October 2010
get students in the middle of the summer vacation, however we used the details that we had taken during the name survey earlier in the year. This did give us all of our student testers, and we recruited 2 members of academic staff by asking Subject Librarians to contact lecturers they knew were helpful. We conducted 10 tests in total, with 7 of the testers coming from the Business, Environment & Society faculty, and 3 coming from Engineering & Computing. We were unable to recruit students from the Health & Life Sciences faculty, possibly due to course structures and placements. All testers had used the library catalogue and/or the eLibrary (MetaLib) before, but had mixed library search skills.
Question 2 - Can you tell us what floor it is on and what number it’s at in the Library? In this question we wanted to see where the testers would find the location of the book from on the screen. 9 out of 10 testers were able to give verbal confirmation of the floor and number that the book was held at. The other tester provided confirmation of the floor, but not the classmark. The majority of testers found this information in the availability line in either the brief or full screen. 2 testers found this information in the locations tab. 2 testers needed prompting to give either the floor or the classmark, and the average time taken to find this information was around 19 seconds.
• 2 x academic staff • 5 x undergraduate students (two 2nd years and three 3rd years) • 3 x postgraduate students
Question 3 - Can you tell us how many copies are out on loan?
We conducted the usability tests in a usability lab within the university. The lab allowed us to record audio and video of the participants, along with a close-up screen capture which showed how the participants navigated around the screen. Within the analysis below, only the audio and screen-captures were used.
Only 3 of the testers managed to complete the task of finding out how many copies were on loan without prompting. 4 testers were unable to complete the task without prompting – many clicked on all tabs except the Locations tab. Some testers commented that they would not look there as it would only tell you where the book was held and not how many we had. Many testers thought that these details would be in the Details or More tabs. There were 3 testers who were unable to complete the task. One of these was due to not recognising the ‘expand’ buttons in the locations tab when there was more than one sublibrary. 3 testers thought that the ‘availability line’ status was clickable and that this was all the information there was available. Recommendations: • Re-label the locations tab to something that is more intuitive e.g. one tester suggested ‘availability’ – the ‘locations’ tab has been re-named as ‘Availability’. • Ensure that if there are multiple sub-libraries that they expand automatically so that all copies are displayed – Ex Libris responded that this is not possible at this time. • For Ex Libris: possibility of making the availability line status as clickable, taking users into the ‘locations’ tab?
To become familiar with the usability lab environment, and to test the questions that we had written, we asked a member of library staff with some experience of enquiry work to run through the questions for us as a pilot study. This provided valuable feedback on the wording of some questions which we were able to amend before beginning the testing for real. The tests themselves consisted of 12 scripted questions along with time for discussion at the end (including questioning about the location of facets and the participants’ understanding of Reviews & Tags). Each test took around 45 minutes to complete, including discussion. We tried to use scenario-based questions so that students would feel that the tasks they were being asked to do would represent how they would approach the library search interface in ‘real life’. Analysis and Recommendations
Question 4 - If all copies of this edition were out on loan, can you see if there are any other editions available to borrow and if so how many editions?
Section 1 - Searching for books and e-books Question 1 - You want to find out if the Library has the book “Operations management” by Nigel Slack from your reading list. Can you check if the Library holds this book?
9 out of 10 testers completed the task by acknowledging the ‘View x versions’ link and providing confirmation of the number of editions available. Of these, 4 testers had already clicked on the ‘View x versions’ link as part of the previous task and had already noted that it took them to a list of editions of the book. This indicated that we may need to change the labelling of the link to ensure that students would know that it meant ‘editions’ of the book and not ‘copies’. The one tester who did not complete the task was unsure of what an ‘edition’ meant and did not notice the link in either full or brief details screens. During test 7, the ‘View x versions’ link disappeared from the full details screen and so the tester was prompted to return to the brief results. Other testers had seen this link when the system was working correctly. Recommendations: • Re-name ‘View x versions’ as ‘View x editions’ – this has been done. • Ensure that the edition statement appears within the record – we have
All 10 testers were able to complete the task of recognising whether the library stocked a particular book, however 4 testers required prompting to ensure that they had seen that we held the specific book in question. Of these 4 testers, 2 picked the same titled book by a different author, 1 picked a differently titled book by the same author, and 1 became distracted by the ‘view 3 versions’ link of a different book. 5 of the 6 testers who completed the task without prompting were able to do so in less than 1 minute. It was noted that all testers recognised the search box immediately, and 1 tester commented about being in the Books etc tab already.
8
Today’s Tabbloid PERSONAL NEWS FOR b.showers@jisc.ac.uk
22 October 2010
added this to the ‘Details’ tab.
OSINW link opened in a Primo window (or preferably both open in a non-Primo window). • Possibly attempt to re-name OSINW text to something clearer
Question 5 - You are at home and need to find some books and don’t have time to come into the Library. Can you find a list of e-books on the subject of human resource management?
KEVEN 5 of the 10 testers were able to find a list of e-books using the e-book facet on the results screen, 4 testers were able to complete the task with prompting and one tester did not complete the task, but then found the answer whilst completing task 6. Several testers used the ‘Show Online Resources’ link but did not notice that this included journals and ebooks. Several testers tried to find an ‘e-books’ option before attempting their search, 3 testers used the advanced search and looked in the format option expecting there to be an ‘e-book’ choice. 3 testers thought that ebooks would be in the e-shelf area and were confused by this. 7 of the testers saw the e-book facet – 4 without prompting. Recommendations: • Include brief instructions to users on the front page telling them that they can refine their search after retrieving a list of results. • Re-name ‘e-shelf’ as this caused confusion as testers believed this was where they would find e-resources – this has been re-named as My Favourites. • Explore further ways of distinguishing between journals and books in the ‘Show Online Resources’ results list
Goggling at KEVEN on Google OCT 21, 2010 12:10P.M. I don’t know why this hasn’t occured to me before, but as we are nearly at the end of the JISC funded stage of this project, I’ve just bashed vufind keven into Google to see if it has had any impact on the wider community. I was very humbled to see that folks within this JISC LMS project strand have been taking an interest in what we have been doing. We’ve obviously had some comments via these blog posts, and further interest or support has been expressed in the form of many tweets. Details of the project have also been picked up more widely in some cases. CILIP’s Multimedia Information TechnologyGroup used our project as an example of how the VuFind interface can be used: http://mmitblog.wordpress.com/2010/07/16/vufind-library-resourceportal-out-of-beta/
Question 6 - Please try to access one of these items Elsewhere, there has been interest in our plan to add QR codes to catalogue records: http://vufind.org/jira/browse/VUFIND308?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aalltabpanel
In this task we wanted to see whether users would be able to see how to open an e-book from the results list, and whether they would be happy with viewing an e-book in the ‘letterbox’ preview screen that is automatically opened when a user clicks on ‘View Online’. There were two potential ways of opening the e-book into a bigger screen – an icon on the right-hand side that would enlarge the e-book into a bigger Primo window (the same view as would happen if a user was in the full-details screen rather than the brief results list view). The second option was a link on the left-hand side that said ‘Open Source in a New Window’ (OSINW) which would open the e-book in a full non-Primo window. Firstly, all of the testers noticed the ‘View Online’ link when either in the brief or full details screens and understood this to mean opening the ebook. Almost all of the testers were unhappy about attempting to view the e-book in the small letterbox view, and many were also reluctant to view the e-book in the bigger Primo window. 5 testers from the brief view clicked on the icon to open the e-book in a bigger screen. Only one tester saw the OSINW link from the brief details screen. Only 4 testers in total saw the OSINW link in either the brief or full details screens. 4 testers attempted to use the internal options within the e-book itself to try and make the e-book bigger – several testers became quite frustrated at not being able to open the e-book in a non-Primo window. Recommendations: • Configure Primo to ensure that when clicking on ‘View Online’ the ebook would automatically open in a full-screen, non-Primo window (preferred option) – this is being looked in to by Ex Libris and a fix is due to be released in an upcoming Service Pack. • If above not possible – swap the links from the OSINW and icon options so that the icon would open a non-Primo window and the
Eoghan may be please to know that we have implemented this in KEVEN after all. Report post
SWWHEP LIBRARIES SHARED RESOURCE DISCOVERY
The best laid plans of mice and men .... OCT 21, 2010 11:27A.M. The original timetable for this project looked like this: March 2010 – Initial “go live” of public beta service June 2010 – completion of feedback-gathering exercise from beta testing August 2010 – launch of full, live service Septemebr 2010 – project review October 2010 – Options for sustainability of the service beyond the
9
Today’s Tabbloid PERSONAL NEWS FOR b.showers@jisc.ac.uk
22 October 2010
SWWHEP project funding period and handover plan It didn’t quite work out that way! There was an embryonic version of the beta service available by the end of March, but there was still too much work to be done on it before it could be properly publicized or used. The reasons for this were various, but included dependencies on outside sources for the development of addition Jangle features, and problems getting permission for access to one of the institution’s LMS’s in order to enable and develop its VuFind connector. As a result, the service wasn’t fully in beta until the summer, when feedback (mainly from staff) started rolling in. “Live beta”, launching the service as a main catalogue, occurred in September, with students being training on its use and providing feedback. We continue to act on feedback, and will go on rolling out enhancements on a monthly basis for the time being.
AquaBrowser word cloud After the recent JISC LMS meeting in Glasgow, and some internal discussion around the first phase of wordcloud development, some issues were identified for examination:
ENCLAVE
Is the future of search in the cloud?
1.
Single word vs phrase searching
Phase 1 of development had stripped all association from the search terms, including breaking up phrases into single words. This was initially done to allow the frequency of search words to be counted so that we could then represent the most searched for words more prominently in the resultant word cloud. This was okay for the original purpose, which was a static display in the Main Library entrance hall.
OCT 21, 2010 09:47A.M. EnCLaVE has directed one workpackage at the notion that word clouds are a potentially useful search tool for the LMS. AquaBrowser already offers a word cloud, although its graphical representation, is quite different from the increasingly popular ‘Wordle’ approach, which we have adopted at Edinburgh as a way of depicting search activity in Voyager.
Now we have integrated the word cloud tool with the LMS, and made the search terms clickable by users to allow searches to be repeated, it is clear that the entries in the word cloud are not hugely useful for this purpose. Most frequently counted words tend to be words like ‘English’, ‘Scotland’, ‘History’ etc. and these return many 1000’s of hits in our catalogue. The word cloud on the LMS is, therefore, more of a decorative item presently, bringing variety to the search pages, but not adding any additional value to the user’s search. This is very different from the same process in Aquabrowser, as while a search for ‘history’ returns many hits, the word cloud provides information on ways to refine that search.
Voyager word cloud
We ‘are working to change the word cloud from using single words for display to using phrases, as entered by the user. As the incidence of repeated phrases is quite low, we intend to stop counting duplicate searches and just display all the phrases as of the same value and level. We currently parse the last 2000 searches on the Library catalogue to create the wordcloud, for a phrase based word cloud we aim to just present the last 20 searches instead.
The Aquabrowser approach is not based on previous searches but rather word associations, translations, spelling variations and thesaurus terms:
2.
Search context
The other area we are investigating is looking at the context of a search.
10
Today’s Tabbloid PERSONAL NEWS FOR b.showers@jisc.ac.uk
22 October 2010
The current word cloud is generated after we have stripped everything but search terms from the search logs. When the word cloud term is clicked it does a keyword search for that term, whether it was originally an author search or a title search. This affects the quality of the search returns, and the value of using the word cloud.
Intended outcome(s) The objectives of the project were three-fold: 1. To undertake user research and persona development. Information gathered from the contextual enquiry and stakeholder interviews were used to create a set of personas which will benefit the project and the wider JISC community. The methodologies and processes used were fully documented in the project blog.
As we move to phrase searching we will be looking at using the last 20 searches as they are, rather than breaking these up to facilitate word counting and emphasis, we will also look to retain the type of search index used originally and embed that in the presentation via the word cloud. This should make the word cloud more useful to users as it will now recreate much more of what the original users intended when making their search.
2. To evaluate the usefulness of resource discovery services. Contextual enquiry was conducted to engage a broader base of users. The study determined the usefulness on site and off site which will provide a more in-depth understanding of usage and behavioural patterns.
We aim to have this work completed and can report further by end October. We hope to evaluate what users think of the finished word cloud as a tool for searching the Library Catalogue, ideally by connecting with the UX2.0 team who ran the AquabrowserUX JISC LMS project.
3. To evaluate the usability of resource discovery systems. Using the personas derived from the user research, typical end users were recruited to test the usability of the AquaBrowser interface. A report was published which discusses the findings and makes recommendations on how to improve the usability of UoE’s AquaBrowser.
RESEARCHING USABILITY
Aquabrowser blog OCT 20, 2010 12:11P.M.
The challenge There were a number of logistical issues that arose after the project kicked off. It became apparent that none of the team members had significant experience in persona development. In addition, the external commitments of subcontracted team members meant that progress was slower than anticipated. A period of learning to research established methodologies and processes for conducting interviews and analysing data took place. Consequently the persona development took longer than anticipated which delayed the recruitment of participants for usability testing (Obj3). The delay also meant that participants would be recruited during the summer months when the university is traditionally much quieter. This was a potential risk to recruit during this time but did not end up being problematic to the project. However the extension of time spent gathering qualitative data meant that it was not possible to validate the segmentation with quantitative data. This was perhaps too ambitious for a project of this scale.
Screen shot of University of Edinburgh’s AquaBrowser with resource discovery services highlighted.
The challenge of conducting the contextual enquiry within the library was to find participants willing to be observed and interviewed afterwards. The timing once again made this difficult as it took place during exams. The majority of people in the library at that time were focussed on one task which was to revise for exams. This meant that persuading them to spend ten minutes talking to researchers was understandably difficult. In addition to this, the type of users that were common in the library at that time were limited to those revising and whose needs were specific to a task and did not necessarily represent their behaviour at other times of the year.
Background The aim of the AquabrowserUX project was to evaluate the user experience of AquaBrowser at the University of Edinburgh (UoE). The AquaBrowser catalogue is a relatively new digital library service provided at UoE alongside the Classic catalogue provided via Voyager which has been established at the university for a number of years. A holistic evaluation was conducted throughout with a number of activities taking place. These included a contextual enquiry of library patrons within the library environment, stakeholder interviews for persona creation and usability testing.
Ensuring that real use data and participation was captured during the contextual enquiry was also a challenge. Capturing natural behaviour in context is often difficult to achieve and carries a risk of influence from
11
Today’s Tabbloid PERSONAL NEWS FOR b.showers@jisc.ac.uk
22 October 2010
the researcher. For example, to observe students in the library ethically it is necessary to inform subjects that they are being observed. However, the act of informing users may cause them to change their behaviour. In longitudinal studies the researcher is reliant on the participant selfreporting issues and behaviour, something which they are not always qualified to do effectively.
• To ensure realism of tasks in usability testing, user-generated tasks should be created with each participant. • Involve as many stakeholders as possible. We did not succeed in recruiting academic staff and were therefore unable to evaluate this user group however, the cooperation with Information Services through project member Liza Zamboglou did generate positive collaboration with the library during the contextual enquiry, persona interviews and usability testing.
Recruitment for the persona interviews and usability testing posed a challenge not only in finding enough people but also the right type of people. Users from a range of backgrounds and differing levels of exposure to AquaBrowser who fulfil the role of one of the personas could be potentially difficult and time-consuming to fulfil. As it turned out, recruitment of staff (excluding librarians) proved to be difficult and was something that we did not manage to successfully overcome.
• Findings from the user research and usability testing suggest that resource discovery services provided by AquaBrowser for UoE can be improved in order to be useful and easy to operate. • Looking back over the project and the methods used to collect user data we found that contextual enquiry is a very useful method of collecting natural user behaviour when used in conjunction with other techniques such as interviews and usability tests.
Established practice Resource discovery services for libraries have evolved significantly. There is an increasing use of dynamic user interface. Faceted searching for example provides a “navigational metaphor” for boolean search operations. AquaBrowser is a leading OPAC product which provides faceted searching and new resource discovery functions in the form of their dynamic Word Cloud. Early studies have suggested a propensity of faceted searching to result in serendipitous discovery, even for domain experts.
• The recruitment of participants was successful despite the risks highlighted above. The large number of respondents demonstrated that recruitment of students is not difficult when a small incentive is provided and can be achieved at a much lower cost than if a professional recruitment company had been used. • It is important to consider the timing of any recruitment before undertaking a user study. To maximise potential respondents, it is better to recruit during term time than between terms or during quieter periods. Although the response rate during the summer was still sufficient for persona interviews, the response rate during the autumn term was much greater. Academic staff should also be recruited separately through different streams in order to ensure all user groups are represented.
Closer to home, The University of Edinburgh library have conducted usability research earlier this year to understand user’s information seeking behaviour and identify issues with the current digital service in order to create a more streamlined and efficient system. The National Library of Scotland has also conducted a website assessment and user research on their digital library services in 2009. This research included creating a set of personas. Beyond this, the British Library are also in the process of conducting their own user research and persona creation.
Conclusions and recommendations The LMS advantage Overall the project outcomes from each of the objectives have been successfully delivered. The user research provided a great deal of data which enabled a set of personas to be created. This artifact will be useful to UoE digital library by providing a better understanding of its users. This will come in handy when embarking on any new service design. The process undertaken to create the personas was also fully documented and this in itself is a useful template for others to follow for their own persona creation.
Creating a set of library personas benefits the University of Edinburgh and the wider JISC community. The characteristics and information seeking behaviour outlined in the personas have been shown to be effective templates for the successful recruitment of participants for user studies. They can also help shape future developments in library services when consulted during the design of new services. The persona hypothesis can also be carried to other institutes who may want to create their own set of personas.
The usability testing has provided a report (Project Posts and Resources) which clearly identifies areas where the AquaBrowser catalogue can be improved. The usability report makes recommendations that if implemented has potential to improve the user experience of UoE AquaBrowser. Based on the findings from the usability testing and contextual enquiry, it is clear that the contextual issue and its position against the other OPAC (Classic) must be resolved. The opportunity for UoE to add additional services such as an advanced search and bookmarking system would also go far in improving the experience. We recommend that AquaBrowser and other institutes also take a look at the
The usability test report highlights a number of issues, outlined in Conclusions and Recommendations, which the university, AquaBrowser and other institutions can learn from. The methodology outlined in the report also provides guidance to those conducting usability testing for the first time and looking to embark on in-house recruitment instead of using external companies. Key points for effective practice
12
Today’s Tabbloid PERSONAL NEWS FOR b.showers@jisc.ac.uk
22 October 2010
report to see where improvements can be made. Evidence from the research found that the current representation of the Word Cloud is a big issue and should be addressed.
Conference Dissemination • UPA Conference 2010: Day 1
The personas can be quantified and used against future recruitment and design. All too often users are considered too late in a design (or redesign and restructuring) process. Assumptions are made about ‘typical’ users which are based more opinion than in fact. With concrete research behind comprehensive personas it is much easier to ensure that developments will benefit the primary user group.
• UPA Conference 2010: Day 2 User Research and Persona Development (Obj1) • Persona Creation: Data Gathering Evaluation • User Research and Persona Creation Part 1: Data Gathering Methods
Additional information Project Team
• User Research and Persona Creation Part 2: Segmentation – Six Steps to our Qualitative Personas
• Boon Low, Project Manager, Developer, boon.low@ed.ac.uk – University of Edinburgh National e-Science Centre
• User Research and Persona Creation Part 3: Introducing the Personas
• Lorraine Paterson, Usability Analyst, l.paterson@nesc.ac.uk – University of Edinburgh National e-Science Centre
• Audio files from each participant interview [link forthcoming once sound files are converted to suitable format]
• Liza Zamboglou, Usability Consultant, liza.zamboglou@ed.ac.uk – Senior Manager , University of Edinburgh Information Services
• Personas: http://bit.ly/digitallibrarypersonas
• David Hamill, Usability Specialist, web@goodusability.co.uk – Self-employed
Usefulness of Resource Discovery Services (Obj2) • Going “in the wild”
Project Website • Contextual enquiry wiki page • Aggregation of project blogs: http://pipes.yahoo.com/pipes/pipe.run?_id=5c4f2c6809c9b91018d8e74eb2dfd366 Usability of Resource Discover Services (Obj3) • Project Wiki: https://www.wiki.ed.ac.uk/display/UX2/AquaBrowserUX
• Recruitment evaluation and screening for personas • Realism in testing with search interfaces
PIMS entry • Usability Test Report – The University of Edinburgh AquaBrowser by David Hamill http://bit.ly/aquxusabilityreport
• https://pims.jisc.ac.uk/projects/view/1721 Project Posts and Resources
• Usability testing highlight videos (x3):
Project Plan
• http://bit.ly/aquxvideo1
• AquabrowserUX project IPR
• http://bit.ly/aquxvideo2
• User Study of AquaBrowser and UX2.0
• http://bit.ly/aquxvideo3
• Usefulness of Project Outputs (Studies, UCD)
AquabrowserUX Final Project Post
• AquabrowserUX Project Budget
• Final Project Post (this blog post)
• AquabrowserUX Project Team and User Engagement • AquabrowserUX Projected Timeline, Work plan and Project Methodology
13
Today’s Tabbloid PERSONAL NEWS FOR b.showers@jisc.ac.uk
22 October 2010
14