STAFF RESOURCES

Ask a Librarian LIVE Pilot Project Report

May 21, 2002

As a pilot project, between April 8th and May 17th 2002 the Rutgers Libraries allowed users to ask questions in “real time” using “LiveAssistance,” a web-based call center product from International Business Systems (IBSI), a Chantilly, Virginia based company.

Pre-Trial Preparation:

The LiveAssistance administrative setup was quite easy to use.  Where there were glitches LiveAssistance was very responsive in addressing the problem; a few enhancements were requested and readily provided.

The software allows for the creation of “predefined quotes;” i.e., preset answers to frequently asked questions; frequently required urls; greetings, etc.  A series of these based on our experience with the email AAL service was developed prior to the course of the trial.

The software also allows for the creation of an Exit Survey; a survey was designed prior to the start of the trial and was automatically pushed to each user at the completion of each session.

LiveAssistance resides on the company’s server and does not require any plug-ins or downloads on the part of either the operator or the user.

Staffing/Training:

Ten reference librarians participated in the pilot; all but one of these was currently, or had previously been, a member of the email Ask a Librarian team.

As we were beginning the project at a later point in the semester than originally anticipated, no formal training was attempted.  Librarians were encouraged to practice with each other, or connect as both User and Librarian, in order to become familiar with the layout and features of the software.

Ask a Librarian LIVE:

A link to “Ask a Librarian LIVE” was placed on the Libraries Ask a Librarian / FAQs page [ http://www.libraries.rutgers.edu/rul/ask_a_lib/ask_a_lib.shtml ].  The link launched a customized pop-up box which requested the User’s name, email address, affiliation, and question.  The user would then click on “Enter Chat” to connect to a Librarian.

The Ask a Librarian LIVE link was available from April 8th through May 17th.  The service was available Monday – Friday, 1 pm to 4 pm.

In the 72[1] hours that the service was available between April 8th and May 10th, a total of 110 “chats” were attempted.  On average there were 5 attempts per three-hour session.  67 (61 percent) were successful (i.e., the librarian and the user connected); 43 (39 percent) were unsuccessful (i.e., the librarian and the user did not connect).  In most cases, the failure to connect was due to system/network problems; in other cases the user abandoned the connection prior to speaking to a librarian.

The average transaction took about six and a half minutes; this is considerably below the normally reported 15-17 minute average.  This lower transaction time probably is at least partially due to the fact that the range of questions paralleled those submitted to the regular AAL service and so in many cases (requests for PINs; questions about renewing, etc.) required nothing more than a referral to the appropriate form or forum.  The longest transaction (a “do you have” question from someone working with journal title abbreviations) took 34 minutes.

Ask a Librarian LIVE
April 8-May 10
Summary Statistics
Total Hours Available 72
 
Chats Attempted: 110
Mondays: 38 35%
Tuesdays: 21 19%
Wednesdays: 17 16%
Thursdays: 16 15%
Fridays: 18 16%
 
Average Attempts per Session: 5
Average Chat Time: 0:06:27
Median Chat Time: 0:05:25
 
Disconnected; Abandoned, etc.: 43 39%

User Reaction:

Most of those who used the service seemed to be highly satisfied, even when they were merely referred to a form or back to the email Ask a Librarian service.  Just knowing that they had made contact seemed enough to generate a level of satisfaction.

Twenty-nine users (approximately 40% of all users who connected successfully) completed the Exit Survey.  Of these twenty-nine, twenty-four identified themselves as Rutgers affiliates while five identified themselves as “Other.”

To the question “How satisfied were you with the quality of the answer you received?  Please rate your level of satisfaction, with 1 as the lowest score and 5 as the highest score” nineteen of the twenty-eight respondents (68 percent) rated their satisfaction at the “5 – High-Very Satisfied” level.

User Satisfaction Exit Survey Results

Librarian Reactions[2]:

Most librarians that participated in Ask a Librarian LIVE found it to be an interesting experience; most indicated that they would be willing to participate in any continued pilot.  The ability to let users choose the format that is the most comfortable for them as well as the clearly high user satisfaction rate were cited as the most positive aspects of this kind of service.

ISSUES:

Equipment:

Inadequate equipment resulted in a high level of frustration for librarians whose office PCs were slow or otherwise outdated.  The use of a PC without a sound card or speakers was especially problematic.  Since the system is designed to make an announcement whenever a user is connecting to the call center, the lack of a sound card/speakers required constant vigilance on the part of the Librarian to not miss any incoming calls.  It also resulted in a stream of “WAV file not found” message screens each of which had to be closed manually.

Network Problems:

Most participants experienced intermittent network problems that resulted in users being unable to connect to the librarian, or in the user and/or librarian being disconnected from LiveAssistance.  On at least four of the afternoons during which the service was offered, the problem was severe and constant.[3] 

Software:

Although the software is basically quite easy to use, most librarians found the LiveAssistance Operator screen fragmented and confusing.  Most felt that they would have benefited from additional training and the practice time needed to become familiar with the screen layout.  Some did not realize that they could forward session transcripts to the User after each session or that the User would automatically be asked to complete a survey at the end of each session.

Most found the “Quotes” feature to be extremely useful.

The ability to push web pages to users was found to be extremely useful.

The fact that a record is created even for users who fail to connect to the librarian and that in most cases that record includes the user’s name, email, and question is also very useful—in a number of cases users who failed to connect were contacted and their question answered via email.

The fact that a transcript could be sent to the user at the end of each session also meant that the user would have a record of instructions, titles, urls, etc. referred to during the session.  When appropriate, a transcript could also be forwarded to another librarian for additional consultation.

Librarian/User Interaction:

A number of the librarians found both the mechanics of doing reference in a “chat” mode and the limitations on the kind of answers that could be supplied in that mode to be extremely frustrating.

Unlike email reference in which a librarian has a set period within which to respond to a query, or the Reference Desk, where it is understood that the librarian might have stepped away or be out in the Reference area helping someone, in “real time” a librarian must be available and waiting to answer during specific scheduled times.  Some librarians found it difficult to sit “trapped in their office” for three straight hours.  [Although others thought that being forced to spend that time in their office allowed them to get a lot of other work done!]  This was also an issue for librarians in special libraries where they might be the only librarian available and have to abandon an in-house user to answer a remote user.

Even twenty seconds seems like a long time in “live” mode.  There was a constant need to reassure the user that they were still connected and that someone was still working on their question.

The highest level of frustration seemed to stem from the fact that the librarian had limited/no time to investigate the question; the live format dictates short, “quick and dirty” responses.  Some librarians felt that they’d had to reduce their responses to sound bites and found it difficult to adapt to the more casual, “non-professional” style that seems to work best in this format.

RECOMMENDATIONS:

While some participants felt that the limited usage of the service coupled with the issues noted above did not warrant continuation of a “live” service at this time, most felt that the pilot should be extended through the Fall 2002 semester.

The librarians participating in the pilot were “live” for a maximum of only three sessions.  There was a feeling that continued practice and exposure would do much to alleviate “screen confusion” and allow the librarians to become more comfortable with the format requirements.[4] 

Now that the needs are better understood, any new librarians participants could be given training/orientation that would better prepare them for interacting in this forum.[5]

Extending with the same hours through the summer would give us a different population—one with different time constraints—from which to gather data.

We also thought that it would be useful in the Fall to try a different service schedule.

As part of the Exit Survey, users were asked to select which of several three-hour time slots for this service would best meet their needs.  As might have been expected, the most votes (55%) were cast for the period in which these particular users had asked their questions—1 pm to 4 pm Mondays thru Fridays.

Despite this stated preference, it is recommended that during the fall semester the service be offered Sundays thru Thursdays, 7 pm to 10 pm.  While this is not the standard “peak” period, a “live” service offered during this time slot would more truly be complementary to other reference services and would allow us to better gauge during which period the service is most likely to be used.

We could also use the extended period to explore some features of the LiveAssistance software that were not utilized during the initial trial such as the ability to generate reports based on librarian categorizations of each session.

During the initial pilot, publicity concerning the service was intentionally kept at a minimum so that the librarians could get a feel for the service without being overwhelmed (or raising undue expectations).  For the extended trial, especially in the fall, more effort should be made to publicize the availability of the service.

It is not recommended that the service be expanded past the 15 hour per week limit.

Costs:

Continuing the trial through the Fall semester would cost approximately $1300:

LiveAssistance Startup Fee (remainder):$250.00
LiveAssistance software @ $150 per month:1050.00
 $1300.00

[1] There was a power failure on the New Brunswick campus during  one of the afternoons that the service was being offered; those hours were subtracted from the total.

[2] In addition to communications, problem reports, etc., sent via email, the librarians who participated in the pilot also met on May 15th to discuss their experiences with the service.

[3] On one of those occasions LiveAssistance was contacted and the problem diagnosed as an AT&T pipe that was down between LiveAssistance and Rutgers.  A problem report was filed by both systems’ departments and the problem rectified within 10 minutes of reporting.  So it is possible that reporting as problems occurred may have resulted in fewer problems.

[4] It should be noted that while most librarians did not identify themselves as regular “chatters,” the one librarian who was already comfortable in that mode did not experience many of the frustrations reported by the other participants.

[5] One thing that should probably be emphasized to both new and old participants is that in many cases “live reference” can best be used as a starting point that serves to point the user in the right direction and there is nothing to prevent email follow-ups, etc.  And in some cases the best, and least frustrating and stressful, initial response might be “Are you near a telephone?” or “I’m going to have to do some work on this; can I email you later?”



nxb 5/21/02 Ask a Librarian LIVE Report
 
URL: http://www.libraries.rutgers.edu/rul/staff/groups/rt_ref/reports/ask_live_report.shtml
Website Feedback  |  Privacy Policy

© Copyright 1997-2013, Rutgers University Libraries   (Further Copyright Information)