Selecting and Evaluating Electronic Resources: An Annotated Bibliography

by Susan J. Beck and Howard Dess
Rutgers University Library System
February 24, 1999

Use this annotated bibliography to explore the issues relating to the selection and evaluation of electronic resources. In it we direct librarians to resources pertinent to the issue of database evaluation.

All items included are available on the web for ease of access. The data for this list was found by mining the output of ProQuest Direct and Hot Bot ( searches.

The bibliography includes articles on the following topics:

Evaluating Databases or Truth In Advertising: The Basics

Jacso, Peter. A proposal for database 'nutrition and ingredient' labeling. Database February 1993 16 (1): 7+ Available Online fulltext through Proquest Direct February 24, 1999.
In this editorial, Peter Jacso, advocates labeling database products with descriptions of their coverage, content and accessibility. He presents common misleading discrepancies database producers use when describing their products in relation to size of database, time periods covered and content including length of abstract, number of subject headings, access points and type of document records.

Creation Of Collection Development Policies For Electronic Resources

Johnson, Peggy. Dollars and sense: collection policies for electronic resources. Technicalities. June 1998; 18(6): 10-12. Available Online fulltext through Proquest Direct February 24, 1999.
The author provides guidance on creating a collection development policy for electronic resources. Policies should include information concerning service and support impacts on reference services, acquisitions and cataloging, automated services and facilities.

Developing Evaluation Criteria for Electronic Resources

Metz, Paul; Armstrong, Alan; Beagle, Dave; Brown, Ladd et. al. A Standardized Form for Evaluation and description of electronic resources under consideration by the Virginia Tech University Libraries. Technicalities November December 1998 18(10):9-10. Available Online fulltext through Proquest Direct February 24, 1999.
Collection development librarians at Virginia Tech University created a form to be used when evaluating electronic resources for adding to their collections. Criteria include: general description, suitability to the mission of the institution, costs and pricing including one time fees, estimating indirect costs, licensing issues, functionality, and archiving. The form is included for use.

University Collection Policies for Selection of Electronic Resources

Rutgers University Libraries. Acquiring networked resources: RUL policies and procedures. Available Online: February 24, 1999
This document currently consists of the following sections: Draft principles for acquiring networked resources; How to recommend the acquisition of networked resources; Networked resource recommendation form; Recommended networked resources requiring central funds; and Cataloging policies for electronic journals.

University of Arizona library policy for selecting and acquiring electronic products - June 30, 1996. Available online: February 24, 1999
This is an example of an excellent policy created by librarians from the University of Arizona Library. Their basic premises include guidelines which cover the important issues of resource duplication and overlap. Selection criteria include customer needs, cost considerations, product quality, service concerns such as ease of use, and technical concerns including ease of software/hardware installation. Another evaluation is mandated two years after purchase of the product to make sure it still conforms to the needs of the users and institution. Training and marketing issues are also covered. Appendices include information on copyright statements, lease and purchase agreements, site licenses, and advice to selectors on licenses.

University of California Libraries Collection Development Committee. Principles for acquiring and licensing information in digital formats. May 22,1996. Available online: February 24, 1999
This is a systemwide document used in developing and reviewing proposals and negotiating contracts with providers of digital information. The basic criteria used in this policy includes collection development policy issues such as user needs, maintaining balance among disciplines, acquisitions which offer economies of scale by benefiting the most faculty and students, resources which offer significant added value over printed resources such as timeliness, greater functionality, greater access, remote access, improved resource sharing, ease of archiving, replacing and preservation. Costs and pricing guidelines include the principle that indicates that the system "should not be required to purchase both print and its digital equivalent". Strong licensing agreement principles are demanded by the library system, including permanent rights to information that have been paid for, in the event of cancellation of contracts. Functionality including ease of use interfaces, and appropriate documentation are expected as is the notification of any format or content change by the producer. Archiving responsibilities should be clear.

University of Iowa Libraries. Policy for electronic resources management. June 1996. Available online: February 24, 1999.
This policy provides guidelines for the University of Iowa Libraries in the "selection, acquisituion,provision of access to, and maintenance of electronic resources". Criteria described includes Scope of the policy, selection and acquisitions responsibilities of the library, implementation responsibilities of the library, duplicate copies, copies in multiple formats and networking, replacements, gifts, preservation, deselection and policy review.

Consortial Database Selection and Evaluation Issues

British Columbia Electronic Library Network. Fulltext Initiative - Database Selection Criteria. Available online: February 24, 1999.
This is the selection criteria document developed by ELN for selection of full text databases. The criteria includes the following categories: access and authentication, content, functionality, integration and interoperability (Z39.50 implementation), licensing issues, output options, pricing, statistical reporting, technical formats for both fulltext and image, and user support. This policy includes The ELN Fulltext Initiative Product Evaluation Form which allows for product evaluation based on a five item scale from very poorly to vey well for each selection criteria. The Fulltext Database Services User Feedback Form, provides for user evaluation of electronic resources on an easy to use online form which asks the following questions:

  • What fulltext databases did you use today?
  • What subject areas did you search?
  • How relevant were the results you retrieved?
  • Was the search system user friendly?
  • How do you benefit from using fulltext databases? (e.g. saved time, more resources etc.)
  • Please provide any additional comments that will help your library evaluate this service.
  • Will you permit your library or the Electronic Library Network to include your comments in reports to administrators or prospective funders in support of continuation/implementation to fulltext database services?
  • Which institution do you attend?

Klingler, Tom. Sharing the Load: .............the pros and cons of consortial database use.[ ALA Midwinter 1997 ACRL STS Science and Technology Database Discussion Group]. Available online: February 24, 1999.
Tom Klinger of Kent State University provides an interesting analysis of the pros and cons of consortial database use. He focuses on the following topics: Database selection process and consortium governance and considers a con that often "decisions are made at a level too far above the user" in the selection process. He includes criteria for database selection, pros and cons related to database and licensing/pricing for consortiums, asking the question "can member libraries, cancel save or reallocate resources when either high end or low-end databases are acquired?" He also asks the question: " are there selection/cancellation criteria in place?" Other issues incorporated into this discussion include database funding (centralized, distributed shared - who's in control?), To load or not to load which deals with where consortial databases reside and gateways. How will contracting with a variety of vendors impact the consortium. Has the consortium developed performance standards and evaluation processes to "dump the vendor or product? " Other topics include custom data processing, front-ends, access issues such as IP checking, periodicals holdings data, service, client, product support, documentation, training and promotion.

MINITEX Task Force on Database Evaluation. MINITEX criteria for the evaluation of electronic databases Available online: February 24, 1999
The MINITEX Library Information Network - (Minnesota) created these criteria to be used to evaluate specific reference and full text databases for the network. The basic criteria include: database coverage, scope and structure, search system functionality, clients available, display, downloading, printing, email, quality control, management, technical and other. What is unique about this set of criteria, is that MINITEX has prioritized their top five criteria ( technical considerations, database attributes, search usability, result usability, and vendor responsiveness and accountability) and created critical evaluative measures for performance stanbdards.

Evaluation Criteria for Comparing Databases

Anagnostelis, Betsy, Cooke, Alison. Evaluation criteria for different versions of the same database - a comparison of Medline services available via the World Wide Web Available:
This article includes a general review of database evaluation criteria, criteria for evaluating versions of the same database, and evaluations of Internet based Medline services. Eight WWW based Medline Services are evaluated to develop and test evaluation criteria. The criteria tested includes the following: general information on the service, database content, retrieval mechanisms which include an evaluation of general search features, freetext searching, natural language queries, thesaurus searching, command line syntax, display and output functions, ease of use, unique features and help and user support features.

Karp. Rashelle. Comparing three full-text journal services. The Booklist. May 15, 1998; 94(18):1646-1650. February 24, 1999.
The authors compare three full text journal services: UMI's Proquest Direct, EBSCO's EBSCOhost and Information Access InfoTrac. Criteria used in the comparison include: coverage, scope, indexing and search mechanisms, number of hits, database currency, ability to connect to the database, output formats, and statistics and other administrative information.

Hoxmeier, John A. A Framework for Assessing Database Quality. College of Business - Computer Information Systems - Colorado State University - Fort Collins, CO 80523. Available:
Provides a summary overview of database quality assessment criteria and offers a model that provides a framework for establishing such evaluation systems.

Database Quality Criteria. Available:
This presentation is essentially a detailed summary of an earlier article published by Reva Basch in Database Searcher, Oct. 1990. It provides a very thorough and comprehensive listing of the many variables and factors that should be considered when evaluating any database.

Selection Criteria for Quality Controlled Information Gateways

Work Package 3 of Telematics for Research project DESIRE. Available: quality/bibliography.html This work is an exhaustive bibliography on the subject of database evaluation.

Licensing Standards and Guidelines for Acquisition of Electronic Resources

Principles for licensing electronic resources. Final draft July 15, 1997. Available online: February 24, 1999.
These principles were approved by the following library associations: American Association of Law Libraries, American Library Association, Association of Academic Health Sciences Libraries, Association of Research Libraries, Medical Library Association, Special Libraries Association.

Liblicense: Licensing Digital Information: A Resource for Librarians. Available online: February 24, 1999.
Produced at the Yale University Library, Liblicense is a comprehensive guide to licensing issues for librarians.Included is a glossary of Definitions of Words and Phrases CommonlyFound in Licensing Agreements, Licensing Terms & Descriptions, Licenses Provided by Publishers, Licensing Resources, Bibliography of Licensing Sources, National Site License Initiatives, and Authors' Licenses. There ia Liblicense-L discussion group, and the archives are avilable at this site.

For Further Reading

RUSA/CODES Computer-Based Methods and Resources Committee. Selection of Full-Text Fee-Based Internet Accessible Resources Bibliography Available online: February 24, 1999.
This bibliography includes brief annotations to over 60 items published from 1991 to 1997 in books, journal articles and web site relating to the literature of the following areas: Collection Development, Document Delivery, Electronic Publishing, Implications of Electronic Access , Online Searching, Reference Access, Information Retrieval, and Publishing.

RASD/CODES Collection Development Policy Committee. Collection development policies for electronic resources: an annotated bibliography. Available online: February 24, 1999.
This annotated bibliography provides access to 13 items relating to collection development policies for electronic resources covering the time period from 1989-1995.

Website Feedback  |  Privacy Policy
© Copyright 1997-, Rutgers University Libraries