Chapter 17—
On-line Books at Columbia
Early Findings on Use, Satisfaction, and Effect

Mary Summerfield and Carol A. Mandel
with Paul Kantor, Consultant

Introduction

The Online Books Evaluation Project at Columbia University explores the potential for on-line books to become significant resources in academic libraries by analyzing (1) the Columbia community's adoption of and reaction to various on-line books and delivery system features provided by the libraries over the period of the project; (2) the relative life-cycle costs of producing, owning, and using on-line books and their print counterparts; and (3) the implications of intellectual property regulations and traditions of scholarly communications and publishing for the on-line format.

On-line books might enhance the scholarly processes of research, dissemination of findings, teaching, and learning. Alternatively, or in addition, they might enable publishers, libraries, and scholars to reduce the costs of disseminating and using scholarship. For example:

• If the scholarly community were prepared to use some or all categories of books for some or all purposes in an on-line format instead of a print format, publishers, libraries, and bookstores might be able to trim costs as well as enhance access to these books.^[1]

• If on-line books made scholars more efficient or effective in their work of research, teaching, and learning so as to enhance revenues or reduce operating costs for their institutions, on-line books might be worth adopting even if they were no less costly than print books.

• If an on-line format became standard, publishers could offer low-cost on-line access to institutions that would not normally have purchased print copies, thus expanding both convenient access to scholarship to faculty and students at those institutions and publishers' revenues from these books.^[2]

This paper focuses on user response to on-line books and reports on:^[3]

1. the conceptual framework for the project

2. background information on the status of the collection and other relevant project elements, particularly design considerations

3. the methodology for measuring adoption of on-line books by the Columbia community

4. early findings on use of on-line books and other on-line resources

5. early findings on attitudes toward on-line books

Conceptual Framework

The variables representing usage of a system of scholarly communication and research are both effects and causes. Since scholars, the users of the system, are highly intelligent and adaptive, the effect of the system will influence their behavior, establishing a kind of feedback loop. As the diagram in Figure 17.1 shows, there are two key loops. The upper one, shown by the dark arrows, reflects an idealized picture of university administration. In this picture, the features of any system are adjusted so that, when used by faculty and students, they improve institutional effectiveness. This adjustment occurs in the context of continual adaptation on the part of the users of the system, as shown by the lighter colored arrows in the lower feedback loop.

These feedback loops are constrained by the continual change of the environment, which affects the expectations and activities of the users, affects the kind of features that can be built into the system, and affects the very management that is bringing the system into existence. The dotted arrows show this interaction.

Our primary research goal, in relation to users, uses, and impacts, is to understand these relationships, using data gathered by library circulation systems, Internet servers, and surveys and interviews of users themselves.

The On-line Books Collection

The project began formal activity in January 1995. However, discussions with publishers began in 1993, if not earlier. As noted in the project's Analytical Principles and Design document, "The Online Books Evaluation Project is a component of the developing digital library at Columbia University. As part of its digital library effort, the Columbia University Libraries is acquiring a variety of reference and monographic books in electronic format to be included on the campus network; in most cases, those books will be available only to members of the Columbia community. Some of the books are being purchased; others are being provided on a pilot project basis by publishers who are seeking to understand how the academic community will use online books if they become more widely available in the future."

[Full Size]

Figure 17.1.
Interrelation of Factors Involved in the Use and Impact of On-line Books

Design of the On-line Books Collection

When this project was proposed, the World Wide Web was just emerging, and we expected to develop custom SGML browsers, just as other on-line projects were doing at the time. However, by the time the project was ready to mount books on-line,^[4] the Web seemed the best delivery system for maximizing availability of the books to scholars.

Many other on-line projects are providing users with materials in PDF, scanned, or bitmapped format. These formats are effective for journal articles, which are finely indexed through existing sources and which are short and easily printed. However, the greatest potential for added value from on-line books comes with truly digital books. Only this on-line format allows the development of interactive books that take advantage of the current and anticipated capabilities of Web technology, such as the inclusion of sound and video, data files and software for manipulating data, and links to other on-line resources. Perhaps only such enhanced on-line books will offer sufficient advantages over traditional print format that scholars will be willing to substitute them for the print format for some or all of their modes of use and for some or all classes of books.

As of June 1997, the project included 96 on-line texts. The libraries have each book in print form-circulating from the regular collection or from reserves or noncirculating in reference-as well as in one or more on-line formats. Appendix A summarizes the print access modes for all the modern books in the collection.

Methodology for Studying Use of and Reactions to Various Book Formats

The project's Analytical Principles and Design document lays out the evaluation methodology.^[5] Formulated in the first year of the project, this methodology re-

mains the working plan. Here are some of the key measures for documenting use of the on-line books:

• The records of the Columbia computing system provide, for the most part, the use data for the on-line books. For books accessed via the World Wide Web, information on date, time, and duration of session involving an on-line book, user's cohort, location of computer, number of requests, amount of the book requested, and means of accessing the book will be available. These data became available in summer 1997 with the full implementation of the authentication system and related databases.

• Circulation data for each print book in the regular collection provides information on number of times a book circulates, circulation by cohort, duration of circulation, number of holds, and recalls. For most libraries, the data available for reserve books is the same as that for books in the regular collection as the CLIO circulation system is used for both.

• The records of the Columbia computing system provide, for the most part, the use data for the books accessed via CNet, Columbia's original, gopherbased Campus Wide Information System, including the number of sessions and their date and time. These records do not include the duration of the session, the activity during the session, e.g., printing or saving, or anything about the user. Thus, all we can analyze are the patterns of use by time of day, day of week, and over time.

• Until March 15, 1997, for books accessed via CWeb, we knew the use immediately preceding the hit on the book and the day and time of the hit. For data collected through that point, our analysis is constrained to patterns of use by time of day, day of the week, and over time. By manual examination of server data, we counted how many hits a user made on our collection during one session and the nature of those hits.

• Since March 15, 1997, we are able to link user and usage information and conduct a series of analyses involving titles used, number of hits, number of books used, and so on by individual and to group those individuals by department, position, and age. These data do not yet include sessions of use, just the magnitude of overall use during the period. Session-specific data are available starting in fall 1997.

We are using a wide range of tools in trying to understand the factors that influence use of on-line books. Table 17.1 summarizes our complex array of surveys and interviews.

Use of Books in On-line Collection

At this point we will report on (1) trends in use of the on-line books; (2) user location and cohort; and (3) use of the on-line books by individuals. Summarized

below separately are findings for reference works and nonreference books, e.g., monographs and collections.

Reference Books

Three reference works have been available on-line long enough to have generated substantial usage data.^[6] These are The Concise Columbia Electronic Encyclopedia, Columbia Granger's World of Poetry, and The Oxford English Dictionary. Three other titles (Chaucer Name Dictionary, African American Women, Native American Women ) have been on-line only since early 1997, so usage data are very short-term for these titles. All three are accessible both through CNet and CWeb.

Most available reference books are used more heavily on-line than in print.

Of the six reference works in the collection, only The Oxford English Dictionary receives sizable use in its print form in the Columbia libraries. At most a handful of scholars use the library copies of the others each month. As the accompanying tables and figure show, each of these books receives much more use on-line. On-line availability seems to increase awareness of these resources as well as make access more convenient.

Early on-line reference books have experienced falling usage over time, substitution of use of a new delivery system for an old one, or a smaller rate of growth of use than might be expected given the explosion in access to and use of on-line resources in general.

In the early to mid-1990s, novelty may have brought curious scholars to the on-line format somewhat without concern for design, the utility of the delivery system, or the qualities of the books. With enhancement in delivery systems and expansion in the number of on-line books, being on-line is no longer a guarantee that a book will attract users. As access to the Web spreads, new graphical Web delivery systems are offering superior performance that is increasingly likely to draw scholars away from these early, text-based systems. In addition, as more competing resources come on-line and provide information that serves the immediate needs of a user better or offer a more attractive, user-friendly format, scholars are less likely to find or to choose to use any single resource.

The Oxford English Dictionary is the most heavily used reference work in the collection. Its CNet format offers good analytic functionality but it is difficult to use. The CWeb format is attractive and easy to use, but its functionality is limited to looking up a definition or browsing the contents.^[7]

OED CNet usage dropped 59% from fourth quarter 1994 (2,856 sessions) to first quarter 1997 (1,167 sessions). OED CWeb use increased by 27% from fall semester 1996 (1,825 hits) to spring semester 1997 (2,326 hits). The OED had 173 unique users in the period from March 15 to May 31, 1997, with an average of 2.8 hits per user.

The Concise Columbia Electronic Encyclopedia remains on the text-based platform CNet. As Figure 17.2 shows, usage declined 84% over the past three years, from 1,551 sessions in April 1994 to 250 sessions in April 1997. Usage declined most in the 1996-97 academic year; 7,861 sessions were registered from September 1995 to May 1996 and 2,941 sessions (63% fewer) from September 1996 to May 1997.

Columbia now provides CWeb access to the Encyclopedia Britannica (directly from the publisher's server); many scholars may be using this resource instead of the Concise Encyclopedia. Recently the Columbia community has registered about 5,000 textual hits a month on the Encyclopedia Britannica.

Columbia Granger's World of Poetry is available on both CNet and CWeb. The CNet version is a Lynx, nongraphical Web formulation of the CWeb version. This resource, which became available to the community in on-line form in October 1994, locates a poem in an anthology by author, subject, title, first line, or keywords in its title or first line. In addition, it provides easy access to the 10,000 most often anthologized poems. In first quarter 1995, CNet sessions totaled 718; in first quarter 1997, they totaled 90 (or about one a day). CWeb hits totaled about 700 in the first quarter of 1997. Thus, even though it has declined, total usage of Granger's is still considerable.

Garland's Chaucer Name Dictionary was added to the CWeb collection at the end of 1996. Native American Women was added in January 1997, and African American Women went on-line in February 1997. Their early usage on CWeb is shown in Table 17.2.

[Full Size]

Figure 17.2.
Concise Columbia Electronic Encyclopedia Sessions, 1994-1997: CNet

Nonreference Books

The Online Books Evaluation Project includes two types of nonreference books: Past Masters, 54 classical texts in social thought; and modern monographs and collections from Columbia University Press, Oxford University Press, and Simon and Schuster Higher Education. Most of these books came on-line during the 1996-97 academic year.

On-line scholarly monographs are available to and used by more people than their print counterparts in the library collection.

Once a print book is in circulation, it can be unavailable to other scholars for hours (the reserve collection) or weeks or months (the regular collection). An online book is always available to any authorized user who has access to a computer with an Internet connection and a graphical Web browser.

Table 17.3 tracks usage of the contemporary nonreference books in the on-line collection for the last part of the spring semester 1997 and in the print circulation of these titles for the first six months of 1997. Fourteen of these books had no online use during this 2.5-month measurement period; 12 had no print circulations during their 6-month measurement period. In total, the on-line versions had 122 users while the print versions had 75 users. Looking at only the on-line books that circulated in print form, we find 122 on-line users and 45 print circulations, or that there were nearly three times as many on-line users as circulations. These data suggest that, compared for an equal period, these books will have many more users in on-line form than in print form.^[8]

An on-line book may attract scholars who would not have seen it otherwise.

Once a group within the community becomes aware of the on-line books, they are likely to review books in the collection that seem related to their interests-at least while the collection is small. For example, half of the use of Autonomous Agents: From Self Control to Autonomy was from social work host computers. This title might seem related to social work issues even though it is not a social work book or part of the collection of the Social Work Library.

The fifth and sixth most used books-Self Expressions: Mind, Morals, and the Meaning of Life and Bangs, Crunches, Whimpers, and Shrieks -are both philosophy titles.

• Self Expressions is listed in the Current Social Science Web page along with the social work titles. Five of its seven users were from the School of Social Work, one from the Center for Neurobiology and Behavior, and one from Electrical Engineering.

(table continued on next page)

(table continued from previous page)

• Bangs, Crunches, Whimpers, and Shrieks is listed under Physics in the Current Science Web page. Two of its seven users were from the Physics department, another two from unidentified departments, and one each from Electrical Engineering, Engineering, and General Studies.

It is not clear whether scholars' productivity or work quality will be enhanced by such serendipity. The important concept of collation is transformed, in the networked environment, to a diversity of finding and navigational systems. As the online collection expands, browsing will require the focused use of on-line search tools rather than use of project-oriented Web pages. However, the Web's search systems may uncover a book's relevance to a scholar's work when the library catalog or browsing the library or bookstore shelves would not have done so. This new ability to identify relevant books should improve scholars' research and teaching.

Scholars use some on-line books relatively little.

As Table 17.3 shows, use of on-line books is not evenly balanced among titles. Instead it is driven by course demands or other interest in a book. The initial use of the 54 on-line Past Masters classic texts in social thought confirms this finding. In academic year 1996-97, these texts registered a total of about 2,460 hits from the Columbia scholarly community. However, 1,692 (69%) of these hits were on only eight (15%) of the titles, for an average total of 212 hits each, or 24 hits each per month. The other 46 texts averaged about 17 hits each over this period, or about two hits each per month.^[9]

Patterns of usage may be expected to change over time as various texts are used in courses or by researchers and as the Columbia community becomes more aware of the on-line collections. In general, it is likely that nonreference books that are being used in courses, but which the students need not own, will be in greater demand on-line than will books that students must own or books that are of interest to only a small set of scholars.

The data to date suggest that to the extent that there are meaningful costs to creating on-line books and to maintaining them as part of a collection, publishing and library planners must select items for the on-line collection carefully. The decision rules will vary depending on what type of organization is taking on the risks of providing the access to the on-line books.

Some scholars, especially students with a reading assignment that is in the on-line collection, are looking at on-line books in some depth, suggesting that they find value in this means of access.

As Table 17.3 shows, the on-line books averaged 5.5 hits per unique user, suggesting that some users are looking at several elements of the book or at some elements repeatedly.^[10] In fall 1996, three social work books were most intensively used because they were assigned reading for courses. We analyzed the server statistics through the end of 1996 for these books in an effort to learn how deeply the books

were used-to what extent use sessions included book chapters, the search engine, the pagination feature, and so on.

Table 17.4 shows that relatively few sessions (7%-24%) involved someone going only to the Title Page/Table of Contents file for a book. Many sessions (28%-59%) involved use of more than one chapter of the book; sessions averaged 1.4 to 3.5 hits on chapters, depending on the book used. Some users would seem to be repeat users who had set up a bookmark for a chapter in the book or had made a note of the URL because some sessions (9%-17%) did not include a hit on the Table of Contents/Title Page.

Table 17.5, illustrating the distribution of hits on the on-line books collection per unique user over the last part of the spring 1997 semester, indicates that while many users are making quite cursory use of the on-line books, more are looking at multiple files (e.g., reference entry, chapter) in the collection. As Table 17.6 shows, the distribution of unique titles viewed by these users over this period indicates that most users come to the collection to look at a single book. The greatest number of books used by a single person was seven (by two persons).

Not surprisingly, there is a certain correlation between number of hits and number of titles used. Those users with only one hit could only have looked at one title (42% of those using one book). The range of hits among those who used only one book is wide-20 (9%) had more than 10 hits. Six users had more than 25 hits; two of them looked at only one book, one each at two and three books, and two at seven books. These statistics indicate some significant use of the collection as measured by average number of hits per title used.

However, hits on several titles need not indicate heavy use of the on-line books collection. The individual who looked at five books had a total of only six to ten hits, as did four of the seven people who looked at four books (one to two hits each). The person who looked at six books had 11 to 15 hits in total (an average of about two hits per book).

Table 17.7 shows that graduate students tended to have more hits, and undergraduates and faculty, fewer hits.

The preceding discussion highlights the current data on usage by individuals. Using newer data on sessions, we will be able to derive valuable information on user behavior-not only number of books used and hits on those books but parts of the book used and repeat usership. We will begin to see revealed preference in user behavior and will be less reliant on responses to questionnaires.^[11]

Data for the last half of the spring 1997 semester suggest that when a social work book available in both print and on-line formats was used in a course, the share of students using the on-line version was at most one-quarter.

Table 17.3 shows that the four most used nonreference books were all in the field of social work. Almost 91% of the users of these books were from the School of Social Work; they accounted for 98% of the hits on those books. The vast majority of these users (56 of 64) were graduate students. With the exception of the

most used book, Task Strategies, these texts were on reserve for social work courses during the spring 1997 semester.

• Three sections, with a total of about 70 students, used Supervision in Social Work as a key text. Thus, potentially, if all seven graduate students who used this book were participants in these courses, about 10% of the most likely student user group actually used this book on-line during this period.^[12]

• Three other course sections, again with about 70 students in total, used Mutual Aid Groups. This book was a major reading; in fact, one of its authors taught two of the sections in which it was used. Sixteen graduate students used this title for a potential penetration of about 23%.

• Philosophical Foundations of Social Work (as well as Qualitative Research in Social Work ) was on reserve for a doctoral seminar that had an enrollment of 11 students. The instructor reported that this book was a major text in the course that students would have bought traditionally. She did not know how many of her students used the on-line version. If all eight users-seven graduate

students and the one professional student-were class members, that suggests a substantial penetration for that small class. However, it is likely that some of these users were not enrolled in that course.

Location of Use of On-line Books

Scholars are not using on-line books from off-campus locations to the extent expected.

One of the key potential advantages to on-line books is their instant availability to scholars at any location at which they have access to a computer with a modem and a graphical Web browser. This benefit might well lead to substantial use of the on-line books from locations other than the Columbia campus. So far we are seeing only modest use of the books from off-campus.

From May 1996 to March 1997, 11% of the hits on the Columbia University Press nonreference books were dial-up connections from off-campus. Looking at the use of the social work titles, we find that computers in the School of Social Work were responsible for the following shares of hits on the social work titles:

Closer analysis of the usage data finds substantial use from the computer lab in the School of Social Work as well as from faculty computers. This finding suggests that many of the graduate students, most of whom do not live on or near campus, may not have Web access in their homes and, hence, are not equipped at this point in time to take full advantage of the on-line books.^[13] Students who use the on-line books at the School of Social Work, however, avoid walking the several blocks to the social work library, worrying about the library's hours, or encountering nonavailability of the book in its print form. In our interviews, scholars report that key constraining factors to using the on-line books and other Web resources from home are the expense of dialing in to campus or maintaining an Internet account, the lack of sufficiently powerful home computers and Web software, the frequency of busy signals on the dial-up lines, and the slowness of standard modems.

Students residing on campus may have Ethernet connections to the campus network-providing both speedy and virtually free access to the on-line collection.

At the end of the 1996-97 academic year, approximately 2,300 students were registered for residence hall network connections.^[14] With the exception of the three Garland reference books, a very small share of reference collection use occurs on

computers in the libraries; the Columbia community is taking advantage of the out-of-library access to these resources. For example, 42% of the hits on The Oxford English Dictionary in the ten months following May 1996 were from residence hall network connections.

However, these undergraduates have shown little interest in the nonreference books on-line. Residence hall connections accounted for only 1% of the use of the Columbia University Press titles in social work, earth and environmental science, and international relations and 3% of the use of the Oxford University Press titles in literary criticism and philosophy from May 1996 to May 1997. These small shares are not surprising given that few of these books are aimed at the undergraduate audience. The undergraduates' use of the Past Masters classical texts in social thought from Ethernet connections in their rooms is somewhat higher-654 hits, or almost 13% of the total use of those texts from May 1996 to March 1997.

Scholars' Access to On-line Resources

We theorize that scholars with greater perceived access to networked computers and with greater familiarity with on-line resources are more likely first to sample on-line books and later to adopt them for regular use (assuming that books of interest are available on-line). All project questionnaires ask about both these factors. The access question is, "Is there a computer (in the library or elsewhere) attached to the campus network (directly or by modem) that you can use whenever you want?" The question about use of on-line resources asks, "On average this semester, how many hours per week do you spend in on-line activities (Email, Listservs & Newsgroups, CLIO Plus, Text, Image or Numeric Data Sources, Other WWWeb Uses)?" In some cases, the question asks for a single value; in others, it has five spaces in which respondents are asked to enter their hours for each of these activities.

Over 80% of Columbia library users report adequate access to a networked computer.

In the Columbia Libraries annual survey of on-site users in March 1997, 2,367 individuals responded to this question on access to networked computers. Almost 81% answered "Yes." Masters students were least likely to respond positively (67%) while the other scholarly cohorts-faculty, doctoral students, and undergraduate students-ranged from 85% to 87%. Users of science libraries were generally more likely to respond affirmatively.

Columbia library users report an average of about six hours a week in on-line activities with no significant difference across scholarly cohorts.

Even many of the survey respondents who did not claim easy access to a networked computer reported spending considerable time in on-line activities-22% spent four to six hours a week and 23% spent more than six hours a week.

Scholars' Choice among Book Formats

Scholars' patterns of using books in their various formats and their reactions to on-line books are being tracked through a variety of surveys, individual interviews, and focus groups (see Table 17.1).

One survey involves visiting a class session for which an assigned reading was in an on-line book. A question asks which format(s) of the book the student used for this assignment. Responses were distributed as shown in Table 17.8.

In 70% of the responses for fall 1996, as seen in Table 17.8, the student had used his or her own copy of the text. The next most common method was to use a friend's copy (14%). The shares for those two modes are insignificantly different in spring 1997. We are obtaining course syllabi from instructors so that, in the future, we can analyze these responses based on what portion of the book is being used in a course and whether students are expected to purchase their own copies.

Preferences for Studying Class Reading

We obtained far fewer responses (119 in fall 1996 and 88 in spring 1997) as to the preferred mode of studying. Table 17.9 shows that in both semesters, about twothirds of respondents reported a preference for reading their own copy.

Scholars' Reactions to Book Formats and Characteristics

Scholars reporting easy access to a networked computer spend more time on-line and are more likely to prefer to use one of the forms of the on-line book.

In our in-class surveys in spring 1997, students claiming easy access to a networked computer (74% of the 209 respondents) were greater users of on-line resources overall. Only 27% of students claiming easy access reported as few as one to two hours on-line a week, while 53% of those lacking easy access had this low level of on-line activity. About 31% of the former group spent six or more hours a week on-line while 18% of the latter group did.

About 26% of the easy access group gave some form of on-line book (reading directly on-line, printout of text, or download of text and reading away from the Web) as their preferred method of reading an assignment for which an on-line version was available, while only 13% of the students lacking easy access did so.

This combination of responses suggests that, over time as members of the scholarly community obtain greater access to computers linked to the Web, on-line books will achieve greater acceptance.

Students report that they particularly value easy access to the texts that are assigned for class and an ability to underline and annotate those texts.

Students seek the ability to print out all or parts of the on-line texts that they use for their courses, again indicating their desire to have the paper copy to use in their

studying. Computer access to a needed text is not equivalent to having a paper copy (whole book or assigned portion) in one's backpack, available at any time and at any place (see Table 17.10).

The cross-tabulation of preferred method of use and reasons for that preference produces logically consistent results. For example, all the respondents who gave "Printout using non-JAKE printer" or "Download of on-line text to disk to be read away from CWeb" as their preferred method gave "Less costly" as one of their reasons, while few of those students who preferred their own copy gave that reason.

If the effective choice for completing a required reading is between borrowing a book from the library, probably on a very short-term basis from reserves, and accessing the book on-line, the student is facing a parallel situation of needing to photocopy or print out the reading to obtain portable, annotative media.^[15] However, the on-line book's advantages are that it will never be checked out when the student wants to use it and that it will be accessible from a computer anywhere in the world at any time (as long as that computer has an Internet connection and a graphical Web browser).

In surveys and interviews, scholars indicate that they value the ability to do searches, to browse, and to quickly look up information in an on-line book.

They also like the ability to clip bits of the text and put them in an electronic research notes file. Willingness to browse and to read on-line for extended periods varies from person to person, but it does not seem to be widespread at this time.

Some scholars perceive gains in the productivity and quality of their work in using on-line books, particularly reference books.

Two key questions asked on all our questionnaires, other than those distributed in class, seek to determine the effect of on-line books on scholarly work:

1. In doing the type of work for which you used this book, do paper books or on-line books help you be more productive?

2. Do you find that you are able to do work of higher quality when you use paper books or on-line books?

The questionnaire offers a range of seven responses from "Much greater productivity (quality) with paper" through "No difference" to "Much greater productivity (quality) with on-line" plus "Cannot say."

As Table 17.11 shows, 52% of OED users felt that they were as productive or more productive using the on-line OED, while 39% of the users of the other on-line books felt that they are as productive or more productive using the on-line format. These responses are somewhat puzzling because the reference book most used on-line is The OED, suggesting that scholars do value it, and the CWeb version of the on-line OED provides as much if not more utility than does the print version (with the exception of being able to view neighboring entries at a

glance). Thus, one might expect the productivity rating for the on-line ODE to be higher.

The distribution of responses to the quality of work question supports the print format in general, although 47% of ODE users and 43% of the users of all the other books felt that quality was as good or better with on-line books.

Table 17.12 shows considerable correlation in the responses to these two questions-those who supported the paper version for productivity tended to support it for quality as well.

In the last part of the spring 1997 semester, 52% of the on-line book users who went to the on-line survey responded to it, but only 15% of users chose to click on the button taking them to the survey.

Designing an on-line survey that is available to the reader without overt action might enhance the response rate significantly. We are working on doing that using HTML frames on the on-line books. We are also experimenting with other methods of reaching the users of the on-line books, e.g., registration of users that will bring e-mail messages about new books in their field while also enabling us to query them about their reactions to on-line books.

Conclusions

These preliminary results of the Online Book Evaluation Project suggest that, at this early point in its development, the on-line format is finding a place in the work patterns of scholars who have had an opportunity to try it.

Interviews and focus groups substantiate the findings from the server data and surveys. Together they suggest the following about scholars' reactions to the online format:

• It is a convenient way to access information in reference books and potentially to do textual analyses in individual books or whole databases like the OED.

• Using a search function, one can quickly determine if a book or set of books addresses a topic of interest and warrants further investigation.

• It is an easy way to browse through a book to determine whether it is worth deeper exploration or whether only a small section is pertinent to one's work. If the latter is the case, it is as easy to print out that small section of the on-line book as it is to take the typical next step of photocopying that section of the paper book.

• A scholar who wants to read and annotate only a modest section of a book, say a chapter or an essay for a course assignment, will find that accessing and printing out the section from the on-line book can be quicker than doing the equivalent with a library copy of the paper book.

• Ready access from any location at any hour and not worrying about whether the book sought is on the library shelf are valued features of the on-line format.

On the other hand, if scholars want to read much or all of a book, they are likely to prefer the traditional format. If the book is core to their research or to a course, scholars are likely to prefer to own a copy. If they cannot afford such a copy, if the book is of more passing interest, or if they cannot obtain a print copy, scholars would typically prefer to retain a library copy for the duration of their interest in the book. If they cannot do so, say because the book is on reserve, scholars must decide among their options, e.g., buying their own copy or using an on-line copy, and decide which option is next preferred.

Over the duration of this project, we will continue to add books to the on-line collection and to pursue our explorations of scholars' reactions to this format. We will look for trends in the perceived accessibility of on-line books and in the desirability of this format for various uses. We will seek to measure the frequency with which scholars read such substantial portions of books borrowed from libraries that they will continue to seek library access to paper copies. In a related effort, we will assess the extent to which libraries now satisfy scholars' desires for access to such copies. If a library did not have a book in its collection in print format but did offer on-line access, a scholar would face a different trade-off between the two formats.

At the same time we will pursue our analyses of the cost and intellectual property issues involved in scholarly communication in an effort to determine whether the on-line book format can contribute to the perpetuation of research and learning and to the dissemination and preservation of knowledge.