Statement from Sage (Bad Research) (3 messages) Marcia Tuttle 01 Aug 2002 12:39 UTC
----------1
Date: Thu, 1 Aug 2002 07:57:28 -0400
From: Albert Henderson <chessNIC@compuserve.com>
Subject: Statement from Sage (Bad research)
on Wed, 31 Jul 2002 Frieda Rosenberg <friedat@EMAIL.UNC.EDU> wrote:
[snip]
> Thanks, Dan, for getting the argument "back on track." I don't think
> that splitting hairs about abstractions does anything for arguments
> about research quality or library funding, but for what it's worth, you
> raised questions of historical interest. The quoted Mr Price pegged the
> modern form of the scientific article at "about a century" ago, after a
> long period of resistance from scientists who felt that only monographs
> could justly cover a subject. (New ways of doing things are always
> resisted!) Price also noted the rise of collaborative research, which
> compensated for scientists with "less than one paper" in them (his
> words) and allowed "fractional scientists" to do research! Donald deB.
> Beaver, a former collaborator of Price, offers this in his recent
> article, "Reflections on scientific collaboration (and its study)"
> (Scientometrics, 52:3(2001):365-77: "Teamwork, or giant collaborations,
> represents a new paradigm for the organizational structure of
> research." He describes how "giant teams" can now deploy great numbers
> of students who can bring in a publishable amount of data in three
> months in contrast to the five years previously required by a single
> researcher with his own student help. If this isn't a change, I don't
> know what is...
Sorry I couldn't wait for ILL to produce your source.
A similar article appeared in JASIST [610-614. 2001]
Price's (Lotka's) law can indeed break down when you
have articles authored by 100 or more researchers and
you limit your data to physics institutes! When applied
to the entire academic R&D universe, however, the
effect is insignificant. The misnamed 'new paradigm'
is an abberation. It appears only under the most
extreme disturbances.
When Price described 'big science' in 1963, he wrote,
"if we know how many papers are published in a field,
we can compute the number of men who have written them."
[LITTLE SCIENCE BIG SCIENCE p. 63] I really don't think
that anything has changed since then.
Best wishes,
Albert Henderson
Former Editor, PUBLISHING RESEARCH QUARTERLY 1994-2000
<70244.1532@compuserve.com>
----------2
Date: Thu, 1 Aug 2002 07:57:29 -0400
From: Albert Henderson <chessNIC@compuserve.com>
Subject: Re: Statement from Sage (Bad research)
on Tue, 30 Jul 2002 Peter Picerno <ppicerno@NOVA.EDU> wrote:
[snip]
> The argument "Referees are not provided with libraries that are
> comprehensive enough that they can actually check unfamiliar sources and
> verify the claims on which a particular piece of research is based ..."
> particularly caught my attention because it makes no sense at all in these
> days of lightening-swift ILL and document delivery (not to mention
> e-publishing). If a referee doesn't know about ILL and Document Delivery,
> then one wonders if they should be refereeing another author's work at all.
The most recent studies indicate that neither
authors nor referees bother to check sources.
For example, a number of top medical journals ask
their authors to interpret data "in light of the
totality of the available evidence." They don't,
of course. Moreover, the referees let the authors
get away with it. [J A M A 276:637-639. 1996]
That observation barely hints at the cost of poor
preparation. Last year, a Johns Hopkins volunteer
died. Of the inquiry that followed, the Chronicle
of Higher Education wrote, "In particular, the
office noted that researchers had "failed to obtain
published literature about the known association
between hexamethonium and lung toxicity" and that
the substance was not currently approved by the
U.S. Food and Drug Administration for use in humans."
[47(47):A25. 2001]. Some medical librarians have
noted this information could be located on Google!
Why isn't a medical librarian part of the research
team that prepared and reviewed the proposal?! My
impression was that the author probably relied on
one of the johnny-come-lately databases (that
covers only the last few years) and the reviewers
rubber-stamped the proposal. In short, having a
great collection and all the latest techno-gizmos
is no good if authors and referees don't use them.
A 1994 General Accounting Office study of peer
review as applied to grant proposals indicated,
"Although most reviewers reported expertise in
the general areas of the proposals they reviewed,
many were not expert on closely related questions
and could cite only a few, if any, references.
This lack of proximate expertise was most pronounced
at NIH. However, although this raises questions about
the relative adequacy of NIH reviews and ratings, the
greater proximity of NSF reviewers makes them
potentially more vulnerable to apparent or actual
self-interest in their reviews." GAO/PEMD-94-1.
Of course the greatest conflict of interest is to
be found in the management of institutions that
prepare the proposals. They have the most to lose
from a peer review bottleneck of detailed criticism
that would send proposals back to the drawing board.
It is to the benefit of institutional cash flow if
authors and referees break the rules like so many
Nancy Drews.
I doubt that a few technological innovations
will change human nature. What is needed is
a financial reform of research preparation to
provide (a) referees with more time to prepare
critical reviews, (b) richer information
resources, and (c) better management of authorship
and peer review to engage specialists when
appropriate.
> Besides, the quoted statment implies that the purpose of libraries is to
> acquire 'unafamiliar sources' so that the occasional referee can wander in
> and check a reference.
I will make it clear: The purpose of libraries
is to support learning. A major purpose of
peer review is to educate the referees. To these
ends, it would make sense if every referee's
library made it possible to check all authors'
sources and if review management made checking
sources mandatory.
> If that, indeed, is their purpose, I'm afraid that
> university administrations would be quite justified in slashing budgets and
> personnel. Rather than being a morgue for little-used and little-demanded
> information, most academic libraries strive to be a lively place where the
> majority of its users information needs are met.
Unfortunately, when their information needs are
not met, users write off the library as a resource
[as Michael K Buckland observed in BOOK AVAILABILITY
AND THE LIBRARY USER. Pergamon 1975]. The lively
'Starbucks' approach may address some social needs,
but advanced study requires rich collections and
convenient hours.
Best wishes,
Albert Henderson
Former Editor, PUBLISHING RESEARCH QUARTERLY 1994-2000
<70244.1532@compuserve.com>
----------3
Date: Thu, 1 Aug 2002 07:57:27 -0400
From: Albert Henderson <chessNIC@compuserve.com>
Subject: Re: Statement from Sage (Bad research)
on Wed, 31 Jul 2002 Dan Lester <dan@riverofdata.com> wrote:
[snip]
> As soon as you have a magic answer on how to get increased R&D funding
> into the library, be sure to let us know. You could sell libraries
> the secret and retire a wealthy man. It certainly isn't for lack of
> trying by librarians that most of us have been unsuccessful in getting
> significant chunks of the R&D dollars for the library.
Looking for a magic idea? Herbert S White had one.
In his White Papers column, he observed that in the
competition for budget dollars, the failures of a
library are more important than markers of
satisfaction. (For some reason, librarians would
rather emphasize the latter.) Crime statistics, for
example, measure the inadequacy of law and order. The
FBI's annual publicity makes headlines and propels
police chiefs' annual budget requests through the
approval process. Perhaps failures of library
collections could support libraries' budget requests.
[LJ 120,1 (Jan., 1995): 58, 60.] In other words,
librarians and libraries would fare better by
emphasizing the numbers of information requests that
could _not_ be satisfied. I have offered a 'Collection
Failure Quotient' -- the ratio of interlibrary
borrowing and collection size [access v. ownership] --
for this purpose. [Journal of Academic Librarianship.
26,3:159-170. 2000.] There are other approaches
to measure dissatisfaction including interlibrary
borrowing failures, patron interviews, etc. Harold
Varmus complained to Congress that many researchers
used grant money to subscribe to journals [which
should be available in the library]. A census of
private subscriptions would be an interesting
indicator of dissatisfaction.
Speaking mainly about the expensive science serials
that support sponsored R&D, another magic answer
should lie in research overhead. Overhead accounts
for 1/3 of $30 billion federally sponsored academic
research spending (2000). Libraries are designated as
an overhead factor. How much of the $10 billion
overhead reimbursements supports libraries? It
shouldn't be too hard to identify the costs
associated with those expensive science journals.
The definition of overhead expenses to be covered by
reimbursements was originally agreed as "full
accountable costs."
If library overhead were handled in ways that
actually reflected the ways that libraries are
used by the researchers who prepare, review, and
execute sponsored research, I believe library
funding would be better.
Unfortunately, library reimbursements are
negotiated by financial managers, not by
librarians. As a result, library reimbursements to
go administrative slush funds, not to libraries.
Worse, the money can be lost due to bureaucratic
incompetence. The Inspector General of the National
Science Board found examples costs that the General
Accounting Office identified as unallowable or
questionable. One example given was $7 million in
library costs claimed by Stanford because the
university "did not use the default method specified
by OMB Circular A-21." [Federally Sponsored Research:
How Indirect Costs are Charged by Educational and
Other Research Institutions.]
> AH> The SERIALS PRICES PROJECT REPORT of the Association
> AH> of Research Libraries (1989) made 'excessive publication'
> AH> a leading factor in its propaganda campaign of the early
> AH> 1990s. The theme was amplified by SCIENCE, THE SCIENTIST,
> AH> 60 MINUTES, and THE NEW YORK TIMES, whose editors never
> AH> bothered to check the reliability of the ARL as a
> AH> objective source.
>
> As always, one man's "objective source" is another man's "biased
> source". We all have our own agendas, and we're certainly familiar
> with yours. I believe, Mr. Henderson, that we'd all be able to work
> together for a common goal if you weren't so busy biting the hand that
> feeds you. I don't know of a single librarian that doesn't feel the
> need for more funding for materials of all types, and the staff to
> support their acquisition, storage, and access. I also don't know of
> a single librarian that doesn't regularly make pleas to the university
> administration for greater funding and the reasons therefor.
First, I am not biting the hand that feeds me. I
have campaigned for better library funding for
quite a long time now. What puzzles me is the
number of contra librarians.
To address your point, can you can tell me the
official position on library overhead of the
Association of Research Libraries, the ACRL,
or the American Library Association?
Moreover, can you tell me why the ACRL standards
for college and university libraries no long offer
anything in the way of objective measures by which
to gauge whether a library is acceptable or not.
> Just because librarians are taking advantage of new technologies to
> obtain materials that researchers (and others) request doesn't mean
> that if it were "the old world" instead of "the new world" we wouldn't
> love to have more shelves filled with these items.
>
> AH> The same sort of peer review that serves editors
> AH> supports approvals of academic research grants now
> AH> in the tens of billions of dollars with huge
> AH> overhead allowances going to profitability.
>
> AH> It is pitiful.
>
> I know you're really convinced of this "profitability" in academia.
> Profitability in the business world can produce fortunes for top
> executives and profits for shareholders, as well as income for the
> employees.
>
I credit Thorsten Veblen with identifying the
problem. He pointed out that the university "is a
corporation with large funds, and for men biased by
their workaday training in business affairs it comes
as a matter of course to rate the university in terms
of investment and turnover." [The Higher Learning in
America. 1918; reprint 1993 p. 62] Similar
observations were set out by Robert Nisbet and Edward
Shils. I have gone to financial statements and
statistics for evidence (easily found). Clearly
knowledge has lost priority to money.
> If that profitability were present in academia, the same should hold
> true for the university. Those of us who are employees get income for
> doing our job. There are no shareholders as such. The top university
> administrators certainly make six figure salaries, but I've not read
> of any of them being taken away in handcuffs because they've diverted
> funds to their million dollar mansions, bought any private jets, or
> had interest free loans of tens of millions of dollars.
See my article in SCHOLARLY PUBLISHING [Wiley 2002.
p 8] for profit and spending trends 1970-1995.
> AH> I have made a point of the ratio of
> AH> interlibrary borrowing to total numbers of volumes,
> AH> something that I call COLLECTION FAILURE QUOTIENT,
> AH> but very little about acquisitions spending.
>
> That number will continue to increase as the amount of publishing
> increases, and as the prices of those publication continue to increase
> faster than almost any other component of the economy.
Publishing activity increases with the growth
of spending on academic R&D. Why doesn't library
spending keep up? Are libraries part of research,
or not?!
Albert Henderson
Former Editor, PUBLISHING RESEARCH QUARTERLY 1994-2000
<70244.1532@compuserve.com>