Statement from Sage (Bad Research) (3 messages) Marcia Tuttle 01 Aug 2002 12:39 UTC

----------1
Date: Thu, 1 Aug 2002 07:57:28 -0400
From: Albert Henderson <chessNIC@compuserve.com>
Subject: Statement from Sage (Bad research)

on Wed, 31 Jul 2002 Frieda Rosenberg <friedat@EMAIL.UNC.EDU> wrote:

[snip]

> Thanks, Dan, for getting the argument "back on track."  I don't think
> that splitting hairs about abstractions does anything for arguments
> about research quality or library funding, but for what it's worth, you
> raised questions of historical interest.  The quoted Mr Price pegged the
> modern form of the scientific article at "about a century" ago, after a
> long period of resistance from scientists who felt that only monographs
> could justly cover a subject.  (New ways of doing things are always
> resisted!)  Price also noted the rise of collaborative research, which
> compensated for scientists with "less than one paper" in them (his
> words) and allowed "fractional scientists" to do research!  Donald deB.
> Beaver, a former collaborator of Price, offers this in his recent
> article, "Reflections on scientific collaboration (and its study)"
> (Scientometrics, 52:3(2001):365-77:  "Teamwork, or giant collaborations,
> represents a new paradigm for the organizational structure of
> research."  He describes how "giant teams" can now deploy great numbers
> of students who can bring in a publishable amount of data in three
> months in contrast to the five years previously required by a single
> researcher with his own student help.  If this isn't a change, I don't
> know what is...

        Sorry I couldn't wait for ILL to produce your source.

        A similar article appeared in JASIST [610-614. 2001]
        Price's (Lotka's) law can indeed break down when you
        have articles authored by 100 or more researchers and
        you limit your data to physics institutes! When applied
        to the entire academic R&D universe, however, the
        effect is insignificant. The misnamed 'new paradigm'
        is an abberation. It appears only under the most
        extreme disturbances.

        When Price described 'big science' in 1963, he wrote,
        "if we know how many papers are published in a field,
        we can compute the number of men who have written them."
        [LITTLE SCIENCE BIG SCIENCE p. 63] I really don't think
        that anything has changed since then.

        Best wishes,

Albert Henderson
Former Editor, PUBLISHING RESEARCH QUARTERLY 1994-2000
<70244.1532@compuserve.com>

----------2
Date: Thu, 1 Aug 2002 07:57:29 -0400
From: Albert Henderson <chessNIC@compuserve.com>
Subject: Re: Statement from Sage (Bad research)

on Tue, 30 Jul 2002 Peter Picerno <ppicerno@NOVA.EDU> wrote:

[snip]

> The argument "Referees are not provided with libraries that are
> comprehensive enough that they can actually check unfamiliar sources and
> verify the claims on which a particular piece of research is based ..."
> particularly caught my attention because it makes no sense at all in these
> days of lightening-swift ILL and document delivery (not to mention
> e-publishing). If a referee doesn't know about ILL and Document Delivery,
> then one wonders if they should be refereeing another author's work at all.

        The most recent studies indicate that neither
        authors nor referees bother to check sources.
        For example, a number of top medical journals ask
        their authors to interpret data "in light of the
        totality of the available evidence."  They don't,
        of course. Moreover, the referees let the authors
        get away with it. [J A M A 276:637-639. 1996]

        That observation barely hints at the cost of poor
        preparation. Last year, a Johns Hopkins volunteer
        died. Of the inquiry that followed, the Chronicle
        of Higher Education wrote, "In particular, the
        office noted that researchers had "failed to obtain
        published literature about the known association
        between hexamethonium and lung toxicity" and that
        the substance was not currently approved by the
        U.S. Food and Drug Administration for use in humans."
        [47(47):A25. 2001]. Some medical librarians have
        noted this information could be located on Google!
        Why isn't a medical librarian part of the research
        team that prepared and reviewed the proposal?! My
        impression was that the author probably relied on
        one of the johnny-come-lately databases (that
        covers only the last few years) and the reviewers
        rubber-stamped the proposal. In short, having a
        great collection and all the latest techno-gizmos
        is no good if authors and referees don't use them.

        A 1994 General Accounting Office study of peer
        review as applied to grant proposals indicated,
        "Although most reviewers reported expertise in
        the general areas of the proposals they reviewed,
        many were not expert on closely related questions
        and could cite only a few, if any, references.
        This lack of proximate expertise was most pronounced
        at NIH. However, although this raises questions about
        the relative adequacy of NIH reviews and ratings, the
        greater proximity of NSF reviewers makes them
        potentially more vulnerable to apparent or actual
        self-interest in their reviews." GAO/PEMD-94-1.

        Of course the greatest conflict of interest is to
        be found in the management of institutions that
        prepare the proposals. They have the most to lose
        from a peer review bottleneck of detailed criticism
        that would send proposals back to the drawing board.
        It is to the benefit of institutional cash flow if
        authors and referees break the rules like so many
        Nancy Drews.

        I doubt that a few technological innovations
        will change human nature. What is needed is
        a financial reform of research preparation to
        provide (a) referees with more time to prepare
        critical reviews, (b) richer information
        resources, and (c) better management of authorship
        and peer review to engage specialists when
        appropriate.

> Besides, the quoted statment implies that the purpose of libraries is to
> acquire 'unafamiliar sources' so that the occasional referee can wander in
> and check a reference.

        I will make it clear: The purpose of libraries
        is to support learning. A major purpose of
        peer review is to educate the referees. To these
        ends, it would make sense if every referee's
        library made it possible to check all authors'
        sources and if review management made checking
        sources mandatory.

>                          If that, indeed, is their purpose, I'm afraid that
> university administrations would be quite justified in slashing budgets and
> personnel. Rather than being a morgue for little-used and little-demanded
> information, most academic libraries strive to be a lively place where the
> majority of its users information needs are met.

        Unfortunately, when their information needs are
        not met, users write off the library as a resource
        [as Michael K Buckland observed in BOOK AVAILABILITY
        AND THE LIBRARY USER. Pergamon 1975]. The lively
        'Starbucks' approach may address some social needs,
        but advanced study requires rich collections and
        convenient hours.

        Best wishes,

Albert Henderson
Former Editor, PUBLISHING RESEARCH QUARTERLY 1994-2000
<70244.1532@compuserve.com>

----------3
Date: Thu, 1 Aug 2002 07:57:27 -0400
From: Albert Henderson <chessNIC@compuserve.com>
Subject: Re: Statement from Sage (Bad research)

on Wed, 31 Jul 2002 Dan Lester <dan@riverofdata.com> wrote:

[snip]

> As soon as you have a magic answer on how to get increased R&D funding
> into the library, be sure to let us know.  You could sell libraries
> the secret and retire a wealthy man.  It certainly isn't for lack of
> trying by librarians that most of us have been unsuccessful in getting
> significant chunks of the R&D dollars for the library.

        Looking for a magic idea? Herbert S White had one.
        In his White Papers column, he observed that in the
        competition for budget dollars, the failures of a
        library are more important than markers of
        satisfaction. (For some reason, librarians would
        rather emphasize the latter.) Crime statistics, for
        example, measure the inadequacy of law and order. The
        FBI's annual publicity makes headlines and propels
        police chiefs' annual budget requests through the
        approval process. Perhaps failures of library
        collections could support libraries' budget requests.
        [LJ 120,1 (Jan., 1995): 58, 60.] In other words,
        librarians and libraries would fare better by
        emphasizing the numbers of information requests that
        could _not_ be satisfied. I have offered a 'Collection
        Failure Quotient' -- the ratio of interlibrary
        borrowing and collection size [access v. ownership] --
        for this purpose. [Journal of Academic Librarianship.
        26,3:159-170. 2000.] There are other approaches
        to measure dissatisfaction including interlibrary
        borrowing failures, patron interviews, etc. Harold
        Varmus complained to Congress that many researchers
        used grant money to subscribe to journals [which
        should be available in the library].  A census of
        private subscriptions would be an interesting
        indicator of dissatisfaction.

        Speaking mainly about the expensive science serials
        that support sponsored R&D, another magic answer
        should lie in research overhead. Overhead accounts
        for 1/3 of $30 billion federally sponsored academic
        research spending (2000). Libraries are designated as
        an overhead factor. How much of the $10 billion
        overhead reimbursements supports libraries? It
        shouldn't be too hard to identify the costs
        associated with those expensive science journals.

        The definition of overhead expenses to be covered by
        reimbursements was originally agreed as "full
        accountable costs."

        If library overhead were handled in ways that
        actually reflected the ways that libraries are
        used by the researchers who prepare, review, and
        execute sponsored research, I believe library
        funding would be better.

        Unfortunately, library reimbursements are
        negotiated by financial managers, not by
        librarians. As a result, library reimbursements to
        go administrative slush funds, not to libraries.

        Worse, the money can be lost due to bureaucratic
        incompetence. The Inspector General of the National
        Science Board found examples costs that the General
        Accounting Office identified as unallowable or
        questionable. One example given was $7 million in
        library costs claimed by Stanford because the
        university "did not use the default method specified
        by OMB Circular A-21." [Federally Sponsored Research:
        How Indirect Costs are Charged by Educational and
        Other Research Institutions.]

> AH>         The SERIALS PRICES PROJECT REPORT of the Association
> AH>         of Research Libraries (1989) made 'excessive publication'
> AH>         a leading factor in its propaganda campaign of the early
> AH>         1990s. The theme was amplified by SCIENCE, THE SCIENTIST,
> AH>         60 MINUTES, and THE NEW YORK TIMES, whose editors never
> AH>         bothered to check the reliability of the ARL as a
> AH>         objective source.
>
> As always, one man's "objective source" is another man's "biased
> source".  We all have our own agendas, and we're certainly familiar
> with yours.  I believe, Mr. Henderson, that we'd all be able to work
> together for a common goal if you weren't so busy biting the hand that
> feeds you.  I don't know of a single librarian that doesn't feel the
> need for more funding for materials of all types, and the staff to
> support their acquisition, storage, and access.  I also don't know of
> a single librarian that doesn't regularly make pleas to the university
> administration for greater funding and the reasons therefor.

        First, I am not biting the hand that feeds me. I
        have campaigned for better library funding for
        quite a long time now. What puzzles me is the
        number of contra librarians.

        To address your point, can you can tell me the
        official position on library overhead of the
        Association of Research Libraries, the ACRL,
        or the American Library Association?

        Moreover, can you tell me why the ACRL standards
        for college and university libraries no long offer
        anything in the way of objective measures by which
        to gauge whether a library is acceptable or not.

> Just because librarians are taking advantage of new technologies to
> obtain materials that researchers (and others) request doesn't mean
> that if it were "the old world" instead of "the new world" we wouldn't
> love to have more shelves filled with these items.
>
> AH>         The same sort of peer review that serves editors
> AH>         supports approvals of academic research grants now
> AH>         in the tens of billions of dollars with huge
> AH>         overhead allowances going to profitability.
>
> AH>         It is pitiful.
>
> I know you're really convinced of this "profitability" in academia.
> Profitability in the business world can produce fortunes for top
> executives and profits for shareholders, as well as income for the
> employees.
>
        I credit Thorsten Veblen with identifying the
        problem. He pointed out that the university "is a
        corporation with large funds, and for men biased by
        their workaday training in business affairs it comes
        as a matter of course to rate the university in terms
        of investment and turnover." [The Higher Learning in
        America. 1918; reprint 1993 p. 62] Similar
        observations were set out by Robert Nisbet and Edward
        Shils. I have gone to financial statements and
        statistics for evidence (easily found). Clearly
        knowledge has lost priority to money.

> If that profitability were present in academia, the same should hold
> true for the university.  Those of us who are employees get income for
> doing our job.  There are no shareholders as such.  The top university
> administrators certainly make six figure salaries, but I've not read
> of any of them being taken away in handcuffs because they've diverted
> funds to their million dollar mansions, bought any private jets, or
> had interest free loans of tens of millions of dollars.

        See my article in SCHOLARLY PUBLISHING [Wiley 2002.
        p 8] for profit and spending trends 1970-1995.

> AH>         I have made a point of the ratio of
> AH>         interlibrary borrowing to total numbers of volumes,
> AH>         something that I call COLLECTION FAILURE QUOTIENT,
> AH>         but very little about acquisitions spending.
>
> That number will continue to increase as the amount of publishing
> increases, and as the prices of those publication continue to increase
> faster than almost any other component of the economy.

        Publishing activity increases with the growth
        of spending on academic R&D. Why doesn't library
        spending keep up? Are libraries part of research,
        or not?!

Albert Henderson
Former Editor, PUBLISHING RESEARCH QUARTERLY 1994-2000
<70244.1532@compuserve.com>