Contributory inequality alters assessment of academic output gap between comparable countries

ContentslistsavailableatScienceDirect

Journal of Informetrics

jo u r n al hom e p ag e :w w w . e l s e v i e r . c o m / l o c a t e / j o i

Contributory inequality alters assessment of academic output gap between comparable countries

Nils T. Hagen

FacultyofBiosciencesandAquaculture,UniversityofNordland,N-8049Bodø,Norway

a rt i c l e i n f o

Articlehistory:

Received21February2015

Receivedinrevisedform29May2015 Accepted1June2015

Keywords:

Coauthorcredit Harmonicformula Bibliometricbias Publicationmetrics Inequality

a b s t ra c t

AnelitesegmentoftheacademicoutputgapbetweenDenmarkandNorwaywasexam- inedusingharmonicestimatesofpublicationcreditforcontributionstoScienceandNature in2012and2013.Denmarkstillleadsbutthegapnarrowedin2013asNorway’scredit increased58%,whileDenmark’screditincreasedonly5.4%,eventhoughNorwayhad36%

fewer,andDenmark40%more,coauthorcontributionsthanin2012.Concurrently,the creditproducedbytheleastproductivehalfofthecontributionsrosetenfoldfrom0.9%

to10.1%forNorway,butdroppedfrom7.2%to5.7%forDenmark.Overall,contributory inequalityasmeasuredbytheGinicoefficient, fellfrom0.78to0.51forNorway,but rosefrom0.63to0.68forDenmark.Neithergapnarrowingnorthepositiveassociation betweenreducedcontributoryinequalityandincreasedcreditweredetectedbyconven- tionalmetrics.Conventionalmetricsareconfoundedbyequalizingbias(EqB)whichfavours smallcontributorsattheexpenseoflargecontributors,andwhichcarriesanelementof reversemeritocracyandsystemicinjusticeintobibliometricperformanceassessment.EqB wascorrectedbyusingallrelevantbylineinformationfromeverycoauthoredpublica- tioninthesourcedata.ThisapproachdemonstratesthefeasibilityofusingEqB-corrected publicationcreditingapassessmentatthenationallevel.

©2015TheAuthor.PublishedbyElsevierLtd.ThisisanopenaccessarticleundertheCC BY-NC-NDlicense(http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction

Themainobjectiveofthisworkistouserecentadvancesinbibliometriccreditallocationtogainnewinsightsintothe academicoutputgapbetweentwocomparablecountries,inthiscaseDenmarkandNorway.Previousstudiesinvariably ratedtheacademicoutputofDenmarkaboveNorway(e.g.Glänzel,2000;ResearchCouncilofNorway,2014;Schneider, 2010;vanLeeuwen,2012),butallgapsizeestimatesbasedonconventionalbibliometricmethodsareinaccuratebecause theydonotadequatelyaccountforthesizeofeachcountry’scontributiontointernationallycoauthoredpublications.

Almost200countriescontributetotheglobalproductionofacademicpublicationsandtheoutputofmanyischanging rapidly(NationalScienceBoard,2014).Everyone,therefore,wantstoknowhowwelltheyaredoinginrelationtoothers andtheirsharedconcernistoavoidbeingmisguidedbyinaccurateinformation.Suchconcernoverperceivedacademic outputgapsbetweencountrieshasfuelledresearchpolicydebatefornearlytwocenturies,andtheconceptofacademic productivityasaninput/outputratiohasevolvedinacontextofinternationalcomparison(Godin,2006,2009;Nowotny, 2007).AnearlyexampleisCharlesBabbage’sconcernaboutthedeclineofscienceinEnglandrelativetoGermany(Babbage, 1830;Foreigner,1931);aconcernquantifiedbyEdwardFranklandin1871(Devonshire,1872,p.371:5866)(cf.Braun,1993;

E-mailaddress:[email protected]

http://dx.doi.org/10.1016/j.joi.2015.06.002

1751-1577/©2015TheAuthor.PublishedbyElsevierLtd.ThisisanopenaccessarticleundertheCCBY-NC-NDlicense(http://creativecommons.org/

licenses/by-nc-nd/4.0/).

Cardwell,1972;Nye,1984),andechoedinthemoderndebateaboutthesametopic150yearslater(Martin,1994).Ona broaderscale,thedynamicnatureofinternationalsciencetodayisreflectedincurrenttrendssuggestingthattheacademic pre-eminenceoftheUSmaybesurpassedbyChinaintheforeseeablefuture(Leydesdorff,2012).

Theglobalgrowthinacademicoutputhasbeenaccompaniedbyanunabatingincreaseininternationalcollaboration (Aksnes,Frølich,&Slipersæter,2008;Leydesdorff&Wagner,2008).Morethanaquarterofallpublicationsintheworld areproducedbymulti-nationalteams,andseveralsmallercountriesproducemorethanhalfoftheirresearchpapersin collaborationwithinternationalpartners(RoyalSocietyUK,2011).Collaborationhasmanybenefits,isoftenencouraged bypolicymakersandfundingagencies,andisconsideredessentialforgroundbreakingresearchwheretherequiredeffort isbeyondthecapacityofasinglenation(Bidault&Hildebrand,2014;Sonnenwald,2007).Participationininternational top-levelresearchisthereforeregardedasanindicationofnationalcompetitiveabilityandacademicachievement.

Butinconventionalmeasuresofacademicoutput,collaborationisamajorsourceofinaccuracy.Theroutineapproachis toinflatepublicationcountsbyissuingfullpublicationcredittoeverycountryincludedinthelistofauthoraffiliations.The otherconventionalapproachistodivideoneunitofcreditequallyamongapaper’scoauthors,andthentallythefractions foreachcountry.Suchfractionalpublicationcountingcorrectsforinflationarybias,butnotfortheequalizingbias(EqB) whichistheinevitableconsequencewheneverthecoauthorsofapaperhavenotcontributedequally.

EqBskewsbibliometricassessmentsandaccountsforamassiveshiftofcreditfromprimarytosecondaryauthors.Asa result,biasedequalcreditscoresproducedistortedpublicationperformancerankingsthatarefundamentallydifferentfrom rankingsobtainedfromestimatesofactualcoauthorcredit(Hagen,2014a).Furthermore,thepowerfuldistortionaleffectof EqBisinevitablycompoundedinderivedbibliometricindicesandindicators.EqBmayalsoprovideanincentiveforunethical behaviour,includingunwarrantedclaimsforhonoraryauthorshiporgiftauthorship.

Thekeytomorereliablepublicationcountingistoensureaccurateaccreditationofcoauthorsbyincludingallrelevant bylineinformation.Thisbottom-upapproachisfacilitatedbytheharmonicformula,whichprovidesequitabledistributionof coauthorcreditforscientificpaperswithahierarchicalbylinestructure(Hagen,2008,2013).Italsoaccommodatesadditional bylineinformationwhich,forinstance,mayindicatetheequalityofsomeorallcoauthors,orthepresenceofaseniorauthor.

RecentstudiesoffieldspecificpublicationpatternshaveusedtheharmonicformulatopartiallyeliminateEqB(Fernandes, 2014;Walters&Wilder,2015),butfuturestudiesmustalsoincludeadditionalbylineinformationaboutequalityorseniority inanefforttocompletelyeliminateEqB.

Here,Iuseanevidence-basedinformetricapproachtoestimatetheeffectofinternationalacademiccollaborationon measurementsofpublicationoutputfortwocomparablenations,DenmarkandNorway.First,Iaddresstheneedforimproved accuracybyportrayinghowthecombinationofincreasedoutputandincreasedinternationalcollaborationoverthepast fourdecadeshasgeneratedawideningzoneofoverlapbetweentheupperandlowerboundariesofthetwocountries’total publicationoutput.Second,Iprovideaclose-upoftheacademicoutputgapbetweenDenmarkandNorway,byanalyzing indetailtheirscientificcontributionstothetwotop-tierjournalsScienceandNaturefortheyears2012and2013.Third,I quantifytheinequalityprofileofeachcountry’scontributions,andprovidenewinformationabouttherelationshipbetween contributoryinequalityandcumulativetop-leveloutput.Andfinally,IconcludebydemonstratinghowEqBalteredthe perceiveddirectionofannualchangeinthetop-levelacademicoutputgap.

2. Background:accountingforcoauthorshipinpublicationcounting

TheuseofpublicationcountsasaquantitativebaseforresearchpolicywaspioneeredbyFrankland’stestimonytothe BritishRoyalCommissionin1871,whenheusedpublicationcountsfortheyear1866toprovideaquantitativeassessmentof thealledgedacademicperformancegapwhichshowedBritainlaggingbehindGermany(Cardwell,1972;Devonshire,1872, p.371:5866).Atthetimecoauthorshipwasnotaconfoundingissuebutinaccuracyduetointernationalcollaborationhad alreadyenteredthepicture.ItwasFranklandwhopointedoutthatthecountforBritainunderestimatedthegapbecauseit includedpublicationsfromGermanscientistsresidinginBritainwhohadreceivedtheirtraininginGermany.

2.1. Inflatedpublicationcounting

Inflatedpublicationcountingwasinitiatedin1917,whenS.I.Franz(Franz,1917,p.202,footnote1)decidedtoassignfull valuepublicationcredittobothindividualsofajointpublication(cf.Godin,2006).Thisapproachprovidedastrongincen- tiveforjointauthorshipwhichexpandedseamlesslytomultipleauthorship.Inflatedcountinghasdominatedquantitative researchoutputanalysiseversince.

Butinflatedpublicationcountingisconfoundedbytwoseparatesourcesofbias.Inflationarybias,generatedbyissuing onefullunitofcreditrepeatedlytoeachcoauthororparticipatingcountry;andequalizingbias(EqB),generatedbyignoring differentialcontribution(Hagen,2008).

2.2. Fractionalpublicationcounting

Nearly50yearsago,asmultipleauthorshipbecameincreasinglycommon,PriceandBeaver(1966)introducedthepractice offractionalcountinginaninfluentialpaperwhichsetalongstanding,unintendedprecedentforusingfractionalcountingin

conjunctionandcomparisonwithinflatedcounting(e.g.Gauffriau,Larsen,Maye,Roulin-Perriard,&vonIns,2008;Martin, 1994;NatureIndex,2014a).

Wedefinefractionalproductivityasthescoreofanauthorwhenheisassigned1/nofapointfortheoccurrenceofhisname amongnauthorsontheby-lineofasinglepaper.Thusamanwithonepaperofwhichheisthesoleauthor,asecondof whichheisoneoftwoauthors,andathirdinwhichheisoneoffive,willhaveafractionalproductivityof1.7andafull productivityofthreepapers.(Price&Beaver,1966,p.1014–1015)

Despitesomesupport(Lindsey,1980;Narin,1976),fractionalcountingwasnotanimmediatesuccess,and15yearslater Pricelamentedthat:

...publicationsorcitationsmustbedividedamongalltheauthorslistedonthebyline,andintheabsenceofevidenceto thecontraryitmustbedividedequallyamongthem.Thuseachauthorofathree-authorpapergetscreditforone-third ofapublicationandone-thirdoftheensuingcitations....Anytimeyoutakeacollaboratoryoumustgiveupashareof theoutcome,andyoudiminishyourownshare.Thatisasitshouldbe;todootherwiseisaverycheapwayofincreasing apparentproductivity.(Price,1981)

Latelyfractionalcountinghasfoundpracticalapplicationasaproxymeasureforcoauthorcreditinmetrics-basednational researchfundingsystemsinNorway,Denmark,andelsewhere(Hicks,2012).Thisiscauseforconcernaslessthan4%ofall coauthoredresearchpapersintentionallychoosetoindicateequalcontributionbylistingthecoauthors’namesinalphabet- icalorder(Waltman,2012).Theimplicationisthatover96%ofallcontemporarypapersaresusceptibletoconfoundingby equalizingbias(EqB)whenpublicationcreditiscountedfractionally.

TheimmediateeffectofEqBistooverestimatethecreditofsmallcontributionsandunderestimatethecreditoflarge contributions(Hagen,2008).EqBalsoprecludesassessmentoftheeffectofcontributoryinequalityoncumulativepublication output.Fractionalcountingmaythereforemisrepresentthepublicationperformanceofcomparablenationsbyexaggerating orconcealingthegapbetweenthem.Eitherway,thereisariskthatinaccurateestimatesofpublicationperformancemay inspiremisguidedandcounterproductiveinitiativeswithunanticipatedundesirableoutcomesforthescientificcommunity.

2.3. Harmonicestimatesofactualpublicationcredit

Theharmonicformula,althoughnotnamedassuchatthetime,wasfirstproposedbyHodgeandGreenberg(1981)as aresponsetoPrice’s(1981)pleafordivisionofpublicationcreditamongalltheauthorslistedonthebyline.Unfortunately forgottenuntil2008(Hagen,2008,2009),ithassincebeenvalidatedagainstempiricaldataandhasoutperformedother creditallocationschemesfromthebibliometricliterature(Hagen,2010,2013).

Theharmonicformulaowesitstransparencytothreesimpleethicalcriteria(Hagen,2008,2010):

1.Onepublicationcreditissharedamongallcoauthors

2.Thefirstauthorgetsthemostcredit,andingeneraltheithauthorreceivesmorecreditthanthe(ith+1)thauthor 3.Thegreaterthenumberofauthors,thelesscreditperauthor.

Onerecentstudyfoundthattheharmonicformulaexplainednearly97%ofthevariationinacompositeempiricaldataset composedof3independentsetsofdatafromthescientificsubfieldsofchemistry,medicineandpsychology(Hagen,2013);

thatis,acoefficientofdeterminationR²=0.9676wasobtainedforthelinearregressionbetweenvaluespredictedbythe harmonicformulaandtheempiricaldatapoints.Incontrast,fractionalcreditexplainedlessthan40%ofthevariation,with EqBaccountingfortheroughly60%discrepancybetweenthetwofigures.Afollow-uptheoreticalanalysisshowedthatthe amountofEqBincreasesasthenumberofcoauthorsincreases,andsuggestedthatfractionalcreditscoresarelikelytobe dominatedbyEqBwhenthenumberofcoauthorsexceeds12(Hagen,2014b).

Inshort,theharmonicformulamakesitpossibletoassesstherelativecontributionofcollaboratorsbyprovidingaccurate, parsimoniousandverifiablequantificationofhierarchicalbylines.Theprospectofimprovedaccuracyhasimplicationsfor applicationscurrentlyafflictedbyEqB,includingassessmentoftheeffectofinternationalcoauthorshiponthepublication gapsbetweennationstates.Italsomakesitfeasibletostudypreviouslyunexploredaspectsofpublicationdatainunprece- denteddetail,forexample,bydocumentingtherelationshipbetweencontributoryinequalityandcumulativepublication output.

2.4. Top-downversusbottom-upallocationofpublicationcredit

Proxyindicators,suchasinflatedandfractionalcounting,quantifypublicationcreditaccordingtoatopdownapproach whichallocatesthesameamountofcredittoallcoauthorsirrespectiveoftheiractualcontribution.Theadvantageisthatauto- matedcreditallocationisunproblematic.Butthedisadvantageisthatthecreditofallcoauthorswhohavenotcontributed equallyisseverelybiasedbyEqB(Hagen,2014a).

In contrast,a bottom-upapproachusesallrelevant evidenceinthesourcedatatoprovideanaccurateinformetric estimateofactualpublicationcreditforallcoauthors.Credit,therefore,isnotallocatedequallyunlessthereisevidence indicatingequalcontribution.Whenthereisnoevidenceofequalcontribution,creditmustbeallocatedaccordingtothe

availableinformation.Suchinformationusuallyconsistsofahierarchicalbylinestructurebutmayalsoincludeadditional informationindicatingspecificdeparturesfromthehierarchy,suchastwoormoreequalcoauthors,orthepresenceofa seniorauthor.Allocationofhierarchicalinformationiseasilyautomatedusingtheharmonicformula(e.g.Walters&Wilder, 2015).

Advantagesofthebottom-upapproachincludeincreasedaccuracy,transparency,reproducibilityandcredibility.How- ever,afullimplementationofautomatedbottom-upallocationisstillhamperedbynon-standardisedcodingofrelevant additionalinformation,bythepersistenceofincomplete,inaccurateorambiguousbylineinformation,andbytheongoing evolutionoflargerandmorecomplexauthorcollaborations.Toresolvethesechallengescompletelywillrequireaconcerted multidisciplinaryeffortbyallinvolvedstakeholders.Meanwhile,thepreferredremedyistoencourageauthorstostatetheir contributionsmoreexplicitly,bymakingeveryefforttorespecttheinformationtheydoprovide.

3. Materialandmethods

3.1. Conventionalpublicationcountsandtheprevalenceofinternationalcollaboration1973–2013

PublicationcountswereobtainedbysearchingtheWebofScience(WoS)databaseforarticlesandreviewsincluding eitherDenmarkorNorwayintheauthorsaddressfieldduringthetimespan1973–2013.Domesticonlypublicationcountsfor eachcountrywereobtainedbyexcludingallforeign“Countries/Territories”fromtheresult.Theprevalenceofinternational collaborationwascalculatedastheproportionofnon-domesticpapers.

3.1.1. Doublingtime

Constantdoublingtimeis adefiningfeatureofexponentialgrowth(e.g.Price,1951).Iassumedthat thegrowthin publicationoutput,measuredasthenumberofpapers includinga DanishorNorwegianauthoraffiliation,wasnearly exponentialbetween1973and2013(cf.Fig.2A),andcalculatedthedoublingtimeDas:

D=(t1−t0)/d, t:time interval,d:number of doublings. (1) Herethetimeintervalis(t1−t0)=2013−1973=40years.Thedenominatordisthenumberofdoublingsbetweenthe first(P₀)andthelast(P₁)publicationcount.Anestimatefordwasderivedasfollows:

2^dP0=P1 (2)

2^d=P1/P0 (3)

log₂(2^d)=log₂(P1/P0) (4)

d=log₂(P1)−log₂(P0) (5)

3.2. Top-levelpublications2012–2013

Top-levelpublicationswereidentifiedbyrestrictingthepublicationnamestoScienceorNature,andbylimitingthe timespantoincludeonly2012and2013.Alltop-levelpublicationsthusidentifiedweredownloadedfromthepublishers, andallrelevantbylineinformationwasverifieddirectlyfromthepdf-filesoftheoriginalpapers.Thisresultedin125Danish papersand44Norwegianpapers(Table2).However,9ofthesepapersincludedaffiliationsfrombothcountries,sothetotal countwas125+44−9=160papers.TwoDanishpapersincludedacoauthorwitha“Presentaddress”inNorway.Thiswas interpretedasanindicationthattheNorwegianaffiliationhadnotbeeninvolvedintheproductionofthepaper,andhence thesepaperswerenotincludedintheNorwegiancount.

For4papers,wherethebylineincludedbothagroupauthorandnamedcoauthors,oneindividualwaslistedtwice;once aspartofthegroupandonceasanamedindividualcoauthor.Inthesecasesthegroupasawholewasaccreditedaccording toitsbylineposition,andthentheperson’sportionofthegroupcontributionalongwithcreditforthenamedcontribution wereaddedtogetherandrecognizedasthatindividual’scontribution.

Theendresultwasalistof476Danishand123Norwegianindividualcoauthorcontributions(Table2).Fortwopaperswith alphabeticalbylines,authorshipcreditwasdividedequallyamongthecoauthors.Authorshipcreditforallothercontributions wascalculatedusingtheharmonicformula.

3.2.1. Harmoniccredit

HarmonicauthorshipcreditfortheithauthorofapublicationwithNcoauthorswascalculatedaccordingtothefollowing formula(Hagen,2008;Hodge&Greenberg,1981):

Harmonicithauthorcredit= 1/i

1+(1/2)+···+(1/N) (6)

A

Leimkuhler curve

0 10 20 30 40 50 60 70 80 90 100

B

Lorenz curve

0 10 20 30 40 50 60 70 80 90 100

Coauthor Contributions (%)

0 10 20 30 40 50 60 70 80 90 100

Publication Credit (%)

C

Leimkuhler curve

Lorenz curve Gini coefficient

0 10 20 30 40 50 60 70 80 90 100

Fig.1.Schematicdiagramofinequalitymeasures.(A)CoauthorcontributionsaresortedindecreasingorderinaLeimkuhlercurve.(B)Coauthorcontrib- utionsaresortedinincreasingorderinaLorenzcurve.(C)TheGinicoefficientGisdepictedastheareabetweentheLeimkuhlerandLorenzcurves.The diagonallinerepresentsG=0,ahypotheticalstateofnoinequality.

3.2.2. Additionalbylineinformation

Correspondinglastauthor.Thepresenceofacorrespondinglastauthor,orinsomecases2,3or4adjacentcorresponding lastauthors,wasinterpretedasanindicationofseniorauthorship,andthesenior(s)contributionwasregardedasequivalent tothecontributionofthefirstauthor.Consequentlytherankofintermediatecoauthorswasdemotedbythenumberofsenior authors(Hagen,2008,Fig.5therein).

Seniorauthorship,thusdefined,wasdetectedin55.6%(n=81)ofthe160papersinthedataset.Themajority(90.1%, n=64)ofthe79paperswithoutaseniorauthorhadacorrespondingfirstauthor;andofthese,onepaperhad2,andanother had3,equallycontributingcorrespondingfirstauthors.Thesecondauthorwasthecorrespondingauthoron4papers(2.5%

ofthetotal);andofthese,onepaperhad2,andanotherhad3,equallycontributingfirstauthors.Thatleavesonly3papers (1.9%ofthetotal)wherethecorrespondingauthorwasneitherthelast,thefirst,northesecondauthor.

Non-correspondinglastauthor.Historically,thereissomeambiguityastowhetheranon-correspondinglastauthoris anhonoraryauthorwhodoesnotmeetrecognizedauthorshipcriteria,orsimplythecoauthorwhohasmadethesmallest contributiontothemanuscript.Assumingthelatter,hastheadvantageofdiscouraginghonoraryauthorshipbyminimizingits gain.However,athirdpossibilityoccursifthelastauthorisinfactanauthenticseniorauthor,butonewhoisstillfollowing thenoblesseobligeconventionofrelinquishingthefirstpositiontoajuniorcolleague.Insuchacasethecontributionof Table1

Conventionalpublicationperformancecomparison.

Country Doublingtime Publicationcount Internationalcollaboration

d(Years) 1973(P0) 2013(P1) 1973(%) 2013(%)

Denmark 13.70 2106 15929 16.1 63.0

Norway 13.77 1586 11882 11.3 60.2

Publication(articleandreview)countsretrievedfromWoS,mid2014.

theseniorauthorgoesundetected,hisorhercreditisunderestimated,andthecreditofthecoauthorsiscorrespondingly overestimated(Hagen,2014a).

Footnotes.Footnotedindicationsofequalcontributionwererespectedbydividingcreditequallyamongthespecified coauthors.Asubsetof79contributionshadfootnotesindicatingsharedfirstauthorship.

Partiallyalphabeticalbylines.Partialalphabeticallistingsinvolvingaportionofthebyline,orthemembersofagroup author,weredetectedin28 papersandrangedinsizefrom5to132individualcoauthors.Forexample,a paperwith 46coauthorslistedthelast43alphabetically(Science337(6094):556–559),andapaperwith85coauthorslistedcoau- thors19–71alphabetically(Nature489(7414):101–108).Coauthorsinthesepartiallyalphabeticallistingsweretreatedas equalcontributors.Alphabeticalsequencesof≤4coauthorswerenotdetected.Ingeneral,however,suchshortalphabetical sequencesareambiguous(Waltman,2012,Section2therein),andequalityshouldthereforebeexplicitlyindicatedinorder toavoidmisinterpretation.

Authoraffiliations.Creditwassubdividedequallyamonganauthor’saffiliationswhentheauthorhadindicatedboth domesticandforeignaddresses.Theendresultwasaverifiedlistofharmoniccreditestimatesfor599individualcoauthor contributions(Table2).

3.2.3. Fractionalandinflatedcredit

Fractionalcreditwascalculatedbydividingcreditequallyamongallcoauthorsofapaper,andthensummingthecredit forthecoauthoraffiliationsfromeachcountry(Aksnes,Schneider,&Gunnarson,2012;Gauffriauetal.,2008;Gauffriau&

Larsen,2005).Inflatedcreditwascalculatedbyallocatingonefullunitofcredittoeachcountryforeverypapertowhichit hadcontributed.

3.2.4. Lorenz/LeimkuhlercurvesandGinicoefficients

Lorenz(1905)curves,andGini(2005)coefficients(Ceriani&Verme,2012),arestandardmeasuresofdistributional inequality.Lorenzcurveswereconstructedbysortingeachcountry’scontributionstoScienceandNatureinincreasingorder, andplottingthecumulativeproportionofpublicationcreditasafunctionofthepercentageofcoauthorcontributionsduring 2012and2013.

Fig.2.Fourdecadesofconcomitantriseinresearchoutputandinternationalcollaboration.(A)Publicationoutputmeasuredasthenumberofallscientific papersincludingaDanishorNorwegianauthoraffiliation.Thedashedcurvesrepresentfittedexponentialgrowthcurves.(B)Prevalenceofinternational collaborationmeasuredasthepercentageofDanishorNorwegianpapersincludingaforeignauthoraffiliation.(C)Publicationoutputgapdepictedas thedifferencebetweenthetwocurvesinpanelA.Thetrendlineisfittedtodatafrom1973until2003,andextendeduntil2013asadashedline.(D) InternationalcollaborationgapdepictedasthedifferencebetweenthetwocurvesinpanelC.Thehorizontallinerepresentsameandifferenceof4.12%.

Leimkuhler(1967)curves(Fig.1A)andLorenzcurves(Fig.1B)aresimilar.Theonlydifferenceisthatthecontributionsare sortedindecreasingorderinaLeimkuhlercurve(Burrell,1991,2005a,2005b).LorenzandLeimkuhlercurvestellthesame story,butwithdifferentemphasis–aLorenzcurvewill,forexample,tellyouthat50%ofthecontributionsproduced10%ofthe publicationcredit,whileaLeimkuhlercurvewilltellyouthat50%ofthecontributionsproduced90%ofthepublicationcredit.

Here,theGinicoefficientisdepictedastheareabetweentheLorenzcurveandtheLeimkuhlercurve(Fig.1C).Itvaries between0(ahypotheticalstateofnoinequality,representedbythediagonalline)and1(ahypotheticalstateofmaximum inequalityrepresentedbytheentireplottingarea).ThisnoveldepictionoftheGinicoefficientdiffersfromthecommonly encountereddepictionwhichonlyincludeshalfofthecoefficient,asportrayedinFig.1B.

GinicoefficientswerecalculatedusingthereldistpackageofRversion3.1.1(http://www.R-project.org).

4. Resultsanddiscussion

4.1. Delineatingthepublicationgap

4.1.1. Conventionalinflatedpublicationcount

TheacademicoutputofbothDenmarkandNorwayhasrisenrapidlyduringthepastfourdecades(Fig.2A).Bothcountries havecontributedtoanincreasingnumberofpublicationsatanearlyexponentialrate,withanestimateddoublingperiod of13.70yearsforDenmarkand13.77yearsforNorway(Table1).Notethatalthoughthepatternofincreaseappearstobe fasterthanexponentialforbothcountriesduringthelastdecade(Fig.2A),thismaybeanartefactofexpandedcoverageby theWoSdatabase(cf.Larsen&vonIns,2010).

A

0 2 4 6 8 10 12 14 16

Publication Output (1000 papers)

Denmark Norway

B

C

1973 1983 1993 2003 2013

Time (year) -6

-4 -2 0 2

Output Gap (1000 papers)

D

1973 1983 1993 2003 2013

Fig.3. Fromgaptooverlap.(A)ResearchoutputofDenmarkdelimitedastheregionbetweenthenumberofallscientificpapersincludingaDanishauthor affiliation,andthenumberofpaperswithDanishonlyaffiliations.ThecorrespondingregionforNorwayisindicatedwithdottedlines.(B)Researchoutput ofNorwaydelimitedastheregionbetweenthenumberofallscientificpapersincludingaNorwegianauthoraffiliation,andthenumberofpaperswith Norwegianonlyaffiliations.ThecorrespondingregionforDenmarkisindicatedwithdottedlines.(C)OutputgapdepictedasthedifferencebetweenDanish onlypapers,andallNorwegianpapers.(D)OutputgapdepictedasthedifferencebetweenDanishonlypapers,andNorwegianonlypapers.

DenmarkhasconsistentlycontributedtomorescientificpapersthanNorway(Fig.2A),andinabsolutetermstheoutput gapbetweenthecountriesincreasedfrom<1000papersperannumin1973toapproximately4000papersperannumin 2013(Fig.2C).Theriseingapsizewasapproximatelylinearuntil2003,whereuponasteeptemporarydeclinewasfollowed byrecovery,suchthatthevaluefor2013fellapproximatelyontheextendedtrendlinefortheperiod1973–2003.

4.1.2. Internationalcollaboration

Internationalcollaborationhasalsorisenrapidlyduringthepastfourdecades.TheproportionofDanishpaperswithat leastoneforeigncoauthoraffiliationincreasedfromlessthan16%in1973toapproximately63%in2013(Fig.2B,Table1).The correspondingfiguresforNorwayare11%and60%.Denmarkmaintainedaslightlyhigherlevelofinternationalcollaboration throughouttheentiretimeperiod(meandifference4.12%,Fig.2D).

4.1.3. Fromgaptooverlap

TheconventionalwholepublicationcountsinFig.2A,forDenmarkandforNorway,areinflatedbythecontribution ofinternationalcollaboratorstothecoauthoredpapers.Theactualpublicationoutputmustthereforebelowerthanthe conventionalpublicationcount,buthigherthanthenumberofdomesticpapers.Thisregionofindeterminacyisdemarcated inFig.3AforDenmark,andinFig.3BforNorway.Thewideningoftheregionofindeterminacyovertimereflectsincreasing levelsofinternationalcollaboration(Fig.2B).In2013theactualpublicationoutputofDenmarkwassomewherebetween 6000and16000paperequivalents,andforNorwaybetween5000and12000paperequivalents.

Priorto1993theDanishoutputremainedconclusivelyabovetheNorwegianoutput,butthesituationchangedwhen theregionsofindeterminacybegantooverlapasthewholepublicationcountforNorwayroseabovethecountofdomestic Danishonlypapers(Fig.3C).Theperiodthereafterhasbeencharacterizedbyawideningzoneofoverlap,whichby2013 hadincreasedtoawholepublicationcountofapproximately6000papers(Fig.3C).

ThenumberofdomesticDanishonlypapershasremainedconsistentlyabout1000papers(mean=997.3)abovethe numberofNorwegianonlypapersthroughouttheentireperiod(Fig.3D).However,withoutaccurateestimatesofeachcol- laborator’scontributiontotheinternationallycoauthoredpapers,itisnotpossibletodeterminewhetherthetotalNorwegian publicationoutputhasinfactremainedbelowthetotalDanishoutput.

Insummary,theutilityofconventionalwholepublicationcountsforcomparisonoftheacademicoutputgapbetween DenmarkandNorwayhasbeenjeopardizedbyincreasedinaccuracyinducedbyincreasinglevelsofinternationalcollabora- tioninbothcountries.Concomitantrapidgrowthofbothpublicationoutputandinternationalcollaborationhasproduced awideningregionofoverlapwhichmaycontainestimatesofeithercountry’sactualacademicoutput.Toproceedbeyond theresultingindeterminacyrequiresanevidence-basedapproachwhichwouldestimatemoreaccuratelythesizeofeach collaboratingcoauthor’sactualcontribution.Thisispossibleusingtheharmonicformula(Hagen,2013),buthasyettobe implementedinafullscalenationalcomparison.Itcan,however,bedonemanuallyforsmallscalecomparisons(Hagen,

A

0 50 100 150 200

Number of Contributions (n) 0

2 4 6 8 10 12 14

Cumulative Publication Credit (paper equivalents)

Norway Denmark

B

0 50 100 150 200 250

Fig.4.Top-levelclose-up.AuthorshipcreditforDenmarkandNorwayinScienceandNature.(A)2012and(B)2013.Thecurvesrepresentcumulative harmonicestimatesofactualcoauthorcontributionsortedinascendingorder.NotetheremarkableincreaseinNorwegianpublicationcreditin2013, despitealargereductioninthenumberofcoauthorcontributions.

Table2

Top-levelpublicationoutput.^aPublicationmetricsforcontributionsfromDenmarkandNorwaytoScienceandNaturein2012and2013.

Country Harmonic Fractional Papers Contributions

2012 2013 2012 2013 2012 2013 2012 2013

Denmark 14.06 14.82 11.57 15.79 57 68 198 278

Norway 4.02 6.35 3.87 4.29 21 23 75 48

Gapsize 10.04 8.47 7.7 11.5 36 45 123 218

aArticlesandreviews.

Table3

Inequalityintop-levelpublicationcredit.^aComparisonofinequalitymeasuresforcontributionsfromDenmarkandNorwaytoScienceandNaturein2012 and2013.

Country Ginicoefficient Leimkuhler50% Lorenz50%

G2012 G2013 2012(%) 2013(%) 2012(%) 2013(%)

Denmark 0.6273 0.6761 92.8 94.3 7.2 5.7

Norway 0.7778 0.5081 99.1 89.9 0.9 10.1

aHarmonicestimatesofpublicationcreditforarticlesandreviews.

A

0 10 20 30 40 50 60 70 80 90

100

B

C

0 10 20 30 40 50 60 70 80 90 100

Coauthor Contributions (%) 0

10 20 30 40 50 60 70 80 90 100

Publication Credit (%)

D

0 10 20 30 40 50 60 70 80 90 100

Fig.5.Inequalitycurves.CumulativepercentagecurvesforcoauthorcontributionandauthorshipcreditinScienceandNature.(A)Denmark2013.(B)Norway 2013.(C)Denmark2012.(D)Norway2012.TheLorenzcurves,belowthediagonal,andtheLeimkuhlercurves,abovethediagonal,arecomplimentary.

TheonlydifferenceisthattheLorenzcurvesaresortedinincreasingorderandtheLeimkuhlercurvesaresortedindecreasingorder.Thecolouredarea betweentheLorenzcurveandtheLeimkuhlercurveisageometricrepresentationoftheGinicoefficient,G,astandardmeasureofinequality.Themedian contributionisindicatedbytheverticaldottedline,anditsintersectionoftheLeimkuhlerandLorenzcurvesisindicatedwithhorizontaldottedlines.

A

3 5 10 15

Denmark Norway

Harmonic Credit (paper equivalents)

B

3 5 10 15

Fractional Credit (paper equivalents)

C

20 30 40 50 60 70

Inflated Credit (N)

D

2012 2013

0 5 10

Harmonic Gap (paper equivalents)

E

2012 2013

Time (year)

0 5 10

Fractional Gap (paper equivalents)

F

2012 2013

0 10 20 30 40

Inflated Gap (N)

Fig.6.Effectofbiasonperceivedchangeinthetop-levelacademicoutputgap.(A)Harmonicestimatesofactualpublicationcredit.(B)Conventional fractionalpublicationcredit.(C)Conventionalinflatedpublicationcredit;alsoknownasfullorwholepublicationcount.(D)Decreasingtop-leveloutput gaprevealedbyharmonicestimatesofactualpublicationcredit.(E)and(F)Effectofbiasonperceivedchangeingapsizebetween2012and2013.The dashedlinesindicatebiaseddata:inpanelsBandEequalizingbias(EqB);inpanelsCandFacombinationofEqBandinflationarybias.Dataforarticlesand reviewsinScienceandNature.

2014b).Thefocusofthenextsection,therefore,istoexpandthescopeoftheevidence-basedapproachbyprovidingasmall scaletop-levelclose-upofthepublicationgapbetweenDenmarkandNorway.

4.2. Atop-levelclose-up 4.2.1. Publicationcredit

Therewerelargechangesfrom2012to2013inthepatternofDanishandNorwegiancontributionstoScienceandNature (Fig.4AandB).In2013,Denmarkmademorecontributionstomorepapers,andhadahigherpublicationscorethanthe yearbefore(Table2).Butthe40%increaseincontributioncount,yieldedonlyamodest5.4%increaseinpublicationcredit.

Incontrast,Norwayexhibitedalarge58%increaseinpublicationcredit,despitealargeconcurrentreductioninthenumber ofcontributionsfrom75in2012,to48in2013(Fig.4AandB).

Aconspicuousfeatureofthesechangingpatternsisthatthelefttailofthecumulativepublicationcreditcurve,i.e.the proportionofsmallcontributions,grewlongerforDenmarkyetshorterforNorway,from2012to2013(Fig.4AandB).

4.2.2. Contributoryinequality

ThelevelofcontributoryinequalitypeakedinthedataforNorway2012,whenthemostproductivehalfofthecontrib- utionsaccountedfor99.1%ofthepublicationcredit(LeimkuhlercurveFig.5D,Table3).Conversely,accordingtotheLorenz curvetheleastproductivehalfofthecontributionsaccountedforonly0.9%ofthecredit.Oneyearlater,inequalitywasmuch

reducedandtheleastproductivehalfofthecontributionsaccountedfor10.1%ofthepublicationcredit(LorenzcurveFig.5B, Table3).

Incontrast,themostproductivehalfoftheDanishcontributionsaccountedfor92.8%ofthepublicationcreditin2012 (LeimkuhlercurveFig.5C,Table3),and94.3%in2013(LeimkuhlercurveFig.5A,Table3),whereastheleastproductivehalf ofthecontributionsaccountedfor7.2%in2012and5.7%in2013.

TheseresultsareconsistentwiththesummaryprovidedbytheGinicoefficients(Table3).Norwayhadamarkedreduction incontributoryinequality,from0.78in2012to0.51in2013,whileDenmarkremainedatacomparativelyhighlevelof inequality,withaslightincreasefrom0.63in2012to0.68in2013.

4.2.3. Outputgap

Harmonicestimatesofactualpublicationcredit(Fig.6A,Table2),indicatedanarrowingofthetop-levelpublicationgap betweenDenmarkandNorwayfrom2012to2013(Fig.6D).Incontrast,thelessaccuratefractional(Fig.6B),andinflated creditestimates(Fig.6C),arrivedattheoppositeconclusionandpurportedthatthegapwaswidening(Fig.6EandF).The discrepancyisadirectcauseofbiasinthefractionalandinflatedestimates.

5. Conclusions

InthispaperIhaveuseddataforarticlesandreviewsinScienceandNaturetodocumentarecentnarrowingofthe top-leveloutputgapbetweenDenmarkandNorway.Denmarkstillleads,butthegapnarrowedbetween2012and2013 becausethepublicationcreditofNorwayroseby58%,whilethecreditofDenmarkrosebyonly5.4%.Thenarrowingofthe gapwasdetectedbyusingharmonicestimatesofactualcontributiontoremoveinflationarybiasandequalizingbias(EqB) fromconventionalpublicationcounts.Thenarrowingwasnotdetectedwhenconventionalinflatedorfractionalpublication countswereusedasproxiesforpublicationcredit.Onthecontrary,theconventionalapproachpurportedthatthegapwas widening.Thisresulthighlightstheutilityofincreasedaccuracyinestimatesofpublicationcreditinnationalcomparisons.

ForNorway,thenarrowingoftheoutputgapwasaccompaniedbyalargedropincontributoryinequality,anda36%

reductioninthenumberofcoauthorcontributions.Inotherwords,fewercontributorswhoproducedmorepublicationcredit.

Incomparison,forDenmarkthenarrowingcoincidedwithaslightincreaseincontributoryinequality,anda40%increase inthenumberofcontributions,thatismorecontributorswhoproducedlesspublicationcredit.Thesedivergingpatterns,in thenumberandinequalityofthetwocountries’contributions,provideaproximateexplanationfortheobservednarrowing oftheoutputgapbydemonstratingthatreducedcontributoryinequalitywasassociatedwithincreasedpublicationoutput.

Theremarkabledynamicsofthesechangeswereobscuredanddistortedbyequalizingbias(EqB)inthefractionalesti- matesofpublicationoutput,andbyacombinationofEqBandinflationarybiaswhenusingconventionalwholecounts asaproxyforpublicationoutput.Specifically,thedivergingpatternsinthenumberandinequalityofthetwocountries’

contributions,suggestthattheconventionalmeasuresdidnotdetectthenarrowingofthegapbecausetheirinherentEqB overestimatedtheimportanceoftheincreasinglynumeroussmallDanishcontributionsandunderestimatedtheimportance oftheincreasedproportionoflargeNorwegiancontributions.

TherecentlylaunchedNatureIndex(2014b)includesatableofcountryoutputsforScienceandNature,whichusesinflated andfractionalpublicationcounts.TheseconventionalmetricsarenotcorrectedforEqB,andmaythereforemisrepresent thegapbetweencomparablecountrieswithhighlevelsofinternationalcollaboration.Theresultsofthispaper,however, indicatethatimprovedaccuracyisreadilyattainablebyincludingallrelevantinformationabouttherelativecontribution ofcoauthors,andbyusingtheharmonicformulatoquantifyhierarchicalbylines.

Equalizingbiasisofspecialconcernbecauseitcarriesalargecounterproductiveelementofreversemeritocracyand systemicinjusticeintobibliometricperformanceassessmentbyfavouringsmallcontributorsattheexpenseoflargecon- tributors.EqBactsinanasymmetricalfashionwhichprecludestop-downcompensationforitsdistortionaleffects.Tocorrect forEqBinnationalcomparisonsrequiresabottom-upapproachwhichincorporatesrelevantbylineinformationfromevery internationallycoauthoredpublicationinthesourcedata.Rightnow,thisstillmanualtaskissufficientlylabour-intensiveto precludeafull-scaleanalysisofthetotalproductivitygapbetweenDenmarkandNorway.However,immediateimplemen- tationofautomatedharmoniccreditallocationcouldcorrectforEqB,andfacilitateunconstrainedbibliometricgapanalysis withunprecedentedaccuracy.Inthemeantime,thetop-tier,close-upapproachadoptedheredemonstratesthefeasibility ofusingEqB-correctedpublicationcreditingapassessmentatthenationallevel.

Acknowledgements

ThankstoH.K.Marshallforimprovingthestyleandflowofthemanuscript,andthankstotheUniversityofNordlandfor providingasupportiveworkenvironmentduringdataanalysisandmanuscriptpreparation.

References

Aksnes,D.W.,Frølich,N.,&Slipersæter,S.(2008).Sciencepolicyandthedrivingforcesbehindtheinternationalisationofscience:thecaseofNorway.

ScienceandPublicPolicy,35,445–457.http://dx.doi.org/10.3152/030234208X331236

Aksnes,D.W.,Schneider,J.W.,&Gunnarson,M.(2012).Rankingnationalresearchsystemsbycitationindicators.Acomparativeanalysisusingwholeand fractionalisedcountingmethods.JournalofInformetrics,6,36–43.http://dx.doi.org/10.1016/j.joi.2011.08.002

Babbage, C. (1830). Reflections on the decline of science in England, and on some of its causes. London: B. Fellowes & J. Booth.

http://books.google.no/books?id=oXbRotQLLtsC&dq=On+the+alleged+decline+of+science+in+England&hl=no&source=gbsnavlinkss

Bidault,F.,&Hildebrand,T.(2014).Thedistributionofpartnershipreturns:Evidencefromco-authorshipsineconomicsjournals.ResearchPolicy,43, 1002–1013.http://dx.doi.org/10.1016/j.respol.2014.01.008

Braun, T. (1993). On some historical antecedents of the belief in the decline of British science. Journal of Information Science, 19, 401.

http://dx.doi.org/10.1177/016555159301900508

Burrell,Q.L.(1991).TheBradforddistributionandtheGiniindex.Scientometrics,21,181–194.http://dx.doi.org/10.1007/BF02017568

Burrell,Q.L.(2005a]).Measuringinformetricsimilarityofconcentrationbetweendifferentdistributions:Twonewapproaches.JournaloftheAmerican SocietyforInformationScienceandTechnology,56,704–714.http://dx.doi.org/10.1002/asi.20160

Burrell,Q.L.(2005b]).SymmetryandothertransformationfeaturesofLorenz/Leimkuhlerrepresentationsofinformetricdata.InformationProcessingand Management,41,1317–1329.http://dx.doi.org/10.1016/j.ipm.2005.03.016

Cardwell,D.S.L.(1972).TheorganisationofscienceinEngland(2ndreviseded.).London:Heinemann.

Ceriani,L.,&Verme,P.(2012).TheoriginsoftheGiniindex:extractsfromVariabilitàeMutabilità(1912)byCorradoGini.JournalofEconomicInequality, 10,421–443.http://dx.doi.org/10.1007/s10888-011-9188-x

Devonshire,W.L.(1872).Royalcommissiononscientificinstructionandtheadvancementofscience.First,supplementary,andsecondreports,withminutesof evidenceandappendices,Vol.I.London:G.E.EyreandW.Spottiswoode.https://archive.org/details/reportroy01greauoft

Fernandes,J.M.(2014).Authorshiptrendsinsoftwareengineering.Scientometrics,101,257–271.http://dx.doi.org/10.1007/s11192-014-1331-6 Foreigner,A.(1931).OntheallegeddeclineofscienceinEngland,withaforewordbyMichaelFaraday(AuthornameisapseudonymforGerardMoll).London:

T.&T.Boosey.http://books.google.no/books?id=oXbRotQLLtsC&dq=On+the+alleged+decline+of+science+in+England&hl=no&source=gbsnavlinkss Franz, S. I. (1917). The scientific productivity of American professional psychologists. The Psychological Review, 24, 197–219.

http://dx.doi.org/10.1037/h0070739

Gauffriau,M.,&Larsen,P.O.(2005).Countingmethodsaredecisiveforrankingsbasedonpublicationandcitationstudies.Scientometrics,64,85–93.

http://dx.doi.org/10.1007/s11192-005-0239-6

Gauffriau,M.,Larsen,P.O.,Maye,I.,Roulin-Perriard,A.,&vonIns,M.(2008).Comparisonsofresultsofpublicationcountingusingdifferentmethods.

Scientometrics,77,147–176.http://dx.doi.org/10.1007/s11192-007-1934-2

Gini,C.(2005).Onthemeasurementofconcentrationandvariabilityofcharacters(OriginallypublishedinItalianinAttidelRealeIstitutoVenetodi Scienze,LettereedArti.1914,Vol.LXXIII–PartII,p.1203–1248.Translation:FulvioDeSantis).METRON–InternationalJournalofStatistics,LXIII,3–38.

ftp://metron.sta.uniroma1.it/RePEc/articoli/2005-1.pdf

Glänzel,W.(2000).ScienceinScandinavia:Abibliometricapproach.Scientometrics,48,121–150.http://dx.doi.org/10.1023/A:1005640604267 Godin,B.(2006).Ontheoriginsofbibliometrics.Scientometrics,68,109–133.http://dx.doi.org/10.1007/s11192-006-0086-0

Godin,B.(2009).Thevalueofscience:Changingconceptionsofscientificproductivity,1869tocirca1970.SocialScienceInformationSurLesSciencesSociales, 48,547–586.http://dx.doi.org/10.1177/0539018409344475

Hagen,N.T.(2008).Harmonicallocationofauthorshipcredit:Source-levelcorrectionofbibliometricbiasassuresaccuratepublicationandcitationanalysis.

PLoSONE,3http://dx.doi.org/10.1371/journal.pone.0004021,e4021:1–e4021:7

Hagen,N.T.(2009).Creditforcoauthors.Science,323,583.http://dx.doi.org/10.1126/science.323.5914.583a

Hagen,N.T.(2010).Harmonicpublicationandcitationcounting:Sharingauthorshipcreditequitably–Notequally,geometricallyorarithmetically.

Scientometrics,84,785–793.http://dx.doi.org/10.1007/s11192-009-0129-4

Hagen, N. T. (2013). Harmonic coauthor credit: A parsimonious quantification of the byline hierarchy. Journal of Informetrics, 7, 784–791.

http://dx.doi.org/10.1016/j.joi.2013.06.005

Hagen,N.T.(2014a]).Countingandcomparingpublicationoutputwithandwithoutequalizingandinflationarybias.JournalofInformetrics,8,310–317.

http://dx.doi.org/10.1016/j.joi.2014.01.003

Hagen,N.T.(2014b]).Reversingthebylinehierarchy:Theeffectofequalizingbiasontheaccreditationofprimary,secondaryandseniorauthors.Journal ofInformetrics,8,618–627.http://dx.doi.org/10.1016/j.joi.2014.05.003

Hicks,D.(2012).Performance-baseduniversityresearchfundingsystems.ResearchPolicy,41,251–261.http://dx.doi.org/10.1016/j.respol.2011.09.007 Hodge,S.E.,&Greenberg,D.A.(1981).Publicationcredit.Science,213,950.http://links.jstor.org/sici?sici=0036-8075%2819810828%293%3A213%3A4511%

3C950%3APC%3E2.0.CO%3B2-X

Larsen,P.O.,&vonIns,M.(2010).TherateofgrowthinscientificpublicationandthedeclineincoverageprovidedbyScienceCitationIndex.Scientometrics, 84,575–603.http://dx.doi.org/10.1007/s11192-010-0202-z

Leimkuhler,F.F.(1967).TheBradforddistribution.JournalofDocumentation,23,197–207.http://dx.doi.org/10.1108/eb026430

Leydesdorff,L.(2012).WorldsharesofpublicationsoftheUSA,EU-27,andChinacomparedandpredictedusingthenewWebofScienceinterfaceversus Scopus.Elprofesionaldelainformación,21,43–49.http://dx.doi.org/10.3145/epi.2012.ene.06

Leydesdorff,L.,&Wagner,C.S.(2008).Internationalcollaborationinscienceandtheformationofacoregroup.JournalofInformetrics,2,317–325.

http://dx.doi.org/10.1016/j.joi.2008.07.003

Lindsey,D.(1980).Productionandcitationmeasuresinthesociologyofscience:Theproblemofmultipleauthorship.SocialStudiesofScience,10,145–162.

http://dx.doi.org/10.1177/030631278001000202

Lorenz, M. O. (1905). Methods of measuring the concentration of wealth. Publications of the American Statistical Association, 9, 209–219.

http://www.jstor.org/stable/2276207

Martin,B.R.(1994).Britishscienceinthe1980–Hastherelativedeclinecontinued?Scientometrics,29,225–256.http://dx.doi.org/10.1007/BF02018382 Narin,F.(1976).Evaluativebibliometrics:Theuseofpublicationandcitationanalysisintheevaluationofscientificactivity.InReportInfullfillmentofcontract

NSFC-627withtheNationalScienceFoundation1800GStreet,N.W.,Washington,D.C.20550,U.S.A.,ComputerHorizons,Inc.,1050KingsHigwayNorth, CherryHill,NewJersey,U.S.A,http://yunus.hacettepe.edu.tr/∼tonta/courses/spring2011/bby704/narin1975eval-bibliometricsimages.pdf National Science Board. (2014). Science and engineering indicators 2014. Report NSB 14-01. Arlington VA, USA: National Science Foundation.

http://www.nsf.gov/statistics/seind14/index.cfm/etc/pdf.htm

Nature Index. (2014a]). Aguideto the Nature Index. Nature, 515, S94. http://dx.doi.org/10.1038/515S94ahttp://www.natureindex.com/country- outputs?region=global&subject=543ddffb140ba0701a7e14bf&sortby=score&generate=Generate

NatureIndex.(2014b]).Countryoutputsinnatureandscience.http://www.natureindex.com/country-outputs?region=global&subject=543ddffb140ba0701a 7e14bf&sortby=score&generate=Generate

Nowotny,H.(2007).Howmanypolicyroomsarethere?Evidence-basedandotherkindsofsciencepolicies.Science,Technology,&HumanValues,32, 479–490.http://dx.doi.org/10.1177/0162243907301005

Nye,M.J.(1984).Scientificdecline:Isquantitativeevaluationenough?Isis,75,697–708.

Price,D.D.S.(1951).Quantitativemeasuresofthedevelopmentofscience.ArchivesInternationalesd’HistoriedesSciences,14,413–421.

Price,D.D.S.(1981).Multipleauthorship.Science,212,986.http://dx.doi.org/10.1126/science.212.4498.986-a

Price,D.D.S.,&Beaver,D.D.(1966).Collaborationinaninvisiblecollege.AmericanPsychologist,21,1011–1018.http://dx.doi.org/10.1037/h0024051 Research Council ofNorway. (2014).Science andtechnology indicatorsfor Norway (Det norskeforsknings- og innovasjonssystemet - statistikk og

indikatorer,indikatorrapporten 2014)(in Norwegian).Report. Postboks2700, St. Hanshaugen, N-0131 Oslo:TheResearch Council ofNorway.

http://www.forskningsradet.no/prognett-indikatorrapporten/Forside/1224698172624

Royal Society UK. (2011). Knowledge, networks and nations: Global scientific collaboration in the 21st century. Report. UK: Royal Society.

http://royalsociety.org/policy/projects/knowledge-networks-nations/report/