Counting and comparing publication output with and without equalizing and inflationary bias

(1)

ContentslistsavailableatScienceDirect

Journal of Informetrics

jou rn al h om ep a ge : w w w . e l s e v i e r . c o m / l o c a t e / j o i

Counting and comparing publication output with and without equalizing and inﬂationary bias

Nils T. Hagen

^∗

FacultyofBiosciencesandAquaculture,UniversityofNordland,N-8049Bodø,Norway

a r t i c l e i n f o

Articlehistory:

Received20November2013

Receivedinrevisedform3January2014 Accepted6January2014

Availableonline6February2014

Keywords:

Publicationperformanceranking Coauthorcredit

Harmonicformula Bibliometricbias Equalizingbias Inﬂationarybias

a b s t r a c t

Thispaperexaminestheeffectsofinflationaryandequalizingbiasonpublicationoutput rankings.Anyidentifiableamountofbiasinauthorshipaccreditationwasdetrimentalto accuracywhenrankingaselectgroupofleadingCanadianaquacultureresearchers.Bias arosewhenpublicationscoreswerecalculatedwithouttakingintoaccountinformation aboutmultipleauthorshipanddifferentialcoauthorcontributions.Theensuingbiased equalcreditscores,whetherfractionalorinflated,producedrankingsthatwerefundamen- tallydifferentfromtherankingofharmonicestimatesofactualcreditcalculatedbyusingall relevantbylineinformationinthesourcedata.Inconclusion,theresultsindicatethatboth fractionalandinflatedrankingsaremisleading,andsuggestthataccurateaccreditationof coauthorsisthekeytoreliablepublicationperformancerankings.

1. Introduction

Tocountpublicationsisthemostbasictaskinevaluativebibliometricsandscientometrics(DeBellis,2009).Theoutcome ofanysuchtask,usuallyarankingorothercomparativeassessment,isdeterminedbyhowthecountableunitsofpublication areselectedandaccredited.Hence,whentworankingsbasedonthesamesetofpublicationsproducefundamentallydifferent results,atleastonemustbemisleadingasaconsequenceofinaccurateaccreditation.Systematicinaccuraciesinaccreditation arisewhenauthorshipcreditisnotdividedamongcoauthors,i.e.inﬂationarybias,orwhencreditisdividedequallyamong coauthorswhohavenotcontributedequally,i.e.equalizingbias(Hagen,2008).Here,thequestionofinterestis:Howserious istheeffectofthesebiases?

Itisknownfromseveralstudiescomparinginﬂatedandfractionalcreditthatinﬂationarybiasalterspublicationperfor- mancerankings(Aksnes,Schneider,&Gunnarson,2012;Gauffriau&Larsen,2005;Gauffriauetal.,2008;Huang&Lin,2011;

Huang,Lin,&Chen,2011;Piro,Aksnes,&Rørstad,2013;Pravdic&Oluic-Vukovic,1991).However,thepossibilitythatboth inﬂatedandfractionalrankingsaremisleadingcannotbedismissedaslongasneitherrankingiscorrectedforequalizing bias.

Lessisknownabouttheeffectofequalizingbias.Onestudyfoundthatrankingsofh-indexscoresweregravelydistorted bybothbiases(Hagen,2008),andanotherstudyestimatedthatequalizingbiasaccountedforapproximately60%ofthe

∗ Tel.:+4775517398.

E-mailaddress:[email protected]

http://dx.doi.org/10.1016/j.joi.2014.01.003

Open access under the CC BY-NC-ND license.

(2)

variationinacompositeempiricaldataset(Hagen,2013).Theseresultssuggestthatthedistortionaleffectofequalizingbias maybecomparabletotheeffectofinﬂationarybias.Furthermore,itisimportanttoaccountforequalizingbiasasitmay affectamajorityofcontemporarypublications(Waltman,2012),andbecauseitcontrovertsthepurposeofperformance rankingbydivertingcreditfromprimaryauthorstosecondaryauthors(Hagen,2008).

InthisstudyIcomparetheseparateandcombinedeffectsofequalizingbiasandinﬂationarybiasontherankingofleading Canadianaquacultureresearchers,andshowthatbothbiaseshaveadetrimentaleffectonaccuracy.Iconcludebydiscussing theneedtomovefromproxiesofauthorshipcredittomoreaccurateestimatesbasedonallavailablerelevantinformation inordertoconstructreliablepublicationperformancerankings.

2. Materialsandmethods 2.1. Empiricaldata

TheempiricaldatawereindependentlyderivedfromarankingofleadingCanadianaquacultureresearchers(Picard- Aitken&Coté,2010,tableXIX,p.34).Thenumberofpublicationsforeachresearcherwasmatchedusingthesametimeframe (1996-2008)andpublicationtypes(journalarticles,conferencepapers,notes,andreviewsfrompeer-reviewedjournals)to extractdataforeachresearcherfromthesamecommercialdatabasevendor(Scopus).Thistaskwasnontrivialasthenumber ofobviousaquaculturepublicationsforeachresearcherrarelycorrespondedtotheirnumberintableXIX(Picard-Aitken&

Coté,2010),andpublicationsofpossiblerelevanceforaquaculturehadtobeomittedoraddedinordertoobtainamatching number.Thisapproachworkedfor35ofthe36researchersintheoriginaltable.Butforoneresearcherwhowaslistedwith 19“aquaculture”publications,Ifounditimpossibletomakeameaningfulselectionofpapersamongtheapproximately70 publicationsprovidedbytheScopusdatabase.Thisresearcherhadconductedbasicresearchusingcellculturesderivedfrom thetissueofaquaculturespecies,anditiseasytoseehowakeywordsearchinalibrarydatabasecoulddeliveranimprecise result.

Theendresultwasareplicatedatasetconsistingof699authorshipcontributionsfrom35researchers,to531research papersfrom120differentjournals(Thereplicatedatasetisavailableonrequest).Only10publicationsweresingle-authored (1.9%of531papers),andIwasunabletodetectanyunequivocalindicationofequalityamongthecoauthoredcontributions (98.6%of689contributionsfrom521papers).However,125contributionscame fromcoauthoredpaperswheresenior authorshipwasindicatedbythepresenceofacorrespondinglastauthor.

2.2. Authorshipquantiﬁcation

Rankingswereconstructedbytallyinginﬂated,fractionalandharmoniccreditscoresforeachofthe35researchersin thereplicatedataset.

Inﬂatedcreditwascalculatedbyusingcontributioncountasaproxyforauthorshipcredit(cf.Picard-Aitken&Coté,2010), i.e.byassigningonefullunitofauthorshipcreditrepeatedlytoeverycoauthor:

Inﬂatedithauthorcredit=1 (1)

FractionalcreditwasobtainedbydividingonefullunitofcreditequallyamongallNcoauthorsofamulti-authored publicationasfollows:

Fractionalithauthorcredit= 1

N (2)

HarmonicauthorshipcreditfortheithauthorofapublicationwithNcoauthorswascalculatedaccordingtothefollowing formula(Hagen,2008,2013;Hodge&Greenberg,1981):

Harmonicithauthorcredit= 1/i

1+(1/2)+···+(1/N) (3)

Iassumedthatthepresenceofacorrespondinglastauthorindicatedaseniorauthorwhosecontributionwasequivalent tothecontributionoftheﬁrstauthor(cf.Buehring,Buehring,&Gerard,2007;Mattssonetal.,2011).Insuchcases,theﬁrst andtheseniorauthorsharethecreditforthe1stand2ndposition,andthisreducesthecreditofintermediatecoauthorsby onepositionasfollows(cf.Hagen,2008,Fig.5):

1standsenior(Nth)authorcredit= 1+(1/2)

2(1+(1/2)+···+(1/N)) (4)

Intermediate(i=2,...,N−1)authorcredit= 1/(i+1)

1+(1/2)+···+(1/N) (5)

Theaccuracyoftheharmonicformulawasunrivalledbyotherformulationsfromthebibliometricliterature,when assessedagainstanempiricalbaseline(Hagen,2010,2013).

(3)

2.3. Rankcorrelation

Kendall’srankcorrelationcoefficient (tau)(Kendall,1938)wasdeterminedforeachpairofharmonic(H),fractional (F),andinflated(I)creditscores,HF,HI,andFI,usingthestatisticalsoftwarepackageR(http://www.r-project.org/).From thesecoefficientvaluesIcalculatedKendall’scoefficientofpartialrankcorrelation(Kendall&Gibbons,1990;Kendall,1942), asfollows:

HI.F=

HI−HFIF

(1−²_HF)

(1−²_IF)

(6) HI.Fisameasureofthecorrelationbetweentheharmonic(H)andinﬂated(I)rankingsaftereliminatingtheeffectoftheir correlationwiththefractionalranking(F).ThetestofsigniﬁcanceforHI.Fwascarriedoutusingthecriticalvalue0.3041for N=35and(1−˛)=0.995fromTableVIinMaghsoodlooandPallos(1981).

3. Results

3.1. Effectofequalizingandinﬂationarybiasoncreditscores

Theinﬂatedcreditscores,i.e.thenumberofpaperseachresearcherhadcontributedtoduringthetimeperiod1996-2008, rangedfrom13to35(Fig.1).Accordingtotheharmoniccreditscores,eachresearcher’sactualcontributionrangedfrom 2.29to10.41paperequivalentsorbetween14.9%and52.5%oftheauthorshipcredit(Table1).Thedifferencebetweenthe harmonicandinﬂatedcreditscoresrepresentsthesumofcontributionsmadebyeachresearcher’scollaboratingcoauthors.

Thisdifferenceisquantiﬁedbyacombinationofequalizingbias,rangingfrom−1.16to3.1,andinﬂationarybiasranging from6.5to24.4.

Inﬂationarybiasaccountsforthelargeandobviousdifferencebetweeninﬂatedandfractionalcreditscores,whereas equalizingbiasaccountsforthenumericallysmallerbutseeminglyequallydisruptivedifferencebetweenharmonicand

Harmonic Inflated

1 5 10 15 20 25 30 35

Authorship Credit (Publication score)

Harmonic Fractional Inflated

Credit Quantification Scheme

Fig.1. Comparisonofharmonic,inﬂatedandfractionalcreditscoresforaselectgroupof35leadingCanadianaquacultureresearchers.Harmoniccredit scoresareestimatesofactualauthorshipcreditbasedoneachcoauthorspositioninthebylinehierarchy,aswellasadditionalinformationindicatingthe presenceofaseniorauthor.Inﬂatedcreditscoresillustratetheeffectofusingcontributioncountasaproxyforauthorshipcredit.Fractionalcreditscores showtheeffectofassumingthatallcoauthorshavecontributedequally.Slantedlinesindicatebias-inducedalterationsofcreditscores,andcrossinglines indicatebias-inducedrankdisplacement.

(4)

fractionalcreditscores(Fig.1).Notethattherangeoffractionalcreditscores,2.72–11.18,iscomparabletotherangeofthe harmoniccreditscores,whichindicatesthattheeffectofequalizingbiasisgreatestforintermediatecreditscores.Notealso thatcrossinglinesinFig.1indicatecreditscoreswhichasadirectresultofaddedbiashavechangedsufﬁcientlytoinduce rankdisplacement.Thesechangesareexaminedinthenextsection.

3.2. Bias-inducedrankdisplacement

Therewasnocoincidencebetweentheharmonicandinflatedrankings.Theranksofallresearchersweredisplacedbythe combinationofequalizingbiasandinflationarybiasthatwasgeneratedbyusingcontributioncountasaproxyforcoauthor credit.Notably,theinflatedrankinghadalargenumberoftiedranks,allofwhichwereresolvedintheharmonicranking (Fig.2).Thetiedranksrepresentresearcherswhohadcontributeddifferentiallytoanidenticalnumberofpapers(Table1), andthecorrespondingrangeofharmonicranksreflectsdifferencesinthemagnitudeoftheirestimatedcontributiontothose papers.Inall,72.2%oftheinflatedranksweretied.Thelargesttiedgroupconsistedof9researcherswhosharedthebottom positionoftheinflatedranking.Intheharmonicrankingtheywereredistributedoverarangeof23steps,fromrank10to rank33,onascalewithonly35ranks(Fig.2).

Thefractionalrankingresolvedthetiesoftheinﬂatedranking,and4ofitsranks(11.4%)coincidewiththeharmonic ranking.Nevertheless,31fractionalranks,representing88.6%oftheevaluatedresearchers,weredisplacedasadirectresult ofequalizingbiasgeneratedbydividingcreditequallyamongcoauthorswhohadnotcontributedequally.Thedisplacement rangedfromanadvantageof13stepstoadisadvantageof21steps,onascalewithonly35ranks(Fig.3).

Althoughthefractionalrankingexhibitedasignificantrankcorrelationwithboththeharmonicandtheinflatedrankings (Table2),therankcorrelationbetweentheharmonicandinflatedrankingswasweaker,and dwindledtoalmostzero (Kendall’s partialrankcorrelationcoefficientHarmonicInflated.Fractional=0.0225), whentheeffectoftheserankingsmutual correlationwiththefractionalrankingwaseliminated.Thisresultsuggeststhattheaddedeffectofinflationarybiasis largelyindependentoftheeffectofequalizingbias.

Table1

Summarydatafor35Canadianaquacultureresearchers.Harmonic(H),fractional(F)andinﬂated(I)creditscoresandrankswithassociatedestimatesof biasandactualcontributionpercentage.

Creditscore Rank Bias Contribution(%)

H F I H F I Equalizing Inﬂationary

10.41 11.05 33 1 2 3 0.64 21.95 31.54

10.14 10.59 35 2 3 1 0.44 24.41 28.98

9.45 9.33 18 3 5 18 −0.12 8.67 52.50

9.21 8.16 26 4 6 8.5 −1.05 17.84 35.42

8.77 9.35 32 5 4 4 0.58 22.65 27.41

8.64 11.18 34 6 1 2 2.55 22.82 25.40

7.92 7.07 22 7 11 13 −0.85 14.93 35.98

7.16 7.28 28 8 10 7 0.12 20.72 25.58

6.53 7.45 26 9 8 8.5 0.92 18.55 25.13

6.06 5.62 13 10 16 32 −0.44 7.38 46.59

5.89 6.47 18 11 13 18 0.57 11.53 32.74

5.72 4.57 13 12 23 32 −1.16 8.43 44.04

5.48 5.78 19 13 15 15 0.30 13.22 28.83

5.47 6.03 29 14 14 6 0.56 22.97 18.85

5.36 7.40 24 15 9 11.5 2.04 16.60 22.32

5.30 6.50 13 16 12 32 1.20 6.50 40.79

5.19 5.56 18 17 17 18 0.37 12.44 28.82

4.79 4.86 19 18 20 15 0.08 14.14 25.19

4.62 3.68 13 19 29 32 −0.93 9.32 35.50

4.61 7.65 31 20 7 5 3.04 23.35 14.88

4.58 3.67 15 21 30 23.5 −0.91 11.33 30.50

4.33 4.85 15 22 21 23.5 0.52 10.15 28.87

4.19 5.50 17 23 19 20 1.31 11.50 24.65

4.11 5.54 25 24 18 10 1.43 19.46 16.44

3.91 4.85 24 25 22 11.5 0.94 19.15 16.29

3.76 3.98 13 26 26 32 0.22 9.02 28.93

3.67 2.79 13 27 34 32 −0.88 10.21 28.23

3.40 3.72 16 28 28 21.5 0.31 12.28 21.27

3.23 2.72 13 29 35 32 −0.51 10.28 24.87

3.06 4.24 16 30 24 21.5 1.18 11.76 19.15

3.01 3.19 13 31 32 32 0.18 9.81 23.18

2.68 4.08 14 32 25 26 1.39 9.92 19.16

2.60 3.78 13 33 27 32 1.19 9.22 19.97

2.30 3.00 14 34 33 26 0.69 11.00 16.44

2.29 3.46 14 35 31 26 1.17 10.54 16.34

(5)

Harmonic Inflated 1

5

10

15

20

25

30

35

Publication Performance Rank

Ranking Scheme

Fig.2. Comparisonofharmonic,fractionalandinflatedrankingsofaselectgroupof35leadingCanadianaquacultureresearchers.Theharmonicranking isbasedonestimatesofactualauthorshipcredit.Fractionalrankingshowstheeffectofequalizingbiasgeneratedbyassumingthatallcoauthorshave contributedequally.Theinflatedrankingillustratestheeffectofinflationarybiasgeneratedbyusingcontributioncountasaproxyforauthorshipcredit.

Linesfromharmonictoinflatedshowthecombinedeffectofequalizingandinflationarybias,linesfromharmonictofractionalshowtheeffectofequalizing bias,andlinesfromfractionaltoinflatedshowtheaddedeffectofinflationarybiasrelativetothefractionalranking.Horizontallinesindicatecoinciding ranks,andcrossinglinesindicatebias-inducedrankdisplacement.

Table2

RankcorrelationanalysisofCanadianaquacultureresearchers.Therelationshipbetweenrankingsofharmonic(H),fractional(F),andinflated(I)estimates ofcoauthorcreditisquantifiedusingKendall’srankcorrelationcoefficient.Kendall’scoefficientofpartialrankcorrelationHI.Fisameasureofthe correlationbetweentheharmonicandinflatedrankingsaftereliminatingtheeffectoftheircorrelationwiththefractionalranking.

Comparison Coefﬁcient Rankcorrelation P-value

Harmonicvs.Inﬂated HI 0.4623 <0.001

Harmonicvs.Fractional HF 0.6908 <0.001

Inﬂatedvs.Fractional IF 0.6515 <0.001

Partialrankcorrelation

Harmonicvs.Inﬂated HI.F 0.0225 >0.05^a

aMaghsoodlooandPallos(1981),TableVItherein.

4. Discussion

Biasedequalcreditscores,whetherfractionalorinflated,producedrankingsthatwerefundamentallydifferentfrom therankingbasedonharmonicestimatesofactualcoauthorcontribution.Theharmonic,fractionalandinflatedrankings wouldhavecoincidedifalloftheCanadianaquacultureresearchershadcontributedequallyandhadthesamenumberof coauthors.Buttherewasnounequivocalindicationofequalcontributionandthenumberofcoauthorsvaried,andsothe lackofagreementbetweentherankingsisexplainedentirelybytheeffectsofequalizingbiasandinflationarybias.

Equalizingbiasskewscreditscoresbysystematicallyfavouringsecondaryauthorsattheexpenseofprimaryauthors, andinﬂationarybiasfavoursresearcherswhohavemanycoauthorsandmakesmallcontributions,overresearcherswho havefewcoauthorsandmakelargercontributions(Hagen,2008).Bothbiaseshaddetrimentaleffectsontheaccuracyofthe rankings.

(6)

1 5 10 15 20 25 30 35 Researchers Arranged In Harmonic Ranking Order

1

5

10

15

20

25

30

35

Publication Performance Rank

Fig.3. Comparisonofbias-inducedrankdisplacement.ThegreendiagonalreferencelinerepresentstheharmonicrankingoftheCanadianaquaculture researchers.Rankdisplacementisindicatedbydistancefromthediagonalline.Inﬂatedranksshowthecombinedeffectofequalizingandinﬂationarybias.

Fractionalranksshowtheresidualeffectofequalizingbiasaftercorrectingforinﬂationarybias.

Canadianaquacultureresearchers’publicationperformancehasbeenrankedwithinflatedpublicationscoresontwo previousoccasions(Picard-Aitken&Coté,2010;Sylvain,1993).Thepresentanalysisisbasedonareplicatedatasetmatching thenumberofpapersforresearchersincludedinthemostrecentranking.Itshowsthatbothinflatedandfractionalrankings wereseriouslyaffectedbybias-inducedrankdisplacement(Figs.2and3).Inotherwords,fractionalcreditscores,although correctedforinflationarybias,arestillconfoundedbyequalizingbias.

Interestingly,whenPravdicandOluic-Vukovic(1991)removedinﬂationarybiasfromthe“normal”inﬂatedpublication scoreof27Croatianchemistryresearchers,then6oftheresearcherswerenolongerincludedamongthetop27(Fig.4).

PravdicandOluic-Vukovic’s(1991)resultshowsthatinflationarybiasnotonlyhasthepotentialtorearrangerankings,but thatnearly25%oftheprospectivecompetitorshadbeenprematurelyexcludedfromtherankingbecausetheyweremissed bytheinflatedrankingprocedure.Byanalogy,theirresultimpliesthatsomeleadingCanadianAquacultureresearchersmay havebeenprematurelyexcludedfromtherankingasaconsequenceofabiasedselectionprocedurerelyingoninflatedcredit scores(Picard-Aitken&Coté,2010).

Previousstudiesextollingtheadvantagesoffractionalcreditdidnotexaminetheeffectofequalizingbias(e.g.Aksnes etal.,2012;Gauffriauetal.,2008;Lindsey,1980;Piroetal.,2013;Põder,2010;Price,1981).Forexample,afterfractionalizing Piroetal.(2013)observedlargerankdisplacementwithatrendtowardsrankreversalwhenNorwegianresearchersin37 subfieldsofsciencewererankedaccordingtotheaveragenumberofpublicationsperpersonineachsubfield(Fig.4).But neitherPiroetal.(2013)norPravdicandOluic-Vukovic(1991)providedacomparativereferencerankingbasedonestimates ofactualcoauthorcontribution.Theirinflatedandfractionalrankingsarebothinfluencedbyequalizingbias,andobjective assessmentofaccuracyrelativetoabiasedreferencerankingisnotpossible(Fig.4).

Aggregaterankingsfromtheliteraturealsodocumentlargeeffectsoffractionalizing(e.g.Aksnesetal.,2012;Gauffriau

&Larsen,2005;Gauffriauetal.,2008;Huang&Lin,2011;Huangetal.,2011).Aggregaterankingsmayincludeadditional inaccuraciesbecausetheyrelyonproxyindicatorsofpublicationoutputwhichaccreditnationsorinstitutionsaccordingto avarietyofinexact“countingmethods”(Gauffriau&Larsen,2005;Larsen,2008).Such“methods”relyonimpreciseheuristic approximationsofauthorshipcredit,i.e.proxyindicatorsbasedonthekindofinformationthatcanbeconvenientlyextracted fromadatabase.Forexample,byaccreditingeverynationorinstitutionmentionedintheauthors’addresslistwithonefull

(7)

Fig.4.Comparisonoffractionalandinﬂatedrankingsfromtheliterature.PravdicandOluic-Vukovic(1991)rankedasubsetof27Croatianchemistry researchers.Thelightbluerectangleidentiﬁes6researcherswhowerenolongeramongthetop27whenpublicationscoreswerefractionalized.Piroetal.

(2013)rankedNorwegianresearchersin37subfieldsofscienceaccordingtotheaveragenumberofpublicationsperpersonineachsubfield.Inflatedranks areaffectedbyacombinationofequalizingandinflationarybias.Fractionalranksarecorrectedforinflationarybias.Questionmarksindicatethatthe effectofequalizingbiasisunaccountedforintheabsenceofarankingbasedonestimatesofactualcoauthorcontribution.Linesshowtheaddedeffectof inflationarybiasrelativetothefractionalranking.Horizontallinesindicatecoincidingranks,andcrossinglinesindicatedisplacementoffractionalranks causedbyinflationarybias.

credit,oroneequalfractionofcredit,irrespectiveofthenumberofcoauthorsfromtherespectivenationorinstitution,and irrespectiveofthecoauthors’actualcontribution.

Inconclusion,anyidentiﬁableamountofbiasinauthorshipaccreditationwasdetrimentaltotheaccuracyoftheranking oftheCanadianaquacultureresearchers’publicationoutput.Biasarosewhenpublicationscoreswerecalculatedwithout takingintoaccountinformation aboutmultipleauthorshipand differentialcoauthorcontributions.Theensuingbiased equalcreditscores,whetherfractionalorinﬂated,producedrankingsthatwerefundamentallydifferentfromtheranking ofharmonicestimatesofactualcreditbasedonallrelevantbylineinformationinthesourcedata.Theresultsdemonstrate thataccurateaccreditationofcoauthorsisthekeytoreliablepublicationperformancerankings.

Acknowledgements

ThankstoH.K.Marshallforimprovingthestyleandﬂowofthemanuscript,andthankstotheUniversityofNordlandfor facilitatingdataanalysisandmanuscriptpreparation.

References

Aksnes,D.W.,Schneider,J.W.,&Gunnarson,M.(2012).Rankingnationalresearchsystemsbycitationindicators.Acomparativeanalysisusingwholeand fractionalisedcountingmethods.JournalofInformetrics,6,36–43.

Buehring,G.C.,Buehring,J.E.,&Gerard,P.D.(2007).Lostincitation:Vanishingvisibilityofseniorauthors.Scientometrics,72(3),459–468.

DeBellis,N.(2009).Bibliometricsandcitationanalysis.Fromthesciencecitationindextocybermetrics.Plymouth,UK:ScarecrowPress.

Gauffriau,M.,&Larsen,P.O.(2005).Countingmethodsaredecisiveforrankingsbasedonpublicationandcitationstudies.Scientometrics,64(1),85–93.

Gauffriau,M.,Larsen,P.O.,Maye,I.,Roulin-Perriard,A.,&vonIns,M.(2008).Comparisonsofresultsofpublicationcountingusingdifferentmethods.

Scientometrics,77(1),147–176.

(8)

Hagen,N.T.(2008).Harmonicallocationofauthorshipcredit:Source-levelcorrectionofbibliometricbiasassuresaccuratepublicationandcitationanalysis.

PLoSONE,3(12),e4021:4021–4027

Hagen,N.T.(2010).Harmonicpublicationandcitationcounting:Sharingauthorshipcreditequitably–Notequally,geometricallyorarithmetically.

Scientometrics,84(3),785–793.

Hagen,N.T.(2013).Harmoniccoauthorcredit:Aparsimoniousquantiﬁcationofthebylinehierarchy.JournalofInformetrics,7(4),784–791.

Hodge,S.E.,&Greenberg,D.A.(1981).Publicationcredit.Science,213(4511),950.

Huang,M.H.,&Lin,C.S.(2011).Countingmethods&universityrankingbyH-Index.ProceedingsoftheAmericanSocietyforInformationScienceandTechnology, 48(1),1–6.

Huang,M.H.,Lin,C.S.,&Chen,D.Z.(2011).Countingmethods,countryrankchanges,andcountinginﬂationintheassessmentofnationalresearch productivityandimpact.JournalofTheAmericanSocietyforInformationScienceandTechnology,62(12),2427–2436.

Kendall,M.,&Gibbons,J.D.(1990).Rankcorrelationmethods(5thed.).London:CharlesGrifﬁn.

Kendall,M.G.(1938).Anewmeasureofrankcorrelation.Biometrika,30(1/2),81–93.

Kendall,M.G.(1942).Partialrankcorrelation.Biometrika,32(3/4),277–283.

Larsen,P.O.(2008).Thestateoftheartinpublicationcounting.Scientometrics,77(2),235–251.

Lindsey,D.(1980).Productionandcitationmeasuresinthesociologyofscience:Theproblemofmultipleauthorship.SocialStudiesofScience,10(2),145–162.

Maghsoodloo,S.,&Pallos,L.L.(1981).AsymptoticbehaviourofKendall’spartialrankcorrelationcoefﬁcientandadditionalquantileestimates.Journalof StatisticalComputationandSimulation,13,41–48.

Mattsson,P.,Sundberg,C.J.,&Laget,P.(2011).Iscorrespondencereﬂectedintheauthorposition?Abibliometricstudyoftherelationbetweencorresponding authorandbylineposition.Scientometrics,87,99–105.

Picard-Aitken,M.,&Coté,G.(2010).BibliometricanalysisofaquacultureresearchatDFOandinCanada.Finalreport.Montréal,Canada:Science-Metrix., http://www.science-metrix.com/pdf/SMDFOAquacultureResearch.pdf.

Piro,F.N.,Aksnes,D.W.,&Rørstad,K.(2013).Amacroanalysisofproductivitydifferencesacrossﬁelds:Challengesinthemeasurementofscientiﬁc publishing.JournalofTheAmericanSocietyforInformationScienceandTechnology,64(2),307–320.

Põder,E.(2010).Let’scorrectthatsmallmistake.JournalofTheAmericanSocietyforInformationScienceandTechnology,61(12),2593–2594.

Pravdic,N.,&Oluic-Vukovic,V.(1991).Distributionofscientiﬁcproductivity–Ambiguitiesintheassignmentofauthorrank.Scientometrics,20(1),131–144.

Price,D.D.S.(1981).Multipleauthorship.Science,212(4498),986.

Sylvain,C.(1993).Canadianresearchactivityinaquaculture:Abibliometricanalysis.Scientometrics,27(3),295–316.

Waltman,L.(2012).Anempiricalanalysisoftheuseofalphabeticalauthorshipinscientiﬁcpublishing.JournalofInformetrics,6(4),700–711.