Quantitative Research

ZRXOG DOVR EH YDOXDEOH WR GLVWLQJXLVK EHWZHHQ SXEOLF DFWRUV :LGHU FRQFHSWXDOL]DWLRQ ZRXOG WKXV EH LQWHUHVWLQJ EXW WKH VHOHFWHG GHILQLWLRQV DUH QRW D WKUHDW WR YDOLGLW\ TXLWH WKH FRQWUDU\ LV WUXH LQ WHUPV RI WKHLU RSHUDWLRQDOL]DWLRQ &RQVWUXFW YDOLGLW\ LV IXUWKHU LQFUHDVHG E\ XVLQJ PXOWLSOH VRXUFHV RI GDWD DQG PXOWLSOH UHVHDUFK PHWKRGV GDWD WULDQJXODWLRQ DQG PHWKRGVWULDQJXODWLRQ-RKQVRQ

External validity *HQHUDOL]DELOLW\ LQ D VWULFW VHQVH ZDV QRW WKH SXUSRVH RI HLWKHURIWKHTXDOLWDWLYHVWXGLHVLQWKLVWKHVLV:KHUHVWDWLVWLFDOJHQHUDOL]DWLRQ UHIHUV WR WKH JHQHUDOL]DWLRQ RI HPSLULFDO GDWD WR D SRSXODWLRQ WKH H[WHUQDO YDOLGLW\LVVXHLQWKLVFDVHLVZKHWKHULWLVSRVVLEOHWRJHQHUDOL]HWKHUHVXOWVWRD EURDGHU WKHRU\ *LEEHUW <LQ 2Q RQH KDQG WKH FDVH VWXG\ LQ SDSHU ,9 FDQ EH VHHQ DV D PXOWLSOH FDVH VWXG\ XVLQJ GDWD IURP YDULRXV RUJDQL]DWLRQV2QWKHRWKHUKDQGWKHFRQWH[WXDOIDFWRUVDWWKHPDFUROHYHODUH UHODWHGWRDVLQJOHFDVHLQRQHFRXQWU\$QDO\VHVDQGUHVXOWVDUHFORVHO\OLQNHG WR WKLV FRXQWU\ DQG IHZ DWWHPSWV DUH PDGH WR UDLVH WKH DQDO\VLV WR D PRUH JHQHUDODQGWKHRUHWLFDOOHYHO7KHGHVFULSWLYHFKDUDFWHUOLPLWVWKHFDVHVWXG\LQ WHUPV RI H[WHUQDO YDOLGLW\DQGLW FDQ RQO\ EHFRQVLGHUHGDVWDUWLQJ SRLQW IRU WKHRU\ GHYHORSPHQW FRQFHUQLQJ FRUSRUDWH JRYHUQDQFH DQG SXEOLF VHFWRU LQIOXHQFH

Reliability$UHFRPPHQGDWLRQWRHQKDQFHWKHUHOLDELOLW\RIWKHDQDO\VHV DQG HQKDQFH FDVH VWXG\ TXDOLW\ LV WR FRQVWUXFW D FDVH VWXG\ GDWDEDVH DQG WR VHSDUDWHWKHHYLGHQFHIURPWKHUHVHDUFKHU¶VLQWHUSUHWDWLRQLQILQGLQJV*LEEHUW HW DO <LQ 7KH TXDOLWDWLYH GDWD ZHUH RUJDQL]HG LQ :RUG DQG ([FHOILOHVDQGWKHTXDQWLWDWLYHGDWDLQDQ6366ILOH'RFXPHQWVZHUHXVHG IRUFRQWHQWDQDO\VHVWKDWLQYROYHGFDWHJRUL]LQJDQGFODVVLI\LQJVWDWHPHQWVDQG PDNLQJFKURQRORJLHVZLWKUHOHYDQWLQIRUPDWLRQ<LQ)RU3DSHU,9WKH VXUYH\ GDWD LQWHUYLHZV DQG DUFKLYDO GDWD ZHUH SODFHG LQWR RQH FDVH GHVFULSWLRQ ,Q WKH SDSHU WKH ILQGLQJV ZHUH SUHVHQWHG LQ WKH UHVXOWV VHFWLRQV ZLWKLQWHUSUHWDWLRQVRIILQGLQJVLQWKHGLVFXVVLRQVVHFWLRQVWRGLVWLQJXLVKWKHP FOHDUO\<LQ

5.7.2 Quantitative Research

7KH NH\ TXDQWLWDWLYH FRPSRQHQW RI WKLV WKHVLV LV WKH VXUYH\ )RU VXUYH\

GHVLJQ PXFK RI WKH YDOLGLW\ DQG UHOLDELOLW\ OLHV LQ WKH FRQVWUXFWLRQ RI WKH

VXUYH\ LWHPV DOWKRXJK TXHVWLRQV RI YDOLGLW\ RI VDPSOHV SURFHGXUHV DQG DQDO\VHVFDQDULVHLQDOOSKDVHVRIUHVHDUFK

7KHGHVLJQRIWKHVXUYH\DQGWKHSURFHGXUHPD\KDYHLQIOXHQFHGWKHUHVSRQVH UDWHVDQGUHVSRQVHSDWWHUQV7KLVFRXOGEHHLWKHULQWKHVHQVHWKDWUHVSRQGHQWV IRXQGWKHUDWKHUORQJTXHVWLRQQDLUHVWRRFRPSUHKHQVLYHDQGWLPHFRQVXPLQJ RU LQ WKH VHQVH WKDW WKH SURFHGXUH PDQDJHUV IRUZDUGLQJ WKH VXUYH\ WR WKHLU GLUHFWRUVPHDQWWKDWGLUHFWRUVLQWHUHVWHGLQSDUWLFLSDWLQJLQWKHVXUYH\GLGQRW UHFHLYHWKHLQLWLDOLQIRUPDWLRQDQGWKHOLQN+RZHYHUWKHUHVSRQVHUDWHVIURP ERWKPDQDJHUVDQGGLUHFWRUVZHUHVDWLVIDFWRU\7KHPDQDJHUVIROORZHGXSE\

VHQGLQJUHPLQGHUVWRWKHLUGLUHFWRUVDQGLWLVUHDVRQDEOHWRH[SHFWWKDWWKHYDVW PDMRULW\ RI GLUHFWRUV ZHUH JLYHQ DQ RSSRUWXQLW\ WR SDUWLFLSDWH 7KLV ZDV FRQGXFWHG DV D ZHEEDVHG VXUYH\ 3UHFRQWDFW FRPPXQLFDWLRQ SHUVRQDOL]HG FRQWDFWDQGQXPEHURIFRQWDFWVIROORZXSUHPLQGHUVDUHIDFWRUVDVVRFLDWHG ZLWK KLJKHU UHVSRQVH UDWHV LQ VXFK UHVHDUFK &RRN +HDWK 7KRPSVRQ

5HVSRQVHUHSUHVHQWDWLYHQHVVLVDQRWKHULPSRUWDQWDVSHFW&RRNHWDO 7KH VXUYH\ ZDV DGGUHVVHG WR DOO 1RUZHJLDQ WRXULVW RUJDQL]DWLRQV 7KH SURFHGXUHWKDWZDVFKRVHQVWDUWLQJZLWKGLUHFWWHOHSKRQHFRQWDFWZLWKDOOWKH RUJDQL]DWLRQV JDYH D JRRG EDVLV IRU LGHQWLI\LQJ WKH RUJDQL]DWLRQV DQG PDQDJHUVZKRFKRVHQRWWRSDUWLFLSDWH,QPRVWFDVHVWKHGDWDORJFRQWDLQHG H[SODQDWLRQVIRUODFNRISDUWLFLSDWLRQ7KHVHH[SODQDWLRQVZHUHV\VWHPDWL]HG ZKLFKVKRZHGWKDWODFNRIWLPHDQGORZDFWLYLW\LQWKHERDUGRIGLUHFWRUVZHUH FRPPRQ FDXVHV %DVHG RQ H[LVWLQJ NQRZOHGJH RI WKH RUJDQL]DWLRQDO GDWD RI WKHPLVVLQJUHVSRQGHQWVWKHUHLVQRUHDVRQWRVXVSHFWDQRQUHVSRQVHELDVIRU PDQDJHUV5HJDUGLQJGLUHFWRUV,KDYHLQVXIILFLHQWNQRZOHGJHRIWKHSRWHQWLDO UHVSRQVHV RI WKRVH ZKR GLG QRW DQVZHU DQG QRQUHVSRQVH ELDV HJ SDUWLFLSDWLRQELDVFDQQRWEHUXOHGRXW

7KH VXUYH\ FRQVLVWHG PDLQO\ RI VWDQGDUG LWHPV WKDW KDYH EHHQ WHVWHG IRU UHOLDELOLW\DQGYDOLGLW\LQSUHYLRXVVXUYH\V³9DOLGDWLRQLVDQRQJRLQJSURFHVV LQZKLFKYDULRXVVRXUFHVRIYDOLGLW\HYLGHQFHDUHDFFXPXODWHGDQGV\QWKHVL]HV WRVXSSRUWWKHFRQVWUXFWYDOLGLW\RIWKHLQWHUSUHWDWLRQDQGXVHRILQVWUXPHQWV´

=XPER &KDQS7KHXVHRIVWDQGDUGLWHPVUHGXFHGWKHULVNRI ELDVHGODQJXDJH GRXEOHEDUUHOHG TXHVWLRQV DQG XQEDODQFHG FDWHJRULHV RIWHQ DVVRFLDWHG ZLWK VXUYH\ UHVHDUFK :H HPSOR\HG EDFN WUDQVODWLRQ WKH

LQVWUXPHQWV ZHUH WUDQVODWHG IURP WKH VRXUFH ODQJXDJH (QJOLVK LQWR 1RUZHJLDQE\RQHWUDQVODWRUDQGWKHQWUDQVODWHGEDFNLQWR(QJOLVKE\DQRWKHU

*XWKHU\ /RZH 7KH VXUYH\ ZDV SUHWHVWHG RQ WZR ERDUGV RI GLUHFWRUV

$QDO\VHVRIWKHVXUYH\GDWDLQFOXGHGGHVFULSWLYHVWDWLVWLFVUHOLDELOLW\VWDWLVWLFV WHVWVRIVXLWDELOLW\IRUIDFWRUDQDO\VLVSULQFLSDOFRPSRQHQWVDQDO\VLVSURGXFW±

PRPHQWFRUUHODWLRQVPXOWLSOHUHJUHVVLRQDQDO\VLVDQGPHGLDWLRQDQDO\VLVDV SUHVHQWHGLQ3DSHUV,,DQG,,,

3DSHU,,UHSRUWVWKHGHYHORSPHQWRIWKH6WUDWHJLF2ULHQWDWLRQVFDOHIROORZLQJ WKHSURFHGXUHVXJJHVWHGE\&KXUFKLOOIRUGHYHORSLQJEHWWHUPHDVXUHV 7KHVHVXJJHVWHGVWHSVZHUHIROORZHGVSHFLI\LQJWKHGRPDLQRIWKHFRQVWUXFW WKURXJKUHYLHZLQJOLWHUDWXUHJHQHUDWLQJLWHPVWRFDSWXUHWKHGRPDLQWKURXJK IRFXVJURXSLQWHUYLHZVZLWKERDUGVDQGFROOHFWLQJVXUYH\GDWDIURPGLUHFWRUV LQRUJDQL]DWLRQVWRIXUWKHUDVVHVVDQGLPSURYHWKHPHDVXUHPHQW7KLVUHVXOWHG LQ WKH VFDOH PHDVXULQJ VWUDWHJLF RULHQWDWLRQV ZLWK D &URQEDFK¶V DOSKD FRHIILFLHQWIRUWKHVFDOHRIZLWKIRUVWDNHKROGHURULHQWDWLRQDQGIRU KROLVWLFRULHQWDWLRQ$OSKDFRHIILFLHQWVEHORZRXJKWWREHLQWHUSUHWHGZLWK FDXWLRQ &KXUFKLOO¶V YLHZ RQ DOSKD LV LQ OLQH ZLWK WKDW RI 1XQQDOO\

ZKRLQGLFDWHGWKDWQHZO\GHYHORSHGPHDVXUHVFDQEHDFFHSWHGZLWKDQ DOSKDYDOXHRIVKRXOGEHWKHWKUHVKROG

7KHGLPHQVLRQVIRXQGDUHFRQVLVWHQWZLWKSURSRVDOVLQUHFHQWWRXULVPUHVHDUFK

%HULWHOOL :DQJ :DQJ .UDNRYHU :DQJ ;LDQJ :LWK VDWLVIDFWRU\ FRHIILFLHQW DOSKDV DQG ZKHQ GLPHQVLRQV WKDW DJUHH ZLWKWKRVHKDYHEHHQFRQFHSWXDOL]HGWKHPHDVXUHLVVDLGWRKDYHDFFHSWDEOH FRQWHQWYDOLGLW\&KXUFKLOO:KDW&KXUFKLOOWKHQUHFRPPHQGV DQG3DSHU,,ODFNVLVDGGLWLRQDOWHVWLQJIRUZKLFKDQHZVDPSOHRIGDWDVKRXOG EH FROOHFWHG WR WDNH LQWR DFFRXQW HUURUV FDXVHG E\ IDFWRUV H[WHUQDO WR WKH LQVWUXPHQW 7KH ILQDQFLDO DQG WHPSRUDO IUDPH IRU WKLV UHVHDUFK DOORZHG QR URRPIRUDGGLWLRQDOGDWDFROOHFWLRQDQGWKHSDSHUWKXVODFNVWKHILQDOVWHSVLQ WKH VXJJHVWHG SURFHGXUH DVVHVVPHQW RI UHOLDELOLW\ FRQYHUJHQW YDOLGLW\ DQG GLVFULPLQDQWYDOLGLW\E\PHDQVRIDGGLWLRQDOGDWDFROOHFWHGIURPDQHZVDPSOH RI VXEMHFWV &KXUFKLOO $Q DOWHUQDWLYH FRXOG KDYH EHHQ WR GLYLGH WKH GDWDVHW LQWR WZR WR FURVVYDOLGDWH VSOLW VDPSOH YDOLGDWLRQ +RZHYHU DQ DUJXPHQWDJDLQVWWKLVZRXOGEHWKDWWKHVDPSOHLVDOLWWOHWRRVPDOO

2ZLQJ WR WKHVH VKRUWFRPLQJV ZH FKRVH WR SXEOLVK WKH UHVXOWV DV D UHVHDUFK QRWH3DSHU,,ZRXOGKDYHEHQHILWHGIURPJLYLQJH[SOLFLWDWWHQWLRQWRWKHODFN RI DVVHVVPHQW RI FRQVWUXFW YDOLGLW\ &KXUFKLOO QRWHV WKDW DSSOLHG UHVHDUFK LQ SDUWLFXODU FDQ EHQHILW IURP H[HFXWLQJ WKH ILUVW IRXU VWDJHV RI KLV SURFHGXUH WR JHW DQ LQGLFDWLRQ RI WKH TXDOLW\ RI D PHDVXUH DOWKRXJK KH LV FRQFHUQHGDERXWWKHVKRUWFRPLQJVWKLVHQWDLOV,WLVRQWKLVEDVLVWKDWWKHVFDOH ZDVGHYHORSHGDQGSXEOLVKHGDVDUHVHDUFKQRWHLQ3DSHU,,

,Q3DSHU,,,ZHVWXGLHGDPXOWLSOHPHGLDWLRQPRGHO0HGLDWLRQFDQEHWHVWHG E\ YDULRXV VWDWLVWLFDO PHWKRGV :H FRQVLGHUHG %DURQ DQG .HQQ\¶V UHQRZQHG VWHSV IRU PHGLDWLRQ EXW FKRVH WR DSSO\ =KDR /\QFK DQG &KHQ¶V UHFRPPHQGHG VWHSV IRU PHGLDWLRQ DQDO\VLV JHQHUDWLQJ ERRWVWUDS UHVXOWV E\ PHDQV RI WKH 3UHDFKHU DQG +D\HV 6366 VFULSW 7KH

%DURQDQG.HQQ\DQG=KDRHWDODSSURDFKHVDUHVLPLODUDVWKH\

XVHUHJUHVVLRQDQDO\VLVWRLQYHVWLJDWHKRZWKHLQGHSHQGHQWYDULDEOHVLQIOXHQFH WKH PHGLDWLQJ YDULDEOHV aSDWK DQG KRZ WKH PHGLDWLQJ YDULDEOHV LQIOXHQFH WKHGHSHQGHQWYDULDEOHbSDWK7KH\ERWKWHVWZKHWKHUWKHLQIOXHQFHRQWKH LQGHSHQGHQW YDULDEOH GLVDSSHDUV ZKHQ WKH PHGLDWLQJ YDULDEOHV DUH LQFOXGHG 7KH PDLQ DUJXPHQW RI =KDR HW DO IRU UHSODFLQJ %DURQ DQG .HQQ\¶V SURFHGXUH LV WKDW QR VLJQLILFDQW ]HURRUGHU HIIHFW RI WKH LQGHSHQGHQW YDULDEOH RQWKH GHSHQGHQWYDULDEOH c SDWK LV QHHGHG WR HVWDEOLVK PHGLDWLRQ FI %DURQ DQG .HQQ\¶V ILUVW FULWHULRQ $OO WKDW PDWWHUV LV WKDW WKH LQGLUHFW HIIHFW a [b SDWK LV VLJQLILFDQW ;KDR HW DO UHFRPPHQG XVLQJ WKH 3UHDFKHU DQG +D\HV 6366 VFULSW ODWHU QDPHG 352&(66 ZKLFK FDOFXODWHV WRWDO GLUHFW DQG LQGLUHFW HIIHFWV WRWDO DQG VSHFLILFDOO\ IRU HDFKPHGLDWRU7KLVDSSURDFKH[WHQGV%DURQDQG.HQQ\¶VFDXVDOVWHSV DV LWIRUPDOO\ WHVWV WKH VLJQLILFDQFH RIWKH K\SRWKHVL]HG PHGLDWHG HIIHFWV E\

PHDQV RI ERRWVWUDSSLQJ %RRWVWUDSSLQJ LV SUHIHUDEOH EHFDXVH LW GRHV QRW DVVXPH QRUPDOLW\ RI WKH GLVWULEXWLRQ RI WKH LQGLUHFW HIIHFWV DQG KHQFH SURYLGHV VWURQJHU SURWHFWLRQ DJDLQVW W\SH HUURUV 6WUXFWXUDO (TXDWLRQ 0RGHOLQJ6(0ZDVFRQVLGHUHGEXWQRWXWLOL]HGEHFDXVHRIWKHVDPSOHVL]H 6(0LVEDVHGRQFRYDULDQFHZKLFKLVRQO\VWDEOHZKHQHVWLPDWHGIURPODUJH VDPSOHV 0RGHOV ZLWK YHU\ FRPSOH[ VWUXFWXUHV FDQ EH PHDVXUHG ZLWK 6(0 EXWHYHQVPDOOPRGHOVUHTXLUHODUJHVDPSOHVPDNLQJVDPSOHVL]HWKHJUHDWHVW OLPLWDWLRQ IRU 6(0 7KH VWXG\ 1 GRHV QRW PDWFK VDPSOH VL]H UHTXLUHPHQWVFDOFXODWHGE\PHDQVRIDQDOJRULWKPDQGVRIWZDUHGHYHORSHGE\

:HVWODQG IRU FRPSXWLQJ ORZHU ERXQGV RQ VDPSOH VL]H LQ 6(0 7KH 3DUWLDO /HDVW 6TXDUH 3/6 DSSURDFK ZDV D SRVVLEOH RSWLRQ EHFDXVH WKLV PHWKRGLVQRWDVGHSHQGHQWRQODUJHVDPSOHV$V3/6DQDO\]HVUHODWLRQVKLSV LQ VWUXFWXUDO PRGHOV DQG LV IDLUO\ UREXVW ZLWK VPDOOHU VDPSOHV &KLQ WKLVDSSURDFKVKRXOGKDYHEHHQFRQVLGHUHGIRUWKLVVWXG\

)XUWKHUPRUHPHGLDWLRQZDVWHVWHGXVLQJFURVVVHFWLRQDOGDWDGHVSLWHWKHIDFWV WKDWPHGLDWLRQFRQVLVWVRIFDXVDOSURFHVVHVWKDWXQIROGRYHUWLPHDQGWKDWVXFK FURVVVHFWLRQDO H[DPLQDWLRQV RI PHGLDWLRQ KDYH EHHQ TXHVWLRQHG IRU ERWK SDUWLDO DQG FRPSOHWH PHGLDWLRQ &ROH 0D[ZHOO 0D[ZHOO &ROH 0D[ZHOO &ROH 0LWFKHOO +RZHYHU FURVVVHFWLRQDO WHVWV RI PHGLDWLRQDUH VWLOOWKH QRUP LQ SUHPLHUMRXUQDOVLQ D UDQJH RI SV\FKRORJLFDO GLVFLSOLQHV0D[ZHOO &ROH%HFDXVHWKLVVWXG\LVRQHRIWKHIHZRQ ERDUGVRIWRXULVWRUJDQL]DWLRQVZHFRQVLGHULWSULPDULO\WREHDVWDUWLQJSRLQW IRU VWXG\LQJ WKH PHGLDWLQJ UROH RI ERDUGURRP EHKDYLRU DQG FURVVVHFWLRQDO GDWDZHUHXWLOL]HGWRWHVWPHGLDWLRQHPSLULFDOO\

)LQDOO\ WKH TXDQWLWDWLYH SDSHUV RI WKLV WKHVLV UHO\ RQ GDWD FROOHFWLRQ FRQFHUQLQJ PXOWLSOH FRQVWUXFWV IURP D VLQJOH VRXUFH 'HVSLWH EHLQJ TXLWH FRPPRQLQVRFLDODQGEHKDYLRUDOVFLHQFHVVXFKUHVHDUFK PD\EHDVVRFLDWHG ZLWK VLQJOHVRXUFH HIIHFWV DQG FRPPRQ PHWKRGV YDULDQFH $YROLR

<DPPDULQR %DVV9DULDQFHWKDWLVDWWULEXWDEOHWRWKHPHDVXUHPHQW PHWKRG UDWKHU WKDQ WR WKH FRQVWUXFWV WKH PHDVXUHV UHSUHVHQW LV D SRWHQWLDO SUREOHP LQ EHKDYLRUDO UHVHDUFK DV RQH PHDVXUHV QRW UHDO UHODWLRQVKLSV EHWZHHQSKHQRPHQDEXWWKHUHVSRQVHFRQVLVWHQF\DQGPHPRU\RILQGLYLGXDOV 3RGVDNRII 0DF.HQ]LH /HH 3RGVDNRII ,Q UHWURVSHFW WKH XVH RI VLQJOH VRXUFHV LQ WKH TXDQWLWDWLYH SDUWV RI WKH WKHVLV DQG ODFN RI SURFHGXUDO DQG VWDWLVWLFDO FRQWUROV IRU VXFK PHWKRGV ELDVHV VKRXOG EH PHQWLRQHG DV OLPLWDWLRQVRIWKLVUHVHDUFK

6 Results

7KLVVHFWLRQVXPPDUL]HVWKHUHVXOWVRIWKHVWXGLHVLQWKHWKHVLV0RUHGHWDLOHG UHVXOWVFDQEHIRXQGLQWKHIRXUSDSHUVUHSRUWLQJWKHVWXGLHV