Increasing perseverance in math: Evidence from a field experiment in Norway

(1)

ContentslistsavailableatScienceDirect

Journal of Economic Behavior & Organization

jo u r n al ho me p ag e :ww w . e l s e v i e r . c o m / l o c a t e / j e b o

Increasing perseverance in math: Evidence from a ﬁeld experiment in Norway ^夽

Eric Bettinger

^a,c

, Sten Ludvigsen

^b,c

, Mari Rege

^c,∗

, Ingeborg F. Solli

^c

, David Yeager

^c,d

aStanfordUniversity,UnitedStates

bStenLudvigsen,UniversityofOslo,Norway

cUniversityofStavanger,Norway

dUniversityofTexas-Austin,UnitedStates

a rt i c l e i n f o

Articlehistory:

Received13December2016

Receivedinrevisedform18August2017 Accepted30November2017

Availableonline5December2017

Keywords:

Non-cognitiveskills Realeffortexperiment Growthmindset Perseverance Fieldexperiment Education

a b s t ra c t

Researchbypsychologistsandeconomistsdemonstratesthatmanynon-cognitiveskills aremalleableinbothchildrenandadolescents,butwehavelimitedknowledgeonwhat schoolscandotofostertheseskills.Inaﬁeldexperimentrequiringrealeffort,weinvestigate howschoolscanincreasestudents’perseveranceinmathbyshapingstudents’beliefsin theirabilitiestolearn,aconceptreferredtobypsychologistsas“mindset.”Usingprotocols adaptedfrompsychology,weexperimentallymanipulatestudents’beliefsintheirability tolearn.Threeweeksafterourtreatment,weﬁndpersistenttreatmenteffectsonstudents’

perseveranceandacademicperformanceinmath.Wheninvestigatingsubsamples,weﬁnd thatstudents,whopriortotheexperimenthadlessofabeliefintheirabilitytolearn, generatethetreatmenteffect.Theﬁndingssuggestthatalow-costinterventionfocusedon students’mindsetcanimprovestudents’engagementandperformance.

1. Introduction

Non-cognitiveskills,suchasself-controlandperseverance,predictsuccessineducationandinlabormarkets(Borghans etal.,2008;Heckmanetal.,2006;Robertsetal.,2007).Whileresearchersarestilltryingtounderstandthecausalmechanisms, thereisstrongevidencethatindividualswithhighernon-cognitiveskillsaremorelikelytograduatefromhighschool,have higherratesofcollegeattendanceandcompletion,higherwagesandbetteremployment,andevenbetterhealthoutcomes (Carneiroetal.,2007;Kautzetal.,2014).Moreover,researchbypsychologistsandeconomistsdemonstratesthatmany non-cognitiveskillsaremalleable(Alanetal.,2016;Durlaketal.,2011;Kautzetal.,2014).Still,however,wehavelimited knowledgeonwhatschoolscandotofostertheseskills.

Whilenon-cognitiveskillsmaybeacquiredthroughmotivationorself-regulationprogramsrelyingondirectinstruction andrepetitivepractice,meta-analysesﬁndthatthisapproachhasmixedresults.Itworksreasonablywellwithyoungchildren,

夽 WearegratefultoElinSvensenintheRogalandCountyschooldistrictwhohasfacilitatedtheRCTandgivenvaluablesuggestionsregardingintervention designandimplementation.WearealsogratefultoLeighLauritzenandNettop-UiSformakingthecomputerprogramfortheﬁeldexperiment,andtothe principal,teachersandstudentsatthehighschoolatwhichtheﬁeldexperimentwasimplemented.Finally,weacknowledgetheRCN(227004and260407) andtheNorwegianMinistryofEducationforfunding.

∗ Correspondingauthorat:UniversityofStavanger,4036Stavanger,Norway.

E-mailaddress:mari.rege@uis.no(M.Rege).

(2)

butshowsweakornulleffectswithadolescents(Durlaketal.,2011;HeckmanandKautz,2013;Yeageretal.,inpress).An alternativeapproachtodevelopingnon-cognitiveskills,testedhere,focusesonstudents’beliefs.Ratherthanthinkingof perseveranceasaﬁxedability,wethinkofperseveranceasaseriesofrepeatedchoices.Wedeﬁneastudentwithhigh perseveranceassomeonewhoconsistentlychoosestoexerthigheffort–shestaysfocusedonchallengingtasks,works hard,anddoesnotgiveup.Asachoice,perseveranceshouldbesensitivetobeliefsaboutthepayoffofeffort.Therefore, changingbeliefsabouteffort–inawaythatlastsovertime–mightresultinchangesinperseveranceevenafterarelatively minimalbelief-changeintervention.

Anextensiveliteratureinpsychologyandlaboreconomicshasdemonstratedthatstudents’beliefs,suchasself-efficacy, locusofcontrolorgrit,predicttheirperseveranceandchoices(AlanandErtac,2015;Bandura,1977;Cobb-Clark,2015;Cobb- ClarkandSchurer,2013;ColemanandDeLeire,2003;Duckworthetal.,2007;DuckworthandSeligman,2005).Although informative,thisresearchiscorrelational.Ouridentificationstrategyistorandomlyassignstudentstoawell-validated psychologicalinterventionthatisknowntochangebeliefsabouteffort:agrowthmindsetintervention(Dweck2006;Yeager andDweck2012),alsoreferredtoasan“incrementaltheoryofintelligence”intervention(DweckandLeggett,1988).Growth mindsetinterventionsshapestudents’beliefsintheirabilitytolearn,andcauselastingimprovementsinschooloutcomes (Aronsonetal.,2002;Blackwelletal.,2007;Goodetal.,2003;Pauneskuetal.,2015;Yeageretal.,2016).Intheinterventions studentslearnaboutthebrain’spotentialtogrowandchangethroughdedication,hardworkonchallengingtasks,finding therightlearningstrategies,andseekingassistancefromothers.Thegrowthmindsetinterventionisdesignedtocounteract the“fixedmindset,”whichisthebeliefthatintelligenceortalentsarefixedtraits.Bycontrast,studentswitha“growth mindset”believethattheirabilitiescanbedeveloped.

Inaﬁeldexperimentrequiringrealeffort,weinvestigatehowschoolscanincreasestudents’perseveranceinmathby shapingstudents’beliefsintheirabilitiestolearnorintheirpotentialtobeneﬁtfromeffort.Werelyontheweb-based mindsetinterventioninYeageretal.(2016),whichwasbasedonpriorworkbyAronsonetal.(2002)andPauneskuetal.

(2015).WeadaptedtheinterventiontotheNorwegianlanguage,cultureandcontext.Thetreatmentconditionexposes studentstogrowthmindsetthroughonlinereadingandwritingexercises.Theseexercisesfocuson(1)howintellectual abilitiesaremalleableandaccordinglyhowthebraincangrowandchange;(2)howhardworkonchallengingexercises improvestheneuralconnectionsinone’sbrain;(3)howgrowthmindsetaidsincopingwithconfusionanddifﬁculty;and (4)howgrowthmindsetcanbeusedtostrivetowardpersonalgoals,givingmorepurposetoexerteffortindifﬁculttasks (Yeageretal.,2014).Thecontrolconditionhasanalogousactivities,whichteachstudentsfactsaboutmemoryandbrain functioning,butdoesnotaddressthemalleabilityofintellectualability.

Inthespringof2016,385Norwegianhighschoolstudentsparticipatedinthefieldexperiment.Duringnormalschool instructionaltime,weintroducedstudentstoawebsite.Eachstudentloggedinindividuallytothewebsiteonpersonal laptops.Oncestudentsloggedin,oursoftwarerandomlyassignedthemtoeitherthemindsetorcontrolconditions.The studentsworkedontheonlinereadingandwritingexercisesduringtwosessionsof45min,twoweeksapart.Inthethird session,wegavestudentsarealefforttask,designedtocaptureafamiliarschoolactivity,inwhichperseveranceiscriticalto learnandsucceed.Specifically,studentsreceivedaseriesof34multiple-choicealgebraquestionssequentially.Thealgebra questionswerechallenging,possiblymakingmanystudentsfrustratedandtemptedtogiveup.Wetoldthestudentsthat wewouldlikethemto“dotheirbest”andthatthey“maylearnsomethingfromworkingonthemathquestions.”Wealso explainedcarefullythatstudents’answerswouldbekeptconfidentialandthattheirperformancewouldnotaffecttheir grade.Asmanystudentsdidnothavetimetofinishallthequestions,or“gaveup”–clickingfastthroughthequestions– welookathowmanycorrectanswersstudentshadonthefirst10,20andall34questions.

Theexperimentalresultsdemonstratethattreatedstudentshavesignificantlymorecorrect answersonthefirst10 questionscomparedtostudentsinthecontrolgroup(19percentofthestandarddeviation),butthereisnosignificant differencebetweentreatedandcontrolwhenlookingatthefirst20orall34questions.Subsampleanalysisdemonstrates thatthetreatmenteffectisentirelydrivenbystudentswhoinitiallyhadafixedmindset.Forstudentwhoenteredthe experimentwithapre-existingfixedmindset,thetreatmenteffectislargeandsignificantforallthreeoutcomemeasures;

treatedstudentsscored35,34and29percentofastandarddeviationhigherthancontrolstudentsonthefirst10,20andall 34questions,respectively.Forstudentswhoalreadyhadagrowthmindsetpre-treatment,therearenosignificanttreatment effects.Ourdescriptivestatisticsdemonstratethatpriortotheexperiment,studentswithlowgradepointaverage(GPA) andstudentsinvocationaltrackshavelessofagrowthmindset.Investigatingthesesubsampleswefindlargeandsignificant treatmenteffects.Amongstudentsinvocationaltracks,thetreatmentincreasedthescoreonall34questionsby25percent ofastandarddeviation.Ourresultsareconsistentwiththehypothesisthatitispossibletoincreasestudents’perseverance inmathbyshapingtheirbeliefsintheirabilitiestolearn,inparticularamongstudentswhoinitiallyhadafixedmindset.

Thispaperrelatestoseveralstrandsofliteratureineconomics.First,asnotedabove,ourworkbuildsontheemerging literatureonnon-cognitiveskillsbyinvestigatingwhetherschoolscanfosterstudents’perseverancebyshapingtheirbeliefs intheirabilitiestolearn.Thisisimportantbecausethestudyoffersanidentiﬁcationstrategyfortestingtheeffectofnon- cognitiveskillsonlifeoutcomes.Second,ourworkalsorelatestorecentdevelopmentsinbehavioraleconomicsofeducation (Kochetal.,2015;Lavecchiaetal.,2014)whichattempttounderstandhowlow-costbehavioralorpsychologicalinter- ventionscanhelpstudentsbetterutilizethelearningopportunitiesalreadywithintheeducationalsystem(e.g.Bettinger etal.,2012;Carrelletal.,2016;CastlemanandPage,2015).Wecontributetothisliteraturebyinvestigatinghowabrief, low-costpsychologicalinterventioncanleadstudentstoincreasetheireffortinalearningtaskthreeweekslater.Third,our mindsetexperimentbuildsonotherbehavioraleconomicsexperimentsdesignedtounderstandindividual’smotivationand

(3)

performanceinrealeffortchoicesinthelaborﬁeld(AzmatandIriberri,2010;Bradleretal.,2016;Erikssonetal.,2009;Koch etal.,2015;Kvaløyetal.,2015).

Finally,ourexperimentcontributestothepsychologicalliteratureonmindset.Thepsychologicalmechanismsformindset effectsareclearlydeﬁnedandstronglyillustratedinlaboratoryexperiments:growthmindsetinterventionschangestudents’

effortbeliefs,theirattributions,andtheirgoals(Burnetteetal.,2013).YetasarguedbyWilsonandButtrick(2016)andMiller etal.(2017),thebehavioralmechanismsthatexplainhowachangeinbeliefscantranslateintoachangeingradesmonthsor yearslater,arenotfullydocumented.Thismakes“longlag”interventioneffectsseem“magical”(YeagerandWalton,2011).

Weshinesomelightintothisbehavioral“blackbox”,bydemonstratingthatthemindsetinterventionaffectsperseverance inarealeffortlearningtaskwhichtookplacethreeweeksaftertheintervention.Whileourstudyfocusesonmindset,it buildsupontheextantliteraturedemonstratingthatbehavioraltrainingcaninﬂuenceacademicoutcomes(e.g.Alanetal., 2016;DuckworthandSeligman,2005;Durlaketal.,2011).

2. Background

2.1. Sampleandinstitutionalcontext

TheparticipantsinourexperimentareNorwegianﬁrstyearhighschoolstudentsinRogalandCounty.WefocusonNorway forthreedistinctreasons.First,todate,theexistingresearchonmindsetfocusesontheUnitedStates.Usingasampleoutside theUnitedStatescanadvanceunderstandingofmindset.ThecapabilityoftheNorwegiangovernmenttotrackstudents throughouttheirlifetime,givesthepossibilitytomeasurethelong-runimpactsofgrowthmindsets.Second,Norway,like manycountries,divideshighschoolstudentsintovocationalandacademictracks.Thistrackingisoftencorrelatedwithprior achievement,andasweshowbelow,thevocationalstudentsinoursamplehavesigniﬁcantlylowerincidenceofgrowth mindsets.Third,oursampleisalsodrawnoutofconvenience.MultiplecoauthorsliveintheRogalandregionofNorwayand haveexistingpartnershipsinimplementingeducationalinterventions.Atthetimeoftheresearch,educationalpolicymakers throughoutNorwaywerediscussingtheacademicresearchongrowthmindsetanditsimplicationsforNorway.

InNorway,students starthighschoolaroundage16 aftertenyearsofcompulsoryschoolingsimilarforeverybody (primaryandmiddleschool).¹Astudentcanapplytoanyhighschoolinhercounty.Whenapplyingastudenthastodecide whethertoenrollinavocationaltrack,whichleadsdirectlytoemployment,oranacademictrack,whichpreparesstudents toattendcollegeafterhighschoolcompletion.Studentsrankthreedesiredchoices,andacceptanceisbasedonstudents’

GPAfrommiddleschool.Allstudentsareguaranteedacceptanceintoahighschoolinthecounty.

Norwegianhighschooltypicallylastthreetofouryearsdependingonthespecificprogram.Only70percentofstudents completehighschoolwithinfiveyears.Forvocationaltrackstudents,completionratesareparticularlylow–only55percent completewithinfiveyears.WhileNorwegianpolicymakershaveaimedseveralreformsatimprovinghighschoolcompletion rates,therehavebeennosystematiceffortstoalterstudents’beliefsintheirabilitiestolearnasameansofimproving educationaloutcomes.

2.2. Conceptualframework

Wedeﬁneastudentwithhighperseveranceassomeonewhoconsistentlyexertshigheffort–shestaysfocusedon task,workshardonchallengingyetpotentiallyrewardingtasks,anddoesnotgiveup.Belowwepresentastylizedmodel illustratinghowshapingstudents’beliefsintheirabilitiestolearncanaffecttheirperseverance.

Consideralearningsituation.Astudentchooseshowmuchefforttoexert.Bystayingfocusedontask,workinghardand notgivingup,thestudentcanbeneﬁtfromlearning.However,theremaybeanopportunitycostofexertinghigheffort–for example,thestudentcannotcheckherphone,daydream,ortalktoaclassmate.

LetpB(e) representthestudent’sexpectedbeneﬁtoflearningwhenexertingeffort,e,whereB>0,B<0andpisthe student’sbeliefinherabilitiestolearn,i.e.howefforttranslateintobeneﬁtoflearning.LetC(e)representtheopportunity costofexertingeffort,whereC>0andC>0.Autilitymaximizingstudentchooseseffortlevele*suchthatC(e^∗)=p·B(e^∗).

Fromthisﬁrstordercondition,itfollowsthat ^de_dp^∗ >0and^d²^e^∗

d²p <0.Thus,wehavethefollowingtwoconjectures:

Conjecture1. Itispossibletoincreaseastudent’seffortbyincreasingherbeliefinherabilitytolearn.

Conjecture2. Theeffort-effectofincreasingthestudent’sbeliefinherabilitiestolearnishigher,thelowerthestudent’s initialbelief.

2.3. Academicmindsetinterventions

Inpsychology,astudent’sbeliefinherabilitiestolearnisreferredtoasheracademicmindset(Dweck,2006).Students witha“ﬁxedmindset”believetheirintelligenceortalentsareﬁxedtraits.Studiesusingsurveymeasuresofmindsetsand

1About9percentofstudentsinoursamplewereolderthan16,meaningthattheyeitherdelayedkindergartenentry,experiencedgraderepetitionat somepointintheiracademiccareer,or“tooktimeoff”aftercompulsoryschooling.

(4)

Fig.1. ContentofComputerProgram.

experimentalmanipulationsofmindsetsinlaboratoryandschoolsettingssuggestthataﬁxedmindsetshapesstudents’

academicachievementsinmanyways(Dweck2006;YeagerandDweck2012;YeagerandWalton,2011).First,students withafixedmindsetavoidacademicchallenges.Theywanteasierproblemsthatwillmakethemlookandfeelsmart(Mueller andDweck1998;Yeageretal.,2016).Second,afixedmindsetleadstounproductivebeliefsaboutefforts.Forexample,a studentwithafixedmindsetmightsay“IfIhavetotryhardatmath,I’mnotsmartatmath”(Blackwelletal.,2007).Last, fixed-mindsetstudentsarelessresilient.Instead,theyhidesetbacksanddeficiencies,notwantingpeopletoseethemas havinglowability.Theyfailtoaskforhelpandsometimesevenlieaboutlowscores(MuellerandDweck,1998).

Bycontrast,students with“growthmindsets”believethat intelligencecangrowand improveinresponsetoeffort, goodstrategies,andhelpfromothers.Fromthisperspective,anacademicchallengeisnotathreattoone’sability;itisan opportunityforlearningandimprovement.Inagrowthmindset,effortisagoodthing:astudentmightsay“tryingharder makesyousmarter”(Blackwelletal.,2007).Inthefaceofadifﬁcultproblem,agrowthmindsetstudentismoreresilient, seeksappropriatehelp,orswitchesstrategies.Thestudentdoesnothideconfusion.Comparedtoaﬁxedmindset,holdinga growthmindsetpredictsmorelearning,betterlearningstrategies,andhighergradesovertime,providedthattheschoolwork ischallenging(Blackwelletal.,2007;YeagerandDweck2012).

Thereissubstantialevidencesuggestingthatparentsandteacherssocializechildren’smindsetsthrougheverydaycom- munication(KaminsandDweck1999;MuellerandDweck1998;Rattanetal.,2015).Subtleverbalfeedbackfromadults canputchildreninaﬁxedmindsetandundermineinternalmotivation.Thiscanhappenevenfromvaluedcaregiverstrying toencouragechildren.Forexample,aclassicpaperbyMuellerandDweck(1998)showedthatpraisingyoungadolescents fortheirintelligence–tellingthemtheywere“smart”whentheydidwell−createdaﬁxedmindsetandunderminedtheir resilienceinthefaceoflaterstruggle.Incontrast,praisingstudents’“processes”(effortsorstrategies)putchildrenina growthmindsetandfosteredresilience.

Recentlyseveralstudieshavedemonstratedthatprecisetheory-basedinterventionscancommunicateagrowthmindset toyouthsandproducelastingimprovementsinstudents’grades(Aronsonetal.,2002;Blackwelletal.,2007;Goodetal., 2003;Pauneskuetal.,2015;Yeageretal.,2016).Theseinterventionsappealtoneuroscienceandevidenceonthemalleability ofthebrain.Tocommunicatethemalleabilityofintelligence,theseinterventionsusephysicalexerciseasametaphorfor growthmindset.Theinterventionsteachthestudentstothinkoftheirbrainsasmuscles,whichgetstrongerasoneexercises them.Theinterventiondepictsnewneuronalconnectionsgrowingasstudentscompletechallengingmathproblems.

3. Experimentaldesign 3.1. Interventionandmeasures

Wedevelopacomputerprogramwiththreeonlinesessions,eachlastingabout45min.Webasecontentandvisuallayout inSessions1and2ontheinterventioninYeageretal.(2016)(alsoseePauneskuetal.,2015).However,bythemeansofa professionaltranslatorandinterviewswithseveralfocusgroupsofNorwegianhighschoolstudents,wecarefullyadapted thematerialtotheNorwegianlanguage,cultureandcontext.Session3consistsofarealefforttaskinwhichthestudents havetosolveaseriesofalgebraquestions.Fig.1illustratesthecontentofthethreesessions.

InSession1studentsﬁrstanswersurveyquestionsdesignedtomeasurestudents’mindsetsatbaseline.²Inparticular, weaskhowmuch,onascalefrom1to6,thestudentdisagrees(1)oragrees(6)withthefollowingstatements(assigned variablenameinparenthesis):

2 Below,weonlylistsurveyquestionsusedforthispaper.Thestudentsreceivedothersurveyitems,designedtoanswerdifferentresearchquestions.

Allsurveyquestionswereidenticalfortreatedandcontrol.

(5)

Fig.2.ScreenShotfromComputerProgram.

•“Youhaveacertainamountofintelligence,andyoureallycan’tdomuchtochangeit”(FixedMindset1);

•“Yourintelligenceissomethingaboutyouthatyoucan’tchangeverymuch”(FixedMindset2);

•“Beinga‘mathperson’ornotissomethingthatyoureallycan’tchange.Somepeoplearegoodatmathandotherpeople aren’t”(FixedMindsetMath);and

•“Whenyouhavetotryreallyhardinasubjectinschool,itmeansyoucan’tbegoodatthatsubject”(FixedMindsetEffort).

Thesemindsetmeasureshavebeenusedandvalidatedinnumerousstudies,demonstratingthattheystronglypredict gradesandperformanceonbehavioraltasks(seee.g.Burnetteetal.,2013;Yeageretal.,2016)

Afterrespondingtothesurveyquestions,thestudentsreceivetheintervention.Thecomputerprogramrandomlyallocates studentstoeitherthetreatmentorcontrolconditions.Thetreatedstudentshavetodothreecognitiveexercises.First, studentshavetoreadanarticleaboutresearchinneurosciencethatdemonstratesthebrain’spotentialtogrowandchange, originallywrittenfortheexperimentinBlackwelletal.(2007)andsubstantiallyrevisedinYeageretal.(2016).Thearticle presentationrunsoverseveralscreens(oneofwhichappearsina singlescreenshotinFig.2)andhasastylizedvisual layoutwithillustrations.Italsousesthemetaphorthatthebrainislikeamusclethatgrowsinresponsetochallenging learningexperiences.Second,studentsareaskedtosummarizethearticleandexplainhowitsmessagerelatestotheirown lives.Linkinginformationtotheselfinthiswaymakesitmoreself-relevantandeasiertorecall(BowerandGilligan,1979;

HullemanandHarackiewicz,2009).Third,studentsareaskedwhatgrowthmindsetadvicetheymightgivetoafriendwho wasstrugglinginschool.Providingadvicetoothersisa“saying-is-believing”tactictoencouragestudentstointernalizethe ideasbyendorsingthemtosomeoneelse(seee.g.Aronsonetal.,2002).

Studentsinthecontrolcondition,likethoseinthetreatmentcondition,readabriefarticleaboutthebrainandanswer reﬂectivequestions.However,theydonotlearnaboutthebrain’smalleability.Instead,theylearnaboutbasicbrainfunctions andtheirlocalization,forexample,thekeyfunctionsassociatedwitheachcorticallobe.Theexperimentalconditionsare designedtolookverysimilartodiscouragestudentsfromcomparingtheirmaterials.Itinvolvesthesametypeofgraphic art–e.g.imagesofthebrain,animations−aswellascompellingstories.

InSession2,theinterventionusesinsightsfromthesocialpsychologyofattitudechangeandpersuasion,whatwecall

“supportivepsychologies”,toallowabrief,onlineinterventiontoachievelastingeffects.Thepurposeofthe“supportive psychologies”istocreateagestaltimpressionthatthegrowthmindsetmessageismemorable,credible,normal,andimpor- tant.Speciﬁcally,Session2repeatsthemetaphorfromSession1,thatthebrainislikeamusclethatgrowsinresponse tochallengingtasks(memorable);Itincludesquotesfromscientistsandcelebritieswhoendorsedthenotionthatthebrain developswhenitlearnsandexplainedhowastrongerbraincouldleadtoimprovementsinpersonalandsocialwelfare(credible);Itleveragessocialnormsbyincludingquotationsfrompastparticipantsendorsingtheprogrammessages(Cialdiniand Goldstein,2004)(normal);Finally,Session2emphasizesapurposeforlearning(Yeageretal.,2014)byincludingprosocial

(6)

beyond-the-selfmotivesforadoptingandusingagrowthmindset(Grant2013;Yeageretal.,2016)(important).Forexample, onescreenintheinterventionreadsasfollows:“Peopletellusthattheyareexcitedtolearnaboutagrowthmindsetbecause ithelpsthemachievethegoalsthatmattertothemandtopeopletheycareabout.Theyusethemindsettolearninschool sotheycangivebacktothecommunityandmakeadifferenceintheworldlater.”

Inadditiontoreinforcingthebriefmindsetintervention,the“supportivepsychologies”areimportantforthefollowing reason:Thisstudyisoneoftheﬁrstpapersineconomicsfocusingonmindset.Oneofourgoalsistoreplicatemindsetas truetotheoriginalpsychologyliteratureaspossible.Somepriorworkbyeconomists(e.g.Dee2014)hasfailedtoreplicate psychologicalinterventionspossiblyinpartbecauseofsubstantialdeviationsfromthepsychologicalintervention.Weare relyingontheexactscript(adaptedtoNorwegianlanguageandcontext)ofthelargestandmosteffectivemindsetstudyto date.

The“supportivepsychologies”dopresentsomeissuesintheinterpretationoftheresults.Inparticular,ourintervention inSession2couldaffectacademicperformancethroughsocialpressure:Theuseofcrediblerolemodelsandsocialnorms maycreatesocialpressureforstudentstoadoptgrowthmindsets.Weareunabletodisentanglewhethertheimpactsderive primarilyfromeducationaboutthemind(Session1)orfromthesocialpressureexertedinSession2.Bothsessionspoint studentstowardimprovinggrowthmindset,butwedonotidentifytherelativestrengthoftheeffectfromthetwosessions.

SimilartoSession1,thecontrolstudents’activityinSession2isdesignedtobeparalleltothetreatmentactivity.Students learnmoreaboutthebrain,butnotaboutitsmalleabilityandgrowthmindset.Ingeneral,wetookeveryprecautiontomake surethattherewasminimaltonocontaminationacrosstreatmentcategoriesduringimplementation.Ifsomecontamination occurredafterimplementation(forexampleifstudentstalkedtoeachotherabouttreatmentmaterial),thisislikelytobias ourestimatedimpactsdownward.

Aftertheinterventionmaterial,Session2providesallthestudentswiththesameseriesofsurveyquestions,measuring students’mindsets,asatthestartofSession1.Thereafterweincludeameasureofchallengeseekingwhichpriorresearch hasassociatedwithgrowthmindsets(Blackwelletal.,2007;MuellerandDweck1998).SimilartoYeageretal.(2016)we letstudentscreatetheirownmathworksheetwhichtheywillhavetoworkoninSession3.Studentscanpickfromeasy questionsfromwhichtheylikelywillnotlearnnewskills,orhardquestions,whichmayrequiremoreeffortbutprovide morelearningopportunities.Asmeasuresofstudents’challengeseeking,weusenumberofveryhardquestionsselected andnumberofveryhardorsomewhathardquestionsselected.ThequestionswereprovidedbyTheNorwegianDirectorate forEducationandTraining,andwecategorizedthemintoeasy,hardandveryhardbasedonpreviousstudents’scoreson eachquestion.

Finally,inSession3studentsﬁrsthavetosolvetworandomlydrawnquestionsfromtheworksheettheycreatedin Session2.³Aftertheworksheetquestions,thestudentshavetoparticipateinarealefforttask,consistingof34multiple choicealgebraquestions,givensequentially.⁴Thealgebraquestionswerechallenging,andseveralstudentsdidnothave timetoworkonallthequestions.Onaverage,thestudentsansweredcorrectly45,41,and37percentoftheﬁrst10,20and 34questions,respectively.

Theon-screenintroductiontothealgebraquestionstellsthestudentsthattheywillbegivenaseriesofalgebraquestions, andthattheyshouldtrytodotheirbesttoﬁndthecorrectanswer.Moreover,itexplainsthatstudentsmightlearnsomething fromworkingonthemathquestions,butthattheirperformancewillnotaffecttheirgrade.Asmanystudentsgaveuporran outoftimeonthelastquestions,weusenumberofcorrectquestionsontheﬁrst10,20,andall34questionsasmeasuresof students’effortonthealgebraquestions.

Notably,thestudentsdidnotknowthattheywouldreceivealgebraquestionsinSession3,sotherewasnowaytoprepare.

Moreover,algebrawasnotonthecurriculuminschoolbetweenSession1andSession3.Thus,ifweﬁndatreatmenteffect onthesemeasures,itisreasonabletointerpretitasaneffectofeffort(studentsaremorefocusedontask,workharderand donotgiveup)andnotthattreatedstudentshaveactuallybecomebetterinalgebra(althoughwecannotruleoutthatsome studentsmayhavebeensufﬁcientlymotivatedtopursueoutsidelearningexperience).

Apossibleconcernwithourrealefforttaskisdemandinducedeffects,thatstudentsintheexperimentalcondition increaseefforttocomplywiththeexperimenters’wishes,andnotbecausetheyhavemoreofagrowthmindset.Tolimit thispossibility,weprovidenopromptsbeforepresentingthetestquestionsinSession3,andtheformatofthesessionhas minimalremindersofthepriorsessions:Experimenterswerenotpresentintheclassroom,andthegraphicsonthetest questionsdidnotreiteratetheexperiment.Furthermore,effectscausedbyachangeinmindsetareexpectedtodifferacross subsamples(seeHypothesis2below),whereastherearenoclearreasontoexpectdifferentialdemandinducedeffects.

Asasecondarymatter,ourdataalsoallowustoinvestigatetimespentoneachalgebraquestion.Ontheonehand,we couldimaginethatstudentswithmoreperseverancemanagetostaymorefocusedandworkharderandhenceareableto movefasterthroughtheproblems.Ontheotherhand,giventhedifﬁcultyofthequestions,studentswithmoreperseverance mayhavespentmoretimetryingtosolveaquestionbeforegivingup.Theymayhaveelectedtotrydifferentapproaches

3 Wedidnotusethisdata,asselectioninSession2(treatmentaffectschallengeseeking)mayaffectperformanceonthesequestions.However,giving thestudentstimeinSession3,toworkonsomequestionsfromtheworksheet,isstillanimportantpartofthedesigntoavoiddeceptioninSession2.

4 Theﬁrst13questionswerethesameforallstudents.Thereafter,thecomputerprogramrandomlyassignedthestudentstooneofthreegroups,and eachgroupreceivedtheremainingalgebraquestionsindifferentorder.ThequestionswereselectedfromthepubliclyreleasedNAEPtestquestionsand theCaliforniaStandardsTestsquestions.

(7)

insteadofmakingarandomguessandmovingontothenextquestion.Assuch,wedonothaveaclearhypothesis,asto howourinterventionaffectedtimeuse.Wheninvestigatingtimespentonthefirst10,20or32min(resultsavailablefrom authorsonrequest),wefindnosignificanttreatmenteffectsneitheronthefullsamplenoronrelevantsubsamples.

3.2. Hypotheses

RecallfromthestylizedmodelinSection2.2,ifthetreatmentincreasesastudent’sbeliefinherabilitiestolearn,this increaseshermarginalbeneﬁtofeffort,andleadstoanincreaseintheoptimaleffortlevelinalearningsituation(Conjecture 1).Assuch,wehypothesize:

Hypothesis1. Thetreatmenthasapositiveeffectonastudent’seffortintherealefforttaskinSession3.

Thestylizedmodelalsodemonstratedthatanincreaseinastudent’sbeliefinherabilitiestolearn,increasesoptimal effortlevelatadiminishingrate(Conjecture2).Assuch,wehypothesize:

Hypothesis2. Treatmenteffectsoneffortintherealefforttaskarelargerforstudentswhoinitiallyhavelowbeliefsin theirabilitiestolearn.

Hypothesis2isalsoconsistentwithseveralstudiesdemonstratingthatinitiallylowperformingstudentsbeneﬁtmore frommindsetinterventions,andthataﬁxedmindsetismorecommonamongthesestudentspriortotheintervention (Pauneskuetal.,2015;Yeageretal.,2016).

4. Sampleandprocedures

Inthespringof2016,allﬁrstyearstudentsatapublichighschoolinruralNorwayparticipatedintheﬁeldexperiment.

Astheschoolservesalargeregion,itislargeandoffersbothavocationalandacademictrack.Participationwasmandatory aspartoftheschoolinstruction,butstudents hadtoconsenttoparticipateintheresearchproject.Whenthestudent loggedontotheﬁrstsession,theyreceivedinformationabouttheresearchprojectandhadtomaketheirconsentdecision.

Wehad458studentsparticipateintheﬁrstsession,amongwhom385studentsconsentedtoparticipateintheresearch project.

Afterastudenthadmadetheconsentdecision,werandomlyassignedthestudenttoeitherthemindset(treatment)or controlcondition.Amongthestudentswhoconsented,22studentshadmissingregistrydataonmiddleschoolgrades.

Another9 studentswere olderthan20 years old.⁵ Droppingthesestudents fromoursample resultedin aSession 1 sampleof354students.AbsenceisamajorconcerninNorwegianhighschools,andweexperiencedsomeattritionin Sessions2and3.FromourSession1sample,289and254studentsparticipated inSessions2and3,respectively.Our balancetest(seeTable1)demonstratesthatattritionwasnotsigniﬁcantlycorrelatedwithtreatmentstatus.Notably,we collecteddataforstudentsinSession3eveniftheyhadnotparticipatedinSession2,i.e.receivedthetreatmentreinforce- ment.

Weimplementedallsessionsinthestudents’classroomduringschoolhours.Thestudentsusedtheirownlaptopcom- putersandheadset.⁶Atthebeginningofeachsession,wereadabriefscripttoallthestudents.Wetoldthestudentsthat theywereabouttologontoacomputerprogramdesignedtolearnaboutthebrainandreflectonlearning.Weaskedthe studentstoworkindependentlyandnottalktootherstudents.Wealsoemphasizedthatstudentsshoulddotheirbestand thattheiranswerswouldbekeptanonymousandnotaffecttheirgrades.Weassuredthemthattheirteacherorschool wouldneverseetheirindividualanswers.Finally,wetoldthestudentsthatthesessionwouldlastfor45minandprovided themwithlogoninformation.InSession1andSession2,membersofourresearchteamadministeredtheprotocolwiththe teacherpresentintheclassroom.InSession3,theteacherswereresponsibleforimplementation.Weprovidedascriptto them.Membersofourresearchteamwerestillpresentattheschoolincasetheteachershadanyquestionsortechnical challenges.Inallthreesessions,ifstudentsfinishedpriortothe45min,theywereaskedtoworkonotherschoolwork.There arefouropen-endedquestionsintreatedSession1.Thefrequencyofstudentsputtingeffortintotheopen-endedquestions suggestthatmoststudentspaidattentiontothetreatmentmaterial.About85percentansweredsubstantively,whichwas definedasanyattemptofasincereanswer.

Studentsloggedonwithauniquestudentnumberandpasswordassignedtoeachstudentbytheschooldistrictadminis- tration.Teacherswereunawareofstudents’treatmentstatus.Forthestudentswhoconsented,theschooldistrictprovidedus withregistrydatautilizingthesameuniquestudentnumber.Theschooldistrictde-identiﬁedthedatabeforetheyprovided thedatatoourteam.Fromtheregistrydata,weemploythefollowingvariables:GPAandmathgrade⁷frommiddleschool,

5Thisimplieslaggingbehindregularschoolprogressionbyatleastfouryears.

6Alaptopismandatoryforschoolwork.Allthestudentshaveschooldistrictsubsidizedlaptops.Wehadsomeextraheadsetstolendtostudentswho didnothaveheadset.

7Sixstudentswereregisteredwithoutamathgrade.Thesemissingobservationswerereplacedbypredictedvalues(predictionbasedonbaselinemindset measures,GPA,genderandvocationaltrack).Thecorrelationbetweenpredictedandobservedvaluesare0.72.Resultsarerobustwithandwithoutthese students.

(8)

Table1

DescriptiveStatisticsandBalanceTest.

Session1 Session2 Session3

Control Treatment Difference Control Treatment Difference Control Treatment Difference

(1) (2) (3) (4) (5) (6) (7) (8) (9)

GPA 0.083 −0.081 −0.165 0.165 0.092 −0.072 0.224 0.141 −0.083

(0.916) (1.072) (0.106) (0.809) (0.899) (0.101) (0.892) (1.029) (0.121)

Mathgrade 0.066 −0.064 −0.130 0.130 0.062 −0.067 0.172 0.141 −0.031

(0.970) (1.026) (0.106) (0.940) (1.005) (0.115) (0.956) (1.029) (0.125)

Vocationaltrack 0.583 0.664 0.082 0.566 0.645 0.080 0.489 0.561 0.073

(0.494) (0.473) (0.051) (0.497) (0.479) (0.057) (0.501) (0.498) (0.063)

Female 0.440 0.525 0.085 0.441 0.548 0.107⁺ 0.511 0.595 0.084

(0.497) (0.500) (0.053) (0.498) (0.499) (0.059) (0.511) (0.492) (0.062)

Olderthan16 0.091 0.078 −0.013 0.055 0.027 −0.027 0.075 0.057 −0.017

(0.289) (0.269) (0.030) (0.304) (0.164) (−0.023) (0.265) (0.234) (0.031)

FixedMindset1 0.049 −0.047 −0.096 0.008 −0.062 −0.071 0.036 −0.108 −0.144

(0.996) (1.004) (0.106) (0.977) .(983) (0.115) (0.988) (0.993) (0.124)

FixedMindset2 0.041 −0.040 −0.081 0.046 −0.050 −0.097 0.027 −0.047 −0.074

(0.993) (1.008) (0.106) (0.988) (0.988) (0.116) (0.957) (0.979) (0.121)

FixedMindsetMath 0.047 −0.045 −0.092 0.040 −0.109 −0.150 0.029 −0.034 −0.063

(1.013) (0.987) (0.106) (0.974) (0.968) (0.114) (0.968) (0.951) (0.120)

FixedMindsetEffort −0.035 0.034 0.070 −0.111 0.000 0.112 −0.090 0.044 0.135

(0.965) (1.034) (0.106) (0.885) (0.987) (0.110) (0.906) (1.004) (0.121)

BaselineGrowthMindset −0.038 0.036 0.075 0.001 0.079 0.078 −0.005 0.054 0.059

(.1.020) (0.982) (0.106) (0.968) (0.936) (0.112) (0.984) (0.957) (0.122)

N 175 179 354 145 144 289 133 121 254

Notes:Foreachsessionsample,columnsprovidethemean(andstandarddeviation)forthecontrolgroupandthetreatedgroup,andtheestimatedcoefﬁcient (robuststandarderror)fromregressingeachcovariateagainsttreatmentstatus.⁺p<0.10,*p<0.05,**p<0.01.

Table2

CorrelationbetweenPre-TreatmentMindsetMeasures.

FixedMindset1 FixedMindset2 FixedMathMindset FixedEffortMindset

FixedMindset2 0.690**

FixedMathMindset 0.281** 0.425**

FixedEffortMindset 0.212** 0.322** 0.299**

BaselineGrowthMindset −0.765** −0.845** −0.696** −0.600**

Notes:⁺p<0.10,*p<0.05,**p<0.01.Session1sample(n=354).

highschooltrack(indicatorforvocational),gender(indicatorforfemale)andage(indicatorforlaggingbehindregularschool progression,i.e.beingolderthan16years).

5. Results

5.1. Balancetestanddescriptivestatistics

Table1presentsourdescriptivestatisticsandbalancetests.ThefourFixedMindsetmeasurescorrespondstothefour surveyquestionsasreportedinSection3.1.Thesevariablesarestandardizedwithmeanzeroandstandarddeviationone usingtheSession1sample,andapositivescoreindicatesafixedmindset.TheBaselineGrowthMindsetmeasureisthe meanofthefourfixedmindsetmetrics,wherethescaleisreversedandstandardized.Hence,apositivescoreindicatesa growthmindset.Fortheeaseofinterpretation,wehavealsostandardizedGPAandmathgradefrommiddleschoolusingthe Session1sample.Olderthan16isanindicatorforlaggingbehindregularschoolprogression.ThecolumnslabeledControl andTreatmentprovidesummarystatisticsforthecontrolandtreatmentgroupineachsession.IntheSession1control group,wecanseethatabout44percentofthestudentsarefemale;58percentareinthevocationaltrack;and9.1percent ofstudentsareoneortwoyearsolderthantheon-trackageof16.Allothercovariatesarestandardized.Whencomparing acrosssessions,thenumbersindicatethattheprobabilityofcompletingallsessionsaresomewhathigherforthosewith ahighGPA,beingonacademictrack,females,andhavingapre-interventiongrowthmindset.Weregresseachcovariate againsttreatmentstatus,andpresenttheresultingcoefficientandrobuststandarderrorincolumnslabeledDifference.We findthattherearesignificantlymorefemalesinthetreatedgroupinSession2(p<0.1),butallothercharacteristicsarewell balancedacrosstreatmentstatus.WeconcludethatrandomizationwassuccessfulandthatattritioninSessions2and3did notleadtosignificantdifferencesintreatmentstatus.

(9)

Table3

PredictorsofBaselineGrowthMindset.

(1) (2) (3) (4) (5) (6)

GPA 0.250** 0.139⁺

(0.052) (0.077)

Mathgrade 0.260** 0.179*

(0.051) (0.075)

Vocationaltrack −0.282* 0.053

(0.109) (0.129)

Female 0.151 0.082

(0.106) (0.105)

Olderthan16 −0.157 0.107

(0.191) (0.193)

R-squared 0.060 0.065 0.016 0.003 0.003 0.066

Notes:⁺p<0.10,*p<0.05,**p<0.01.Dependentvariable:BaselineGrowthMindset.Eachcolumnpresentsaseparateregressionandreportstheestimated coefﬁcient(robuststandarderror)forallincludedcovariates.Session1sample(n=354).

Table4

TreatmentEffectonPost-TreatmentMindsetandChallengeSeekinginSession2.

Post-TreatmentGrowthMindset Choosing“VeryHard”

ChallengeQuestions

Choosing“Hard”or“Very Hard”ChallengeQuestions

(1) (2) (3) (4) (5) (6)

Treatment 0.557** 0.545** 0.234* 0.285** 0.240** 0.300**

(0.113) (0.090) (0.117) (0.110) (0.117) (0.111)

GPA 0.055 0.040 0.119

(0.085) (0.105) (0.105)

Mathgrade 0.228** 0.378** 0.407**

(0.068) (0.084) (0.084)

Vocationaltrack 0.006 0.159 −0.006

(0.114) (0.140) (0.140)

Female 0.002 −0.468** −0.417**

(0.092) (0.113) (0.114)

Olderthan16 0.323 −0.204 −0.094

(0.225) (0.277) (0.277)

BaselineGrowth 0.522** 0.045 0.020

Mindset (0.048) (0.059) (0.059)

R-squared 0.075 0.435 0.010 0.144 0.011 0.141

Notes:⁺p<0.10,*p<0.05,**p<0.01.Firstrowliststhedependentvariable.Eachcolumnpresentsaseparateregressionandreportstheestimatedcoefﬁcient (robuststandarderror)forallincludedcovariates.Session2sample(n=289).Forcolumns(1)and(2):n=288.

InTable2wepresentacorrelationmatrixofourmindsetmeasurespresentedinSection3.1.FixedMindset1⁸andFixed Mindset2representtwodifferentwordingsofthesamequestion,and itshouldnotbesurprisingthattheyarehighly correlated.Wecanalsoseethatthesedirectmeasuresofaﬁxedmindsetarestronglycorrelatedwithourmeasuresof havingaﬁxedmindsetwhenitcomestoMath(FixedMindsetMath)andEffort(FixedMindsetEffort).Thesearemoreindirect measureswhichmeasuretheconsequencesofhavinglessofagrowthmindset.ThroughoutwewilluseBaselineGrowthas ourpreferredmindsetmeasure.

InTable3weinvestigatehowourpre-treatmentcovariatespredictsagrowthmindset.WecanseeinColumns1–3 thatthepresenceofgrowthmindsetseemstobesignificantlymorelikelyforstudentswithhighGPA/mathgradeandfor studentswhodonotattendvocationaltracks.VocationaltracksinNorwaygenerallyincludestudentswithloweracademic credentials.InColumn5weaddallpredictorstothesamemodelandwefindthatthereisonlyasignificantrelationship betweenGPA/mathgradeandgrowthmindset.

5.2. Treatmenteffects

InTable4weinvestigatetreatmenteffectsonoutcomemeasuresgatheredattheendofSession2.First,weseeifthe treatmentaffectedthemeasureofgrowthmindset.Wegatheredthesamemeasuresofmindsetpost-treatment aswe didatbaseline.Post-TreatmentGrowthMindsetisthenconstructedidenticallytoourBaselineGrowthMindsetvariable,and standardizedontheSession2sample.WecanseefromColumn1thatthereisalargeandsigniﬁcanteffectoftreatmenton

8Onestudentdidnotrespondtotheﬁrstﬁxedmindsetquestion.Thismissingobservationwasreplacedbythepredictedvalue(predictionbasedon baselinemindsetmeasures,GPA,genderandvocationaltrack).Thecorrelationbetweenpredictedandobservedvaluesare0.70.Resultsarerobustwith andwithoutthisstudent.

(10)

Table5

TreatmentEffectonEffortinSession3.

ScoreonFirst10Questions ScoreonFirst20Questions ScoreonAll34Questions

(1) (2) (3) (4) (5) (6)

Treatment 0.121 0.193* 0.057 0.134 0.012 0.089

(0.126) (0.089) (0.126) (0.085) (0.126) (0.086)

GPA −0.036 0.004 −0.028

(0.072) (0.068) (0.069)

Mathgrade 0.542** 0.582** 0.551**

(0.067) (0.063) (0.064)

Vocationaltrack −0.643** −0.542** −0.602**

(0.110) (0.103) (0.106)

Female −0.081 −0.134 −0.176*

(0.092) (0.086) (0.088)

Olderthan16 0.492** 0.464** 0.304⁺

(0.184) (0.172) (0.176)

BaselineGrowth 0.021 0.068 0.102*

Mindset (0.048) (0.045) (0.046)

R-squared 0.000 0.506 0.003 0.567 0.004 0.544

Notes:⁺p<0.10,*p<0.05,**p<0.01.Firstrowliststhedependentvariable.Eachcolumnpresentsaseparateregressionandreportstheestimatedcoefﬁcient (robuststandarderror)forincludedcovariates.Allincludedcovariatesarereported,exceptfromincolumns2,4and6whereweadditionallycontrolfor questionassignmentorder(twoindicators).Session3sample(n=254).

growthmindset.Indeed,treatmentincreasesthescoreby56percentofastandarddeviation.InColumn2wecanseethat thisﬁndingisrobusttocontrollingforourpre-interventionvariables,includingbaselinegrowthmindset.

Incolumns3–6,weinvestigatehowtreatmentaffectedchallengeseekingwhenstudentshadtocreatetheirownmath worksheetinSession2.AsexplainedinSection3.1,challengeseekingismeasuredasthenumberof“veryhard”or“somewhat orveryhard”questionschosen.Thechallengeseekingmeasuresarestandardizedwithmeanzeroandstandarddeviation oneusingtheSession2-sample.InColumn3,weseethattreatedstudentschosesigniﬁcantlymore“veryhard”questions comparedtostudentsinthecontrolgroup.Wecanseethattheestimateisrobusttocontrollingforourpre-intervention variablesinColumn4.Itsuggeststhattreatmentincreaseschallengeseekingby29percentofastandarddeviation.In Columns5and6,weusenumberof“somewhathard”or“veryhard”questionsastheoutcomemeasure,andwegetasimilar treatmenteffectonthismeasure.

TheeffectsizesinTable4areslightlylargerthansimilareffectsizesobtainedinevaluationsofthesamematerials conductedintheUnitedStates.Arecentevaluationwith76highschoolsandover16,000studentsfoundaneffectonself- reportedmindsetequalto34percentofastandarddeviationandaneffectonchallenge-seekingof23percentofastandard deviation(Yeageretal.,2016).Anevaluationin11highschools with3276studentsshowedaneffectonself-reported mindsetof30percentofastandarddeviation(Pauneskuetal.,2015).Thepresenteffectsseemtoexceedtheeffectsizesin theUnitedStates,perhapsbecausestudentspaidgreaterattentiontothematerialsintheNorwegiancontext.

InTable5,weinvestigatehowtreatmentaffectedeffortinthealgebraquestionsinSession3.Sincesomestudentsdid nothavetimetofinishallthequestions,or“gaveup”−clickingfastthroughthequestions,⁹welookathowmanycorrect answersstudentshadonthefirst10,20andall34questions.Themeasuresarestandardizedwithmeanzeroandstandard deviationoneusingtheSession3sample.InColumn1,weseenosignificantdifferenceineffortonthefirst10questions.

However,whencontrollingforbaselinevariablesinColumn2,wecanseethattreatedstudentshavesignificantlymore correctanswersonthefirst10questionscomparedtostudentsinthecontrolgroup.ConsistentwithHypothesis1,the estimatesuggeststhattreatedstudentsscored19percentofastandarddeviationhigherthancontrolstudents.Therewere nosignificantdifferencesbetweentreatedandcontrolwheninvestigatingtreatmenteffectsonthefirst20orall34questions inColumns3–6,suggestingnostrongtreatmenteffectontheoverallsample.

InTable6weinvestigatetreatmenteffectsfordifferentsubsamples.First,wecharacterizethestudentstohaveeithera fixedoragrowthpre-interventionmindsetbysplittingthesampleatthemeanofourmindsetmeasure.Consistentwith Hypotheses2,PanelsAandBdemonstratethatthetreatmenteffectdetectedinColumn2ofTable5isentirelydrivenby studentswhoinitiallyhadafixedmindset.Forthesestudents,thetreatmenteffectislargeandsignificantforallthree outcomemeasures.Theestimatessuggestthattreatedstudentsscored35,34and29percentofastandarddeviationhigher thancontrolstudentsonthefirst10,20andall34questions,respectively.Forstudentswhoinitiallyhadagrowthmindset, thereisnosignificanttreatmenteffect;theestimatedcoefficientisevennegativeinallbutonecolumn.

IntheremainingpanelsofTable6weinvestigateifreadilyobservablevariablesfromregistrydatacanhelpusidentify studentsparticularlyresponsivetotreatment.RecallthatTable3demonstratedthat,priortotreatment,aﬁxedmindsetis

9 Eighteenstudents(7percent)didnotﬁnishallquestions.Amongthe236studentswhoﬁnishedallquestions,theaveragetimeperquestiondeclined.

Theﬁrst13questionstookjustunder50sperquestion;questions14–20tookonaverageabout40sperquestion;questions21–27tookabout30sper question,andstudentsspentjustover20sperquestionontheremainingquestions.

(11)

Table6

TreatmentEffectonEffortinSession3.Subsampleanalyses.

ScoreonFirst10Questions ScoreonFirst20Questions ScoreonAll34Questions

(1) (2) (3) (4) (5) (6)

PanelA:Pre-TreatmentFixedMindset(n=130)

Treatment 0.297⁺ 0.348* 0.299⁺ 0.335** 0.245 0.285*

(0.178) (0.135) (0.170) (0.117) (0.170) (0.116)

AdjR-squared 0.014 0.434 0.016 0.538 0.008 0.540

PanelB:Pre-TreatmentGrowthMindset(n=124)

Treatment −0.094 0.022 −0.227 −0.059 −0.266 −0.129

(0.165) (0.114) (0.172) (0.119) (0.169) (0.121)

AdjR-squared 0.005 0.553 0.006 0.563 0.012 0.530

PanelC:VocationalTrack(n=133)

Treatment 0.234⁺ 0.272* 0.217⁺ 0.253* 0.222⁺ 0.251*

(0.133) (0.128) (0.117) (0.109) (0.119) (0.114)

AdjR-squared 0.016 0.131 0.018 0.189 0.018 0.143

PanelD:AcademicTrack(n=121)

Treatment 0.172 0.088 0.056 −0.029 −0.043 −0.110

(0.162) (0.122) (0.178) (0.116) (0.173) (0.121)

AdjR-squared 0.001 0.449 0.008 0.584 0.008 0.524

PanelE:LowGPA(n=129)

Treatment 0.180 0.286* 0.156 .226* 0.121 0.191⁺

(0.135) (0.128) (0.115) (0.108) (0.114) (0.106)

AdjR-squared 0.006 0.146 0.007 0.165 0.001 0.171

PanelF:HighGPA(n=125)

Treatment 0.148 0.074 0.053 −0.030 −0.006 −0.078

(0.165) (0.125) (0.169) (0.123) (0.174) (0.131)

AdjR-squared 0.002 0.458 0.007 0.496 0.008 0.459

Controlvariablesincluded No Yes No Yes No Yes

Notes:⁺p<0.10,*p<0.05,**p<0.01.Firstrowliststhedependentvariable.Eachpanelrepresentsadifferentsample.Forgivensample,eachrowpresentsa separateregressionandreportstheestimatedtreatmentcoefﬁcient(robuststandarderror)andadjustedR-square.Lastrowsindicatewhethertheregression includesthecontrolvariables.ThecontrolvariablesareGPA,mathgrade,vocationaltrack(indicator),female(indicator),olderthan16(indicator),and questionassignmentorder(twoindicators).

particularlyprevalentamongstudentswithlowGPAandstudentsinthevocationaltrack.Thisisalsoclearlycommunicated inthetabulationsinTablesA1andA2intheAppendixA.Thus,weinvestigatetreatmenteffectssplittingthesamplebased ontrackandGPA(atthemedian).¹⁰

InPanelC,wecanseethatamongstudentsinthevocationaltrackthereisalargeandsigniﬁcanttreatmenteffect.Looking attheﬁrst10,20andall34questions,thetreatmenteffectis27,25and25percentofastandarddeviation,respectively.

Studentsintheacademictrackscoredhigherthanotherstudents,andasweshowinPanelD,thereisnosigniﬁcanttreatment effectforthem.

InPanelEweinvestigatetreatmenteffectsforstudentswithalowpre-treatmentGPA.Weseethatthereisasigniﬁcant treatmenteffectonallthreeoutcomemeasures.PanelFdemonstratesthatthereisnosigniﬁcanttreatmenteffectonthe studentswithahighpre-treatmentGPA.

SinceourmostinterestingresultsareinthevocationaltrackandlowGPAsamples,AppendixATablesA3–A5providethe balancetests,inadditiontoeffectestimatesonSession2outcomes,forthesesubsamples.FromthebalancetestsinTables A3andA4wefindthatbothsubsamplesarewellbalancedacrosstreatmentstatusonallcharacteristics,withtheexception ofsignificantdifferencesinFemalesandOlderthan16inthevocationaltracksample,andinFemaleandVocationaltrackin thelowGPAsample.Hence,allourreportedestimatesarecontrolledforallobservables.TableA5demonstratesthatalsoin thesesubsamplestherearelargeandsignificanteffectsonthemeasuresofgrowthmindsetandchallengeseekinginSession 3,andthemagnitudesaresimilartothemagnitudesestimatedforthefullsample.

10ThecorrelationbetweenhavingahighGPAandchoosingtheacademictrackinhighschoolis0.56.Evenifthiscorrelationishigh,theoverlapisbyno meansperfect,andconsequentlyinformativetosplitthesamplebothbasedontrackandGPA.

Increasing perseverance in math: Evidence from a field experiment in Norway

Journal of Economic Behavior & Organization

Increasing perseverance in math: Evidence from a ﬁeld experiment in Norway 夽

Eric Bettinger

, Sten Ludvigsen

, Mari Rege

, Ingeborg F. Solli

, David Yeager

Increasing perseverance in math: Evidence from a ﬁeld experiment in Norway ^夽