From sensing to emergent adaptations: Modelling the proximate architecture for decision-making

architecture for decision-making

Sigrunn Eliassen

, Bjørn Snorre Andersen

, Christian Jørgensen

, Jarl Giske

aDepartmentofBiologyandHjortCentreforMarineEcosystemDynamics,UniversityofBergen,Bergen,Norway

bUniResearchandHjortCentreforMarineEcosystemDynamics,Bergen,Norway

a r t i c l e i n f o

Articlehistory:

Availableonline14September2015

Keywords:

Communityecology Evolutionaryecology Individual-basedecology Architecture

Heuristics

a b s t r a c t

Duringthepast50years,evolutionarytheoryforanimalbehaviourhasbranchedintodifferentmethod- ologicalframeworksfocussing onage-, state-,density-,and frequency-dependentprocesses.These approacheshaveledtovaluableinsightsinoptimalresponses,statedependentchoices,andbehavioural strategiesinsocialcontexts.Wearguethattimeisripeforanintegrationofthesemethodologiesbasedon arigorousimplementationofproximatemechanisms.Wedescribesuchamodellingframeworkthatis basedonthearchitecturalstructuresofsensingandinformationprocessing,physiologicalandneurolog- icalstates,andbehaviouralcontrolinanimals.Anindividual-basedmodelofthisdecisionarchitectureis embeddedinageneticalgorithmthatfindsevolutionaryadaptations.Thisproximatearchitectureframe- workcanbeutilizedformodellingbehaviouralchallengesincomplexenvironments,forexamplehow animalsmakebehaviouraldecisionsbasedonmultiplesourcesofinformation,oradapttochangingenvi- ronments.Theframeworkrepresentstheevolutionoftheproximatemechanismsthatunderlieanimal decisionmaking,anditalignswithindividual-basedecologybyemphasizingtheroleoflocalinformation, perception,andindividualbehaviour.

©2015TheAuthors.PublishedbyElsevierB.V.ThisisanopenaccessarticleundertheCCBY-NC-ND license(http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction

Mosttheoriesforanimalbehaviourhavetraditionallyassumed that individuals have accurate perception of the current envi- ronment,thattheyhavefullinformationonwhichtobasetheir decisions,andthattheymakeoptimalchoicesindependentoftime constraintsortheamountofcomputationrequired.Thisisincon- trasttoobservationsofanimalbehaviourwhereonewouldmost likelyconcludethatanimalsarenotsmartbutquiteoftendoclever things.Thisapparentclevernessmaystemfromtwo sourcesat differenttimescales:

(1)Animalsareflexibleastheye.g.canrespondfastandadequately insituationstheyhaveneverexperiencedbefore.Thissuggests thatbehaviouriscontrolledbyheuristics(Gigerenzer,2004), where theproximatemechanism (the decision-makingpro- cess)hasanarchitecturethatallowsefficientinformationuse anddecision-making.Thisarchitectureenablesminorchanges

Abbreviations: GA, genetic algorithm;GOS, globalorganismic state;IBM, individual-basedmodel;NR,neuronalresponse.

∗Correspondingauthor.Tel.:+4799205975;fax:+4755584450.

E-mailaddress:[email protected](J.Giske).

insensoryinput(e.g.strongersignalsofpredatorpresence)to leadtoverydifferentbehaviours(e.g.terminationoffeeding behaviour)orexperiencefromonesituationtobemadeuseful inanovelcontext.

(2)Smallevolutionarychangesinthisarchitecturemayalterthe behaviouralphenotypequitesubstantially(e.g.vanderPost andSemmann,2011a),verysimilartohowsmallmutationsin theregulationofdevelopmentalpathwayscanopenupmor- phologicaldiversityandinnovations(e.g.Moczeketal.,2011).

InthelanguageofTinbergen(1963),theproximatemechanism hasanarchitecturethatisparticularlygoodatevolvingasthe ultimatedriverschange.

Asaresultanimalbehaviouriscontrolledbyproximateheuris- ticmechanismsthatrestuponaninnovativearchitecture.Still,the proximatemechanismshavelargelybeenignoredinevolutionary andecologicalmodelsdespitethattheyare(i)whatevolve,(ii)what causetheemergenceofbehavioursonecanobserveinthewild andinthelab,and(iii)whatcanbestudiedintermsofneurology, physiology,biochemistry,andgenetics.Inthispaperwedescribe theproximatearchitecturefordecision-making,whichrepresents biologicalprocessesfromsensingviainformationprocessingand decision making through to the physiological and behavioural response.Wearguethatthisframeworkisusefulforunderstanding http://dx.doi.org/10.1016/j.ecolmodel.2015.09.001

0304-3800/©2015TheAuthors.PublishedbyElsevierB.V.ThisisanopenaccessarticleundertheCCBY-NC-NDlicense(http://creativecommons.org/licenses/by-nc-nd/4.

0/).

animal behaviour and that the proximate architecture can be incorporatedmechanisticallywithinanindividual-basedapproach.

Withproperrootinginevolutionaryadaptation,thismaybecome animportanttoolforevolutionaryandecologicalmodelling.

2. Behaviourintheindividual-basedparadigm

Evolutionary theoryfor animal behaviourhasbranchedinto differentmodellingframeworksfocussingonage-,state-,density- , and frequency-dependent processes, with little integration betweenmethodologies. The American statisticianAlfred Lotka was thefirst to model evolutionary adaptation and behaviour, byturningthepopulationgrowthequationofEuler(1760)into an equation for fitness (Lotka, 1907, 1925). In his interpreta- tionof what we nowcall theEuler–Lotka equation,competing resourceinvestmentsandactivitiesoftheorganismareevaluated withacommoncurrency:theircontributionstotheorganism’s expected rateof offspring production. The modellingparadigm isthereforebasedonthepremisethatorganismsmake optimal decisions. Thistradition has developed furtherinto life history theory(Fisher,1930;Murdoch,1966;Roff,1992;Stearns,1992;

Williams,1966),optimalforagingtheory(Emlen,1966;MacArthur andPianka,1966;Charnov,1976),gametheory(FretwellandLucas, 1970; MaynardSmith and Price, 1973), and adaptive dynamics (DieckmannandLaw,1996;Geritzetal.,1998;Metzetal.,1992).

While the Euler–Lotka equation, game theory and adaptive dynamicsare population-basedtoolstounderstandindividuals, optimalforaging theoryandstate-dependent lifehistorytheory (Mangel and Clark, 1986; McNamara and Houston, 1986) can be used and understood from a purely individual perspective.

Althoughthesemethodsareexcellenttoolsinevolutionaryecol- ogy,noneofthemareall-purpose.Optimizationtechniquesexcel atfindingthebestpossiblesolutiontoaproblemwithoutconsid- eringpotentialfitnessvalleyswhich maypreventtheoptimum itselffrombeingreached.State-dependent lifehistorytheoryis excellentforfindingoptimalpolicieswhentheydependonsome (physiological)stateoftheorganism,butattheexpenseofpolicies towardsotherindividuals.Theoppositeisthecaseforthegame theorytradition.

Inthe1970s adifferenttradition, basedonIndividualBased Modelling (IBM; DeAngelis and Grimm, 2014; Huston et al., 1988)arose. Thisparadigm mergedperspectives from artificial life(Langton,1986;vonNeumann,1966)andartificialintelligence (NewellandSimon,1956)withanothermajortraditioninmath- ematical ecology,community ecology. The earlypapers of IBM focusedonforestecosystems(Botkinetal.,1972;ShugartandWest, 1977,1980)andfishpopulations(DeAngelisetal.,1980).Thenew toolwasusedtostudypopulationandecosystemconsequences ofrarephenomena,suchasthedeathofacanopy-formingtreeor thesurvivalofalarvalfishthroughtheearliestlifestages.Fromthis beginning,theindividualbasedapproachhasgivenmoreflexibility inmodellingecologicalinteractions,byallowingdetailedrepre- sentationsofindividualslivingincomplexecologicallandscapes (GrimmandRailsback,2005;Stillmanetal.,2015).Thisisimpor- tantbecauseflexibleanddiversebehaviouralresponsesgenerally observedinnaturearenotfoundinsimplifiedmodels(Evansetal., 2012,2013;Fawcettetal.,2012,2014;McNamaraandHouston, 2009).Whileecologyandevolutionhasbeenintegratedinthearti- ficiallifetradition(e.g.Byrskietal.,2015;deBoerandHogeweg, 2012;Paredis,1995;Ray,1994),thelinktoevolutionarydynamics inindividual-basedmodelshasoftenbeenabsent(Grimm,1999).

Inthefollowing wewill discusssomefeatures oforganisms thatmay betakenadvantageof when modellingevolutionarily adaptive behaviours. These enable the integration of the ulti- mate perspective of optimization models with the proximate

mechanismsimportantinecologicalinteractions.Therearemany methodsavailableforthis,collectivelytermedmulti-scalemodels byHogeweg(2007).Wewillfocusontheproximatearchitecture framework,whichgives arepresentationofbiologicalprocesses from sensing via information processing and decision making through toaction for a wide range of animals.The framework canbeutilizedinecologicalmodelling,inparticularfororganisms thatmakedecisionsbasedonmultiplesourcesofinformation,in complex,variableandevennovelenvironments,wherelong-term fitnessconsequencesofbehaviouralchoicesareunpredictable.

3. Behaviouralcontrolthroughtheproximatearchitecture framework

In this section we describe some key elements in decision making and behavioural control in animals. Aswe move from theidealizedenvironmentstypicalfortheEuler–Lotkamodelsto moderately variable or complex environments, finding optimal responsestoallpossiblesituationswouldrequirehighlyadvanced (GoldsteinandGigerenzer,2011;McNamaraandHouston,2009) andenergeticallyexpensivebrains(Nilsson,2000).Naturalorga- nismsinsteadrelyonsimplerheuristicstohandlelargeamounts and different types of information (Gigerenzer, 2004; LeDoux, 1998).These‘rulesofthumb’havebeenselectedtoperformwell inavarietyofsituations,includingthoseneverencounteredbefore (HutchinsonandGigerenzer,2005).

Behaviouralcontrolinanimalsisorganizedasheuristicsembed- dedwithinanarchitectureofothermodulesorfunctionsofthe organism.Thearchitecturecanbedescribedasaseriesofweakly connectedsurvivalcircuits(LeDoux,2012)whichlinkperceptions tobehaviour.Alsoreferredtoas“theemotionsystem”,itplaysa centralroleinanimaldecisionmaking(Cabanac,1979;Leknesand Tracey,2008;Mendletal.,2009)throughevaluationofperceptions and selectionoftheinstrumentalbehaviouraland physiological responses(deWaal,2011;LeDoux,2000,2012;Panksepp,2005).

However,thisproximatearchitectureisnotlimitedtothosecon- ceptspsychologistscallemotions(Izard,2010),assurvivalcircuits existforalldrivesthatimpactattentionandbehaviouroftheorgan- ism(LeDoux,2012).Weemphasizethatinusingconceptsoften associatedwithhumanfeelings,particularlytheword“emotion”

withreference toLeDoux’swork,we do not imply anymental awarenessoftheseinternalprocessesinanimals.

The architectural structure described above can be imple- mentedinindividual-basedmodelling(Fig.1;Giskeetal.,2013), andinthefollowingwewillsketchtheprocessofdecisionmaking inthisframework,leavingthemoretechnicalmodellingaspects forAppendix.Weuseanexamplefromfishbehaviourtoillustrate theconcepts,butthisspecificformulationisonlyonepossibleway ofoutliningtheproximatearchitecture(seee.g.Eversetal.,2014, 2015).

3.1. Biologicalmechanisms

Specifictothearchitecturalapproachisthelevelofdetailofthe representationofthechainofeventsfromimmediateperceptions toinstrumentalbehaviour(Giskeetal.,2013;LeDoux,2012).First, allperceptions(includingsignalsfromwithinthebody)areevalu- atedinthebrain,wheredifferentcompetingneedsareweighed againsteach othertodeterminethemostimportanttask.Next, theorganismfocusesonsolvingthistask.LeDoux(2012)callsthis chainfromperceptiontobehaviourasurvivalcircuit,andanimals canhaveseveralsuchsurvivalcircuitsrunninginparallel.These mayforinstanceberelatedtohunger,thirst,sleepiness,curios- ity,andfear;thuswemaysaythesurvivalcircuitsarebundled, asillustratedinFig.1.Eachcircuithasdifferentmodules;hunger

Fig.1.Ageneralizedframeworkforbehaviouralcontrolthroughtheproximatearchitectureofdecision-making.Thebraincanholdseveralneurobiologicalstatessimul- taneously(hereexemplifiedwithhungerandfear).Thestrengthofthesestatesdependsonthecombinedneurologicalresponsesfromoneormorestimuli(internal,e.g.

stomachfullness,orexternal,e.g.behaviourofconspecifics).Eachneuronalresponsefunctionisdeterminedbygeneticallyinheritedtraits,whichgivesthepotentialfor differentrelationshipsbetweenstimuliinputandresponsestrengthbetweenindividuals(seeEq.(1)inAppendixandexamplesinFig.2).Inheritedtraitswillalsoopenfor emergentadaptationsduetonaturalselection.Therelativeweightgiventotheneuronalresponsesmayfurtherbemodulatedbyprocessesandstates(notshown),suchas sex,developmentalstage,physiologicalstate,andexperience.Thestrongestneurobiologicalstatedeterminestheglobalorganismicstate,inthiscasewhethertheorganism ishungryorfrightened.Wheninoneofthesestates,theattentionoftheorganismisfocussedonmakingthebestavailablephysiologicalandbehaviouralresponse.

mayforexamplebeevokedbyseeingfood,seeingotherseat,orby physiologicalneedsoftheorganism.Thesurvivalcircuitmustbe understoodoverseveraltimescales,withtheshortestonebeingthe state-dependentresponsetosensoryinformation,atintermediate scalesitincludeslearningandmemory(notincludedinFig.1),and thelongesttimescaleinvolvesgeneticadaptationofthegenepool.

Assuch,itbelongstothewiderclassofmulti-scalephenomena (Hogeweg,2007;vanderPostandSemmann,2011b;vanderPost etal.,2015).

3.2. Appraisalphase

The first phase, of retrieving and comparinginformation, is calledtheappraisalphase (LeDoux,2012).Foreachsurvivalcir- cuit,eachofseveralsensoryinputsevokesaneuronalresponsein thebrain(seeEq.(1)inAppendix).Theweightofagivenresponse maydependonmodulatorymechanisms (notshown inFig.1), forexampleshort-termeffectsofmemory,long-lastingeffectsof learning, and hormones more activein somelife stages of the organism.Theremayalsobelife-longeffectsofinheritedmetabolic propensities,forinstancelinkingmetabolicrate,fooddemand,and aggression,ormetabolicrateandriskwillingness(Houston,2010;

Realeetal.,2010),orduetomanipulationbyparasites(Barberand Dingemanse,2010;Barberetal.,2000).

Thesumofalltheneuronalresponsesinagivensurvivalcir- cuitdeterminesthestrengthof theneurobiologicalstateinthe brain,forinstancehunger,thirst,sleepiness,curiosity,orfear.The

neurobiologicalstatesmaycompeteinawinner-takes-allfashion wherethestrongestdeterminesthecurrentpsychologicalstateof theorganism,whichis calledtheglobalorganismicstate(GOS, LeDoux, 2012).The GOS is a stateof the wholeorganism, not onlythebrain,anditmayhaveneurological,physiological, and behaviouralmanifestations.ThemodeloutlinedinFig.1onlycon- siderstwostates;‘hunger’and‘fear’(seeEversetal.(2014,2015)for asimilarmodelwith‘fear’and‘like’inprimates).Whileitismath- ematicallystraightforwardtoexpandthemodeltoincludemore neurobiologicalstates,thebenefitofamorerealisticrepresentation needstobetradedagainstincreasedmodelcomplexity.

3.3. Responsephase

ThedeterminationoftheGOSisthestartofthesecondhalfof Fig.1,calledtheresponsephase(LeDoux,2012).Here,theorganism willhaveaphysiologicalandabehaviouralresponse,bothaimed atremovingorreducingthecauseoftheGOS.Ifthirsty,theanimal willseekwater,ifhungryitwilllookforfood.Inthisprocess,atten- tionrestrictionisanimportantpartofthephysiologicalresponse, wheretheanimalpaysmoreattentiontoinformationrelevantto itsGOSandless,ornone,toothercues(DukasandKamil,2000;

LimaandBednekoff,1999;Milleretal.,2012;PurserandRadford, 2011;Tombuetal.,2011).

Although mutually exclusive behaviours, such as vigilance and feeding, has been modelled using classical optimization approaches (e.g.Houstonet al.,1993; McNamaraand Houston,

1992),theexistenceofaGOS,attentionrestriction,andnarrow- ingofbehaviouraloptions,representamajorchangeofperspective introducedbytheproximatearchitectureframework.Thecostof attentionrestrictionislowersensitivitytootherstimuli:fright- enedfishmayforinstancehavelowerefficiencyincatchingfood (PurserandRadford,2011).Sohowrealisticisit?First,froman adaptationistperspective,thereisnobenefitinhavingarangeof alternativeglobalorganismicstatesiftheorganismcanexecute itsfullbehaviouralrepertoirefromwithinanyofthem.Themost likelypurposeofbeingsexuallyarousedorofbeingfrightenedisto narrowthefocusofattentiontosolveataskthatisimportantfor fitness.Secondly,theattentionrestrictionmechanismexistseven insophisticatedhumanbrains(Tombuetal.,2011),butthisdoes notmeanthatorganismscannotexperienceaspectsofseveralGOS simultaneously.InsomesituationsdifferentGOSarenotinopposi- tion,butrathercomplementary.Anexampleisthecombinationof

‘like’and‘fear’socialattitudestowardsothermembersofaprimate group(Eversetal.,2014,2015).

Themodel outlinedin Fig.1 doesnotgrade thestrength of theGOS,whileanimalsdo.Whetherananimalisterrifiedoronly alertedwilllikelyimpacthowlongitmaytaketoconsiderother neurobiologicalstates(throughmemory,whichisnotyetincluded inFig.1),andalsowhetheritcanconsidermorethanoneneurobi- ologicalstateatatime.Asticklebackcandynamicallybalanceits hungerdriveanditsperceivedpredationrisk(HellerandMilinski, 1979;MilinskiandHeller,1978)throughagradualregulationof itsattentiontowardspreyandpredators(Milinski,1985).Also,if allneurobiologicalstatesareweak,itmaybeadaptivefortheani- malnottofocus itsattention.Thishasbeentermed‘routinized behaviour’byGuilfordandDawkins(1987),andcantechnicallybe seenasyetanotherGOS.

3.4. Behaviouralpatterns

Giske et al. (2013) modelled spatial behaviour of an open- oceanmidwaterfishusingtheproximatearchitectureframework.

Theoverall model output described a fishpopulationperform- ing diel vertical migration, residing in shallowwaters at night andmigratingtodeeperwatersduringdaytime(ClarkandLevy, 1988;HugieandDill,1994;WernerandGilliam,1984),asobserved inpelagicplanktivores(BalinoandAksnes,1993;Goodsonetal., 1995;Kaartvedtetal.,2008;Torgersenetal.,1997)andmodelled (RoslandandGiske,1997;Stabyetal.,2013).Allmodelspredicta similarbehaviouralpatterns(dielverticalmigrationwithsomever- ticalextensionofthemigratinglayer),whichshowsthatassuming someinstantaneoustrade-offasinoptimizationandgamemodels orassumingattentionrestrictionandsequentialfocussingonthe mosturgenttaskmayresultinthesameoverallspaceusepatterns ofpopulations.

The similarity in movement patterns is mainly due of the strongstructuring forceof light in aquatic environments.Light decaysrapidlywithwaterdepthandimpactsdetectiondistances ofpredatorsaswellasthefish’sown encounterratewithprey (AksnesandUtne,1997).Modellingverticaldistributioneitherasa density-dependentgamethroughidealfreedistribution(Fretwell andLucas,1970)byignoringthephysiologicalstateofindividuals (Giskeetal.,1997)orasastate-dependentlifehistoryoptimiza- tion(MangelandClark,1986;McNamaraandHouston,1986)by ignoringcompetitioninaphysiology-drivenoptimizationmodel (FiksenandGiske,1995)resultedinthesameoverallpicture.The proximate architectureframework can onone handbeusedto evaluatetheimportanceofeachfactor,butitalsoallowsforeven moredetaileddescriptionsoforganismsandenvironments.Aswe willdiscussinthefollowing,oneofthemaindifferencesbetween classicaloptimizationandgametheorymodelsandtheproximate

architectureframeworkisthatcoexistingbehaviouraltypeswith dissimilarpreferencesandspaceusecanemergedinthelatter.

3.5. Consistentbehaviouraltypes

Incorporatingproximatecostsandconstraintsareessentialfor studiesofanimalpersonalities(Bell,2007;BiroandStamps,2008;

Budaev,1997;DingemanseandReale,2005;DingemanseandWolf, 2013;Sihetal.,2004;vanOersetal.,2005).Thegrowinginterest inanimal personalitiesarosefromobservationsof consistencies inindividualphysiologyandbehaviourovertime(Gosling,2001;

Houston,2010;McCraeetal.,2000),butthemechanismsunderly- ingthesebehaviouraldifferencesaregenerallyunknown(Fawcett etal.,2014,2015).Individualsinaproximatearchitectureframe- workarelessflexiblethan‘optimal’individuals.Theyarerestricted bylocalinformationandthearchitectureoftheirheuristics(inthis casetheirindividualneuronalresponsefunctions,seeAppendix) and do not behave according to unconstrained calculations of detailedoptimalpolicies.ThepopulationsmodelledbyGiskeetal.

(2013)evolvedtomaintaintwoormorevariantsofsomeneuronal responsefunctions,i.e.individualsshowedvariationin‘personal- ity’withastronggeneticbasis.Inparticular,consistentindividual differencesemergedalongthesocial/solitary-dimension,predom- inantlywhenhungry.Intheneuronalresponsefunctionthiswas relatedtohowthepresenceofcompetitorswasevaluatedduring feeding,someindividualshadallelescodingforweakdiscomfort, whileothershadneuronalresponsesthatgavestrongdiscomfort when manycompetitors werepresent(Fig.2).Individualswith strongdiscomfortwouldmoreoftenbefoundinlowdensitiesin theoutskirtsoftheverticallymigratingpopulation,whileindivid- ualswhotoleratedcompetitorswereusuallyfoundathighdensities nearthepeakfoodconcentration(Fig.2).Thus,inthearchitectural model,theemergenceofpersistentbehaviouraltypesledtoawider spaceuseofthefishpopulations.Coexistenceofbehaviouraltypes hasalsobeenobservedinevolutionarymodelswithoutrepresen- tationofproximatearchitecture,e.g.Eliassenetal.(2006)andvan derPostetal.(2015).

Theclassicaltextbookexplanationsfordiversitywithinpopu- lations are negative frequency-dependent selection (Ayala and Campbell,1974;Fisher,1930)andgene-environmentinteractions (Lewis,1954)in spatially or temporally variableenvironments.

However,anexplicitrepresentationofthebehaviouralarchitecture isinitselfsufficienttogenerateandmaintainphenotypicvariation (Giskeetal.,2014).Theultimatereasonisthatrandommutations in thegenesunderlyingthearchitectureoftheheuristics area sourceofinternalvariationamongindividuals,withthesameprin- cipaleffectsasexternalenvironmentalvariationfordifferencesin behavioursandphenotypes.

4. Evolvingsolutions:thegeneticalgorithm(GA)

Integratingecologicalandbehaviouralaspectsofthemodelwith evolutionarydynamicsis nota trivial task.Anindividual-based modelembeddedinageneticalgorithmisone wayof studying complex,near-realisticecologicalproblemsforwhichpuregame oroptimizationtechniquesarenotwellsuited.TheGAisapow- erfulalgorithmthatneedstobeusedwithcareandconsideration, andthemethodrepresentsashiftinfocusfromtheequilibrium solutionstoevolutionarychangeinvaryingfitnesslandscapes.

Avarietyofmethodologiescanbeusedtomodelbehaviourin individuals.Ifweassumethatorganismsdonotinteract,thenno feedbacksareinvolvedandoptimizationtechniquescanbeused to identifythe fitness peak (Clark and Mangel, 2000; Houston and McNamara,1999).Inmostcases,however, thebehavioural decisions of individuals have consequences for others, and the

Fig.2. Personalityarisingfromtheproximatearchitectureframework.Apopulationconsistsoftwogroupsoffish(left)whichdiffersignificantlyinonegenedetermining theneuronalresponsetowardscompetitorswhenhungry(seeFig.1).Thebluetype(centre)hasevolvedallelevalues(y=0.1–0.5,seeEq.(1)inAppendix)thatgive strongdiscomfortevenatlowcompetitordensities,whiletheredfish(right,y=9.7–10.0)moreorlessignorecompetitorsandseekthedepthwiththehighestinitialfood concentration.Thebluetypeishencefoundbothshalloweranddeeperthantheredfish,attheoutskirtsoftheverticallymigratingpopulation.

RedrawnfromGiskeetal.(2013).

way others behave will in turn influence the outcome of a behaviouraldecision.Withsuchfrequency-dependentfeedback, wecanusegametheoreticapproachesorhill-climbingalgorithms suchasadaptivedynamicsthatutilizelocalselectiongradientsto improvestrategiesuntiltheyreachanevolutionarilystablestrat- egy(DieckmannandLaw,1996;Metzetal.,1992).Thesemethods workbestwhenoneorafewtraitvaluescharacterizethestrat- egyofeachindividual.Amoredetaileddescriptionoftheorganism requiresseveraltomanyparameters,whichallowbehavioursto dependonsocial context andfeedbacksfrommultidimensional environments.Insuchcasesoneisoftenleftwithindividual-based simulationmodelsastheonlyfeasibleoption.

IBMs are based on one of the most significant biological structures:theorganism.Themostsignificantevolutionarychar- acteristicsofanorganismareitsexistenceandnumberofoffspring.

Thesebookkeepingtraits werealready usedby Euler(1760)to explain differences in the population growth of nations. The strengthoftheIBMapproachistoexpandthedescriptionofindi- vidualtraitstoincludechangesthroughanorganism’slifetimeand variationamong individuals.While thepedigreeofIBMmodels graduallyhasexpandedintoawidebush(GrimmandRailsback, 2005),weconcentrateonIBMasatooltostudypopulationsofhigh spatialandtemporalresolution.AnIBMallowseachorganismtobe tracedinspaceandtime,whichsimplifiesandimprovestherepre- sentationofinteractionsamongindividuals(GrimmandRailsback, 2005;Stillmanetal.,2015),anditprovidesalinktophenomenaand structuresinthephysicallandscape.Hence,thestrongrestrictions indescriptionsoforganismsingametheoryandofcompetitors orenvironmentalchangein state-dependent lifehistory theory (HoustonandMcNamara,1999;RailsbackandHarvey,2013)are muchrelaxedinIBMs(DeAngelisand Mooij,2003,2005).How- ever, a link to evolutionary adaptation is not ensured by the IBMalone. Forthis, a mechanism forheritability oftraits must beadded.

4.1. Thegenepool

Although“individual”byname,IBMsaretoolsforthestudyof populations(GrimmandRailsback,2005).Eveniftheindividualis thefocusofastudy,thegenepoolofthepopulationisthecontinu- ouslyevolvingentityfromwhichtheindividualgetsitsproperties.

WhenJohnHollandunderstoodtheadaptiveforceofnaturalselec- tionon thegene pool, he invented thegenetic algorithm (GA) as anevolutionarily mathematical equationsolver for complex

problems(Holland,1975).ThecentralideaofHolland’sGAwasthat thesolutiontoaproblemcouldbefoundbyiterationofcompeti- tionexperiments.Foreachnewgenerationofexperiments,those withthelowestqualitywerediscardedwhiletheremainingcould beslightlymodifiedbeforethenextround.Inthecontextofabio- logicalIBM,thismeansthatsomeparametervaluesoforganisms arecodedasgenes,whereindividualsthatfailtoreproducedonot passtheirgenesontothenextgeneration.Forthosethatdo,muta- tionsmayalterthegeneswhilesexualreproductionmixesgenes intonewcombinationsinfutureorganisms.

Thus,whiletheGAisoneamongmanymathematicalhillclimb- ingtechniques,it is uniquein itsabilityto mimicevolutionary processesincomplexenvironments.Itisthereforeanaturalexten- sionoftheIBMapproach(Fiksen,2000;Hamblin,2013;Higginson etal.,2015;HuseandGiske,1998;RuxtonandBeauchamp,2008;

Wood and Ackland, 2007). Where the Euler–Lotka equation is ananalyticalsolutiontoanimplicitlyevolvingpopulation(Lotka, 1925),theGAnumericallysimulatesagradualapproachtowards thesolution,similar toan explicitly evolvingpopulation. Com- bining IBM and GA simply means to run the IBM over many generations,passinggenesofthesurvivorsontotheiroffspring.

Inaddition,theGAhasthecapacitytoutilizealltheenvironmental andorganismicdiversitythatanIBMcanoffer.ThecombinedIBM andGAmodelsmayincludeprocessescentralinlifehistorytheory, gametheory,andcommunityecology.Whiletheecologicaltoolbox containswell-establishedmethodsforstudyingorganismal,social orenvironmentalcomplexity,thesituationisdifferentwhentwoor moreofthesecomplexitydimensionsneedtobeconsideredsimul- taneously.ThefactthatevolvingIBMsarenotrestrictedtostable environmentsovergenerations,stablepopulationsizes,andpopu- lationsofidenticalindividuals,showshowversatiletheyarefor studyingnaturalpopulations.Inaddition,strategiesorbehavioural rulesneednotbespecifiedinadvancebutcanemergeasaresult ofdifferentphysiologicalorbehaviouraltrade-offsorlife-history constraintsinvariousenvironments(BurtsevandTurchin,2006;

Fiksen,2000).

4.2. Geneticarchitecture

IntheGAcentraltraitsarecodedasgenesandinheritedbyoff- springofthesurviving andreproducingparent(Hamblin,2013;

Sumidaetal.,1990).Thereare,however,limitationsinthegenetic structuremadeinmostmodels,asourunderstandingofphysio- logicalanddevelopmentalprocessesisstillinsufficienttocreate

genotype-to-phenotypemaps.Broadlytherearethreeapproaches:

tomodelgenesandallelesexplicitlyasdonebypopulationgenet- ics(OttoandDay,2007), tousequantitativegeneticswhichcan incorporateexperimentally observed variance and co-variances betweenphenotypictraits(Dunlopetal.,2009;Lande,1976;Lynch andWalsh,1998),ortoassumeamorelooselinkbetweeninher- itedentityanditsphenotypiceffects butthenlosingtheability tomakepredictionsaboutactualratesofevolution.Oneflexible and commonapproach istoassume that genomesare haploid, andthatreproductionisasexualorarecombinationofthehaploid genomesoftwoindividuals(Hamblin,2013).Foragivenecological scenario,thesolutionfoundusingaGAmaythereforedependon howreproduction,recombination,andmutationareimplemented inthemodel(Hamblin,2013;RuxtonandBeauchamp,2008).

Whengenesareinheritedindependentlyofeachotherandmate choiceisrandom,anyalleleonanygenemaybepairedwithvery differentallelesofothergenesinthenextgenerations.Asacon- sequence, alleles thatare able topersistacrossgenerations (in theGA)arethosethatcansuccessfullyassociatewithmostother allelesinthegenepoolandstillformviableoffspring(Dawkins, 1976).Whilethismaybefineinecologicallandscapeswithone singlepeak, adaptationof thegene pool tomulti-peaked land- scapesmayrequirecoevolutionofspecificallelesofseveralgenes.

Thisismost easilyfacilitatedby co-locationonthesamechro- mosomesothattheallelesareinheritedtogether,oreveninthe sameregionofachromosometopreventseparationintheoffspring throughrecombination.Naturalorganismscommonlyhaveanon- randomarrangementofgenesontheirchromosomes,suppression ofrecombinationoutsidehot-spots(Myersetal.,2005),andanon- randommatingpattern.Ifthegenomeconsistsofmorethanafew genes,themodellerwouldneedtoconsiderwhichgenesshould belinkedthroughinheritance,andwhichtraitsarepreferredin matechoice.We illustratethisbytheGiskeetal.(2013)model offishbehaviour,wherethebehaviouraldecisionsareexplained inFig.1.Itisnotintuitivehow18genesof9neuronalresponse functions,4modulatorygenesandonegene forsexdetermina- tionshouldbearrangedononeorseveralchromosomes(Fig.3).

ThechromosomalarrangementofGiskeetal.(2013)wasthatthe twoparametersdescribinganeuronalresponsemadeuponechro- mosomewhichwasneveralteredbyrecombination.Hence,only mutationcouldchangetheshapeofaneuronalresponsefunction.

Approximatelyhalftheneuronalresponsechromosomesofanoff- springwasinheritedfromeachparent,whichmeansthatcomplex traitsinvolvingthecoevolutionofseveralneuronalresponsefunc- tionscouldnotevolveasstableentities.Similarly,themodulator genesandthesexdeterminationgeneconstitutedonechromosome inheritedwithoutinternalrecombination.Thismeansthatmodula- tionmaybecomesex-specific,andthatanysex-specificbehaviour hadtoberootedinthesemodulatorygenes.Alternativeconfigu- rationswouldbetoarrangegenesrelatedtohungerorfearonthe samechromosome,orallgenesintheappraisalphase(Fig.1)into oneandthoseintheresponsephaseintoanother.Suchalternative geneticarchitectureswouldthroughavoidanceofrecombination withinachromosomeallowcoevolutionofallelesandmaythereby impactthenumberofbehaviouraltypesoranimal personalities (BellandSih,2007;Dingemanseetal.,2010)whichcouldcoexist inapopulation.

4.3. Sensitivityanalysis

Theproximate architectureframework yields anelement of degeneracy(many-to-one-mapping,Wainwrightetal.,2005)for theevolvinggenepoolandforpersonalitytraits(Giskeetal.,2014):

OnecanthinkofFig.1asaroadmap,whereindividualsaswellas populationsdifferinwhichroutesarehighways.Forexample,an organismmaybehungrybecauseitsstomachisnotfullorbecause

itsappetiteincreaseswhenseeingfood.Hungermayalsobecome theGOSiftheorganismdoesnotsenseanypredatorsnearby,orcan hideamongconspecifics.Thisopensforseveralroutestostatesand behaviours,formanydifferentpersonalitytypeswithinapopula- tion,andformanydifferentlyevolvedpopulationstowardssimilar environments(Giskeet al.,2014).Such degeneracyisalsoseen inartificialneuralnetwork models(Duarteetal.,2011;Enquist and Arak,1994; Huse and Giske, 1998), in cellularinnovations (Wagner,2011)indevelopmentalprocessesofanimals(Doyleand Csete,2011;DraghiandWhitlock,2012;KirschnerandGerhart, 2005),andeven inthetranslation ofthegeneticcodetoamino acidsandproteins.Asdegeneracyisnaturalinbiologicalsystems (Wainwrightetal.,2005),apartofthesensitivityanalysisshould betocharacterizeitandstudyhowitaffectstheevolutionofthe populationandthebehaviouralresponsesthroughstandardized virtualexperiments.Here similarresponsepatternscouldresult fromdifferentpathways,andleadtocomparablelifehistorytraits, spaceuse,orpersonality.Evenwithdesktopcomputersanduser- friendlyprogrammingsoftware(e.g.NetLogo,Railsbacketal.,2006;

Sklar,2007)wecanusethesetechniquestotrainourintuitionabout effectsofchangingenvironmentsandproduceinterestingpredic- tionsthatcouldbetestedonnaturalpopulations(Eversetal.,2014, 2015).

Asanillustrationofthetypeofsensitivityanalysisperformed onmodelswithexplicitarchitecture,wehavetestedtheeffectof alteringthearrangementsofthe23genesintheGiskeetal.(2013) model(Fig.3).Whileonecanconstructplausiblebiologicalargu- mentsforallsixchromosomalarchitectures,thereisasfaraswe knownogoodaprioriwaytodetermineapreferenceforoneover theothers.Thebestresult,measuredbythepopulationeggpro- ductionwhichistheproductoffemalesurvivorshipandfecundity, wasobtainedbythefullyadaptivechromosomestructure(number 5). Threeof these six chromosomal arrangementsare approxi- matelyequalwithrespecttooffspringproduction.Hence,there arelikelymanyalmostequallygoodwaysofarranginggeneson chromosomes.Intwoarrangements,eggproductionwasconsider- ablylower(Fig.3),whichcallsforsomeinitialcautionandaneedto experimentwithchromosomestructure.Otherchallengesthatare notconsideredheremayselectfordifferentchromosomalarrange- mentsortheremightbegeneralchromosomestructuresthatare profitableunderarangeofenvironmentalconditions.Ratherthan predefiningdifferentarchitecturalconfigurations,onecouldalso letthechromosomalstructureevolve(e.g.CrombachandHogeweg, 2007,2008),whichwouldalleviatetheproblemofpredefiningand choosingageneticstructure.

4.4. Environmentalvariationandover-fitting

Whatever method usedto investigatethe effectof chromo- somalstructure,thereisariskofover-fittingthesolution(Tetko etal.,1995),sothattheevolvedgeneticstructureexcelsinthecur- rentscenariobutistoospecifictotackleotherrealisticscenarios.

Relatedtothisissueistheproblemofdecidingthenumberofgener- ationstorunintheGA.Thereisalwaysachancethatanimproved solutionwillbefoundaftermoregenerations,but thiswillalso increasetheriskofover-fitting.Environmentalstochasticity,for instanceintheformofintergenerational‘climatevariation’,may preventorreducetheriskofover-fitting(Giskeetal.,2013).Tofacil- itatecomparisonsbetweenpopulations,theevolvedgene pools couldbesimulatedinastandardenvironmenteveryngenerations, orthebehaviouralresponsesofthefinalpopulationdeterminedin standardizedtests.

Runningthemodelforonlyafewgenerationsmay,ontheother hand, leaveinsufficient time for the GAto locate peaks in the fitnesslandscape.Wecommonlyobserveasteepincreaseinfit- nessoverthefirstfewgenerationsintheGA,particularlyifthe

Fig.3. Sensitivitytochromosomalarrangement.(A)Arrangementsofgenescodingforneuronalresponses,modulationandsexon6differentchromosomestructures.The F,S,P,LandClabelsonchromosomesareforgenesrelatedtoneuronalresponsestofood,stomachfullness,predation,lightandconspecifics,respectively.(1)TheGiskeetal.

(2013)chromosomalarchitecture:onechromosomeforeachof9neuronalresponse(NR)functionsandoneforgenderandmodulation,intotal10chromosomes.(2)One chromosomeforgenderandmodulation,oneforallNRsrelatedtofear,andoneforallNRsrelatedtohunger,intotal3chromosomes.(3)As(2),butseparatechromosomes forfearandhungerinappraisalandresponsephasesofFig.1,intotal5chromosomes.(4)Onechromosomeforgenderandmodulation,oneforallNRsrelatedtofood, conspecificsandlight,respectively,andseparatechromosomesforthetwoNRsrelatedtopredatorsandstomachfullness,intotal6chromosomes.(5)Threeequallylarge adaptivechromosomes,whereeachgenecanmutateposition.Offspringwillinheritthechromosomestructureofitssame-sexparent,andtheprobabilityofinheritanceof all(ornone)genesonthechromosomefromsame-sexparentis100%,75%,and50%onchromosomes1–3,respectively).(6)As(5),butwherebothgenesinoneNRmustbe onthesamechromosome.(B)Theeggproductioninfishpopulations(fromtheGiskeetal.(2013)model)withthesechromosomalarrangements.Eachsimulationisinitiated with10,000fishlarvaeandrunfor30,000generations,withsubstantialstochasticenvironmentalvariationatshort-termandgenerationaltimescales.Thebarsshowthe meaneggproductionand95%confidenceintervalamong10replicatesimulationsbetweengeneration20,000and30,000.

populationisseededwithrandomallelevalues.Inthefishmodel ofGiskeetal.(2013,2014),populationeggproductionlevelledoff afteraround1000generations,andAndersen(2014)didnotfind qualityimprovementsinthemodeloutputafter1000generations.

Asthereisalwaysaninherentuncertaintyinevolutionarymod- ellingwhetherthealgorithmhasarrivedatornearafitnesspeak, werecommendrunningmultiplepopulationswithsomeverylong

simulationstodetermineareasonablenumberofgenerations.The adaptationperioddependsonthegoalofthemodelling,asthetime toadaptatraitdependsonthefitnesscostsofsuboptimalsolu- tions(Fisher,1930).Lifehistory,growth,orspaceusepatternsin theGiskeetal.(2013)modelgenerallyconvergeearlyinsimula- tionswhereasgeneticsorpersonality-relatedtraits,takeslongerto stabilize.

4.5. Effectsofimplicitassumptions

Thereproductionschemewillinfluencetheevolutionarypro- cessaswellastheoutputofthemodel.Matechoiceisimportantin sexuallyreproducingorganisms,anddifferentrulesformateselec- tionmaybeimplemented inecologicalmodelling.Forinstance, Giskeetal.(2013)allowedfemalestochooseamateamongmales locatedatthesamedepth.Femalesmatedwiththelargestofthe firstthreemalesencounteredandthisresultedinsexualselection favouringfastergrowthandmorerisk-pronebehaviourinmales.In othermodellingscenarios,femaleshadnomatepreferencewhich resultedinsmallermalesthatwererewardedforhighersurvival ratherthanfastergrowth(Giskeetal.,2014).Asillustratedbythis example,itisquitelikelythatpatternspredictedbyamodelmay dependonfactorsnotconsideredbythemodellerorpresentedto thereader.ThisproblemisnotspecifictoGA,butbecausethealgo- rithmisdesignedtosolvecomplexproblems,asideeffectisthat isalsofindsandexploitsweaknessesinsuchassumptions.Atthe otherendofthemodelcomplexityspectrum,organismsarerep- resentedbysimpleprocessesandfewvariables.Whencomparing thepredictionsofthesemodelstonaturalsystems,theprocesses thatarenotconsideredinthemodelneedtoberepresentedby fixedparametersorthefewprocessesthatareexplicitlymodelled.

Inpractice,thisimpliesthatmanyofthedegreesoffreedomina complexmodelarehardwiredinasimplerone.

4.6. “Handlewithcare”

While our methodology is one step nearer the behavioural architectureofnaturalorganisms,itmayalsobecomputationally moreintensiveandrequireattentiontotechnicaldetails.Think- ingintermsofarchitectureofbehaviouralcontrolwillbeuseful forawiderangeoftheoretical,experimentalandfieldstudies,but importantinsightscanalsobeobtainedthroughindividual-based evolutionarysimulationswithoutbeingexplicitabouttheproxi- matearchitecture(e.g.Higginsonetal.,2015;Hogeweg,2007;van derPostandSemmann,2011b;vanderPostetal.,2015;Woodand Ackland,2007).UsingGAtosimulatebehaviouralresponsesmay invokecostsandbenefitswhichyielda surplusonlywhenana- lyticalmethodsare insufficient(Ruxtonand Beauchamp,2008).

Such situationsare whenenvironmentalvariation isimportant, orwhenbehaviourissimultaneouslyimpactedbyarangeoffac- torswhichmakespureoptimizationorpuregameapproachestoo simplistic. However, it is hard to anticipate what is lost when using simpler models. As discussed above, a model of vertical migrationwithexplicitarchitecturewasnotonlyabletocombine density-dependentandstate-dependenttrade-offs,italsorevealed apotentialforverticalstructuringbasedonconsistentbehavioural types.

5. Individual-basedecology:buildingonthegenepool TheAlfredLotkatraditionin evolutionaryecologyhasledto elegant analytical and numerical findings through life history theory (Fisher, 1930; Mangel and Clark, 1986; McNamara and Houston,1986;Murdoch,1966;Williams,1966),optimalforaging theory(Emlen,1966;MacArthurandPianka,1966),andgamethe- ory(FretwellandLucas,1970;MaynardSmithandPrice,1973).

While there are many reasonsfor continuing the development ofthesemethods,itisnotstraight-forwardtocombinethemin populationmodels(HoustonandMcNamara,1999;Railsbackand Harvey, 2013). While Individual-based Neural network genetic algorithms(HuseandGiske,1998),hedonicmodelling(Giskeetal., 2003),and eco-geneticIBMs(Dunlopet al.,2009)canintegrate acrossage-,state-,anddensity-dependentprocesses,andsolvethe

computationalchallenge,theproximatearchitectureframework describedinthispaperisbasedonmechanismsofdecisionmaking foundinawiderangeofnaturalorganisms(Cabanac,1979;Macnab andKoshland,1972;Mendletal.,2009;Stocketal.,1989)andit canalsobeimplementedinmodels.

Modelsthatconsidertheeffectofindividualvariationandevolv- ing gene pools leadto two important insights that differ from predictionsbasedontraditionaloptimalitymodels.First,asorga- nisms need to solve immediate problems and respond to the currentandlocalenvironmentalconditions,theirresponsesmay notbeoptimalintheclassicalsense.Strategiesthatarerobustand performwellunderavarietyofsituationsareoftenmorebeneficial inthelongrun(YoshimuraandClark,1991;YoshimuraandJansen, 1996)andinter-dependenciesbetweenbehaviour,physiologyor life-historytraitsmayallowforseveralcoexistingsolutionstosim- ilarproblems(MangelandStamps,2001;Stamps,2007).Next,at thepopulationlevelthereisasimilaranalogy,asthediversityinthe evolvedgenepoolhasevolvedforbeingresilienttochangingcondi- tionsandthecoarsercontoursofthefitnesslandscape.Theadaptive diversificationwithinthegenepoolorbetweenpopulationsisfacil- itatedbytheproximatearchitecturefordecision-making,which allowsmanydifferentbutequallyvaluablepathwaysfromsensing throughtobehaviour(Giskeetal.,2014).

Humanimpactontheplanetmakesitincreasinglyimportantto understandcomplexproblemsandinteractionsonmultiplescales.

Community ecologyis thesum of social interactions and local environmentaleffectsonsingleindividuals,whichagainareconse- quencesofadaptedandoftencoevolvinggenepools.Theproximate architectureframeworkisnotonlyausefultoolforfindingadaptive behaviourincomplexsituations,butalsoabetterrepresentation ofthebehaviouralresponsesandtheunderlyinggeneticarchitec- turetypicalfornaturalpopulations.Asamodellingframeworkit mayalsobridgetheindividualfocusofbehaviouralecologywith thepopulationfocusinmanyotherecologicaldisciplines.

Acknowledgements

WethankUtaBergerandVolkerGrimmfortheinvitationto write,DonDeAngelis forguiding usinto individual-basedecol- ogy,DagL.Aksnes, ØyvindFiksen, PaulJ.B. Hart, MarcMangel andVolkerGrimmforstimulatingdiscussions,andtwoanonymous reviewersforveryhelpfulcomments.Thestudywassupportedby RCNgrant222021/F20toS.E.andC.J.andcontributestotheNordic CentreforResearchonMarineEcosystemsandResourcesunder ClimateChange(NorMER).

Appendix. Examplesandfunctions

Giskeetal.(2013,2014)modelledspatialbehaviourinanIBM ofafishpopulationusingtheproximatearchitectureframework (Fig.1),andaGAtoevolveadaptivevaluesofgenes.Allperceptions Pwerescaledlinearlyrelativetothestrongestobservationofeach perception,andtheneuronalresponseRtoeachperceptionPwas modelledasasigmoidalfunction(BrownandHolmes,2001)

R= (P/y)^x

1+(P/y)^x (1)

wherexandyaregeneswithallelevaluesinthe0.1–10.0range, whichgiveresponses0≤R≤1.Thealleleofthey-genedetermines theperceptionPatwhichtheresponseR=0.5,andthexgenedeter- mineshowrapidlytheresponseincreaseswithP.Dependingonthe valuesofxandy,theshapeofthefunctionmayrangefromnear lineartoathreshold-typeresponsewithinthis parameterrange (0≤P≤1).Thisallowsforgradedresponsestoweaksignalsaswell assaturationofinformation(AksnesandUtne,1997;Ashleyetal.,

threeadditivecomponentsinEq.(2)whenevaluatingfearduring predatorattacks(Andersen,2014).Thus,exceptinverycomplex modelenvironments,Eq.(1)willdothejob.

EachneuronalresponseRhasanadditiveeffectononeneuro- biologicalstate(Fig.1).

Hunger=M×(R_Astomach+R_Afood) (3)

Fear=(1−M)×(RA_light+RA_predators+RAconspecifics) (4)

ThesubscriptAindicatesthattheneuronalresponseisusedin theappraisalphase(Fig.1,top).Themodulatorymechanism(M)is inthisexample(Giskeetal.,2013)alife-historytrade-offbetween growthandsurvivalthatmayvarywiththedevelopmentalstateof theorganism.Inthemodel,fourgenesgivetheMvaluesatfour differentbodymasses(with othervaluesfor Mfoundbylinear interpolationbetweenthesepoints).Theglobalorganismicstate (GOS)oftheindividualisthendeterminedbythestrongerofthe neurobiologicalstatesofhungerandfearinthiscase.

Theresponse phasedepends ontheglobal organismicstate, whereattentionnarrowsthebehaviouralresponsetooneadequate forthesituation(Fig.1).Inthecaseofhabitatselectionforapelagic planktivore,theorganismcouldchoosetomoveupordowninthe watercolumnortostayatitscurrentlocation.Thesurrounding depths(z)areevaluatedbynewneuronalresponsesandthefish movestothedepththatmaximizesnetneuronalresponse(Fig.1, bottom).Forhungryfish(Eq.(5))theresponsefromseeingfood givesapositiveeffectwhiletheresponsefromseeingconspecifics hasa negativeeffect. For frightenedfish(Eq. (6))the response fromconspecificshaveapositiveeffectwhilelighthaveanega- tiveeffect,sinceriskisstronglylinkedtolightintensityinpelagic environments,whichdecaysfastwithdepth(Giskeetal.,1994):

Hungryfish:maxz−1,z,z+1(RH_food−RHconspecifics) (5) Frightenedfish:maxz−1,z,z+1(RFconspecifics−RF_light) (6) Eachindividualhad23 genes(9x-and 9y-genes,4Mgenes andonesex-determinationgene)thatdeterminedresponsefunc- tionsandmodulatorymechanism.Evolvinginageneticalgorithm, ittookafewthousandgenerationstofindquasi-stablesolutions.

Muchoftheadaptiveevolutionwasdonewithinthefirstcouple ofgenerations,andmostwasdoneinthefirsthundred.Thelong tailinslowerimprovementisduetotheweakerselectiongradient neartheoptimum(Fisher,1930),plusundulationsinthefitness landscapecausedbydensity-dependentgrowthandsurvivaland frequency-dependentselection(Giskeetal.,2013).

References

Aksnes,D.L.,Utne,A.C.W.,1997.Arevisedmodelofvisualrangeinfish.Sarsia82, 137–147.

Andersen,B.S.,2014.EffectsofGeneticComplexityandTimeResolutioninan Individual-basedModelofMesopelagicFishwithHungerandFear.Department ofBiology,UniversityofBergen,Bergen.

productivity?TrendsEcol.Evol.23,361–368.

Botkin,D.B.,Wallis,J.R.,Janak,J.F.,1972.Someecologicalconsequencesofacom- putermodelofforestgrowth.J.Ecol.60,849–872.

Brown,E.,Holmes,P.,2001.Modellingasimplechoicetask:Stochasticdynamicsof mutuallyinhibitoryneuralgroups.Stoch.Dynam.1,159–191.

Budaev,S.V.,1997.Personalityintheguppy(Poeciliareticulata):acorrelationalstudy ofexploratorybehaviorandsocialtendency.J.Comp.Psychol.111,399–411.

Burtsev,M.,Turchin,P.,2006.Evolutionofcooperativestrategiesfromfirstprinci- ples.Nature440,1041–1044.

Byrski,A.,Drezewski,R.,Siwik,L.,Kisiel-Dorohinicki,M.,2015.Evolutionarymulti- agentsystems.Knowl.Eng.Rev.30,171–186.

Cabanac,M.,1979.Sensorypleasure.Q.Rev.Biol.54,1–29.

Charnov,E.L.,1976.Optimalforaging:themarginalvaluetheorem.Theor.Popul.

Biol.9,129–136.

Clark,C.W.,Levy,D.A.,1988.Dielverticalmigrationsbyjuvenilesockeyesalmon andtheantipredationwindow.Am.Nat.131,271–290.

Clark,C.W.,Mangel,M.,2000.DynamicStateVariableModelsinEcology.Oxford UniversityPress.

Crombach,A.,Hogeweg,P.,2007.Chromosomerearrangementsandtheevolution ofgenomestructuringandadaptability.Mol.Biol.Evol.24,1130–1139.

Crombach,A.,Hogeweg, P.,2008.Evolution ofevolvabilityingene regulatory networks.PLoSComput.Biol.4,e1000112.

Dawkins,R.,1976.TheSelfishGene.OxfordUniversityPress.

deBoer,F.K.,Hogeweg,P.,2012.Co-evolutionandecosystembasedproblemsolving.

Ecol.Inform.9,47–58.

deWaal,F.B.M.,2011.Whatisananimalemotion?In:Miller,M.B.,Kingstone,A.

(Eds.),YearinCognitiveNeuroscience,vol.1224,pp.191–206.

DeAngelis,D.L.,Cox,D.K.,Coutant,C.C.,1980.Cannibalismandsizedispersalin young-of-the-yearlargemouthbass–experimentandmodel.Ecol.Model.8, 133–148.

DeAngelis,D.L.,Grimm,V.,2014.Individual-basedmodelsinecologyafterfour decades.F1000PrimeRep.6,39.

DeAngelis,D.L.,Mooij,W.M.,2003.Inpraiseofmechanisticallyrichmodels.In:

Canham,C.D.,Cole,J.J.,Lauenroth,W.K.(Eds.),ModelsinEcosystemScience.

PrincetonUniversityPress,Princeton,pp.63–82.

DeAngelis,D.L.,Mooij,W.M.,2005.Individual-basedmodelingofecologicaland evolutionaryprocesses.Annu.Rev.Ecol.Syst.36,147–168.

Dieckmann,U.,Law,R.,1996.Thedynamicaltheoryofcoevolution:aderivation fromstochasticecologicalprocesses.J.Math.Biol.34,579–612.

Dingemanse,N.J.,Kazem,A.J.N.,Reale,D.,Wright,J.,2010.Behaviouralreaction norms:animalpersonalitymeetsindividualplasticity.TrendsEcol.Evol.25, 81–89.

Dingemanse, N.J., Reale, D., 2005. Natural selection and animal personality.

Behaviour142,1159–1184.

Dingemanse,N.J.,Wolf,M.,2013.Between-individualdifferencesinbehavioural plasticity withinpopulations: causes andconsequences. Anim. Behav.85, 1031–1039.

Doyle,J.C.,Csete,M.,2011.Architecture,constraints,andbehavior.Proc.Natl.Acad.

Sci.U.S.A.108,15624–15630.

Draghi,J.A.,Whitlock,M.C.,2012.Phenotypicplasticityfacilitatesmutationalvari- ance,geneticvariance,andevolvabilityalongthemajoraxisofenvironmental variation.Evolution66,2891–2902.

Duarte,A.,Weissing,F.J.,Pen,I.,Keller,L.,2011.Anevolutionaryperspectiveon self-organizeddivisionoflaborinsocialinsects.Annu.Rev.Ecol.Syst.42, 91–110.

Dukas,R.,Kamil,A.C.,2000.Thecostoflimitedattentioninbluejays.Behav.Ecol.

11,502–506.

Dunlop,E.S.,Heino,M.,Dieckmann,U.,2009.Eco-geneticmodelingofcontemporary life-historyevolution.Ecol.Appl.19,1815–1834.

Eliassen,S.,Jørgensen,C.,Giske,J.,2006.Co-existenceoflearnersandstayersmain- tainstheadvantageofsocialforaging.Evol.Ecol.Res.8,1311–1324.

Emlen,J.M.,1966.Roleoftimeandenergyinfoodpreference.Am.Nat.100,611–617.

Enquist,M.,Arak,A.,1994.Symmetry,beautyandevolution.Nature372,169–172.

Euler,L.,1760.Recherchesgénéralessurlamortalité:lamultiplicationdugenre humain.Mem.Acad.Sci.Berlin16,144–164.

Evans,M.R.,Grimm,V.,Johst,K.,Knuuttila,T.,deLanghe,R.,Lessells,C.M.,Merz, M.,O’Malley,M.A.,Orzack,S.H.,Weisberg,M.,Wilkinson,D.J.,Wolkenhauer,O., Benton,T.G.,2013.Dosimplemodelsleadtogeneralityinecology?TrendsEcol.

Evol.28,578–583.