Availableonlineatwww.sciencedirect.com
ScienceDirect
ComputersandComposition59(2021)102629
Crossover literacies: A study of seventh graders’ multimodal representations in texts about Pokémon Go
Elin Strømman
1TheDepartmentofTeacherEducation,NTNU,7491Trondheim,Norway Availableonline9February2021
Abstract
Inthisarticle,ananalysisofmultimodalrepresentationsinelementaryschoolstudents’descriptivetextsaboutthemobilegame PokémonGo(PG)isusedtodiscussyouths’newliteracypracticesemergingfromtheirout-of-schoolexperiences(e.g.,gaming, producinggame-walkthroughs,andfanart).Socialsemioticmultimodalanalysesoftwostudents’PGtextsandtheirparticipation intalksaroundtheirtextsareusedtoexemplifywhatoccurssemioticallyinthetranslationofmeaningsanddesignsacrossmodes, media,and sites.Combiningthe analysisofthe meaningpotentialsofmultimodalrepresentations withethnographic accounts of theirusein context producesthe followingfindings:The PGwriting taskconnectswiththe students’life-worldsand the gamingcontextaroundPGpromptsthedesignoftheirmultimodalrepresentations.Thestudentsareactivecreatorsofcontentand demonstratepurposefulandinnovativeusesofsemioticresourcesinself-representation,stancetaking,andaudienceawareness.
TheiPadfacilitatesmultimodalanddigitalcrossoversbetweenleisureactivitiesandschoolsubjects.Theconcludingdiscussion suggestshowgame-basedliteracypracticescouldbetransferredintoacademicsettings.
©2021TheAuthor.PublishedbyElsevierInc.ThisisanopenaccessarticleundertheCCBYlicense(http://creativecommons.
org/licenses/by/4.0/).
Keywords: writing;multimodality;affinityspaces;game-basedliteracypractices;iPads
1. Introduction
Whenrepresentingknowledge,studentsinterpret,select,transform,andcombinethedifferentmodesandsemiotic resourcesavailabletothem,basedonthesocialcontextandthespecificaimstheystrivetoachieve(Kress,2010).Both inandoutofschool,studentscreatedigitalcontentfocusingonspecificinterests,suchasgaming,music,andfanart, andmanystudentsparticipateintheso-calledaffinityspaces,wheretheyinteractintermsofcommonendeavorsand sharedpassions(Gee,2004).Thesesocialnetworksrelyonwritingastheprimarymodeofcommunication(Barton
&Papen,2010),although(asmystudyshows)thiskindofwritingissituatedinmultimodalmeaning-making.Inthis paper,Ipresentfindingsthat canleadtoabetterunderstandingofmultimodalliteracypracticesintablet-mediated environmentsanddiscussthewaysinwhichstudentscanbenefitfromgamingliteraciesthatmakeuseofoverlapping digitalandmultimodalpractices.
E-mailaddress:elin.stromman@ntnu.no
1 Postaladdress:Elvgata8,7350Buvika,Norway.
https://doi.org/10.1016/j.compcom.2021.102629
8755-4615/©2021TheAuthor.PublishedbyElsevierInc.ThisisanopenaccessarticleundertheCCBYlicense(http://creativecommons.org/
licenses/by/4.0/).
InNorway,theconceptsofliteracy,multimodality,anddigitalcompetenceareembeddedinanationwidecurriculum andstudentliteracyachievementiscrucialthroughouttheelementaryGrades1-7andinhighereducation(Grades8- 13).From a Europeanperspective, Nordic countries arefrontrunners inthe deployment of high-speedInternet in schools,andNorwayisalsooneofthetopfivecountriesintermsofstudentaccesstocomputersandtablets(European Commission,2019).Despiteregionaldifferences,elementaryschoolsincreasinglyadopttablettechnology.
Mystudywasconductedinanelementaryschoolthatimplementeda1:1iPadprogramin2012/2013.Studentsin Grades5-7alwayshavetheirpersonaliPadavailable,evenathome,usingitforhomework,gaming,communicating onsocial media,andsoon.The participantsare12-year-old studentswhoare confidentintheir abilitytousethe iPadasatoolforbotheducationalandrecreationalpurposes.Inthispaper,Iexplorewhatcharacterizesthestudents’
digital-semioticpracticesincomposingtextsaboutthemobilegamePokémonGo(PG).Ihaveappliedsocialsemiotic multimodalanalysis(SSMA)ofthemultimodalrepresentationsemerginginthetextcompositionphase,inthefinished texts,andinthestudents’talksaroundtheirtexts.ThismeansthatIhavepaidattentiontothestudents’selectionsand arrangementsofsemioticresources,encompassingtheirreuseofexistingordigitallyadaptedresources,aswellasto theirrepresentationalchoices.Myanalysishasbeenguidedbythisresearchquestion:Howdoesthestudents’workin andaroundPokémonGoinfluencetheirmultimodaldesignsandrolesaswriters?
Thestudents’talksaroundtextsillustratethatwhenfacedwiththePGassignment,theyturntotheirfamiliaritywith thisgameanditsparatextstohelpthemmakesenseoftheassignment’slinguisticandvisualdemands.Thisresultsin innovativeandevolvingmultimodalrepresentations,suchascollages,fanartandlogos,drawingondesignsandskills thatarespecifictogamers’makerculture(Gee&Hayes,2012).Theanalysishereconcentratesontwofocalstudents’
multimodalrepresentationsandthreeselectedliteracyevents:thedesignofthePGheading,thecreationofPGfanart, andthewrittenPGtext.
2. Previousresearch
TheiPadissaidtohaveamultimodalintuitivelogicthat couldfacilitatestudents’learning,extendingtherange of resources for communicationandrepresentation andallowingcombinationsof verbal,visual, andaural modes (Gallagheretal.,2015).Internationalreviewstudiesontabletshaveshownthelackoflongitudinalstudiesmonitoring students’ writing practices with tablets (Haßler, Major,& Hennessy,2016; Zhang &Nouri, 2018).Furthermore, researchintothemodalcomplexityofferedbytabletsthatconsidersthelearningcontextremainsscarce(Simpson&
Walsh,2017).InNorway,researchershaveidentifiedthelackofresearch-basedknowledgeaboutthelearningoutcomes ofeducationaltechnologyusingtablets(Krumsvik,Berrum,&Jones,2018).Inshort,students’multimodalliteracy practiceswithiPadsstillconstituteanemergingareaofeducationalresearch.
2.1. Crossoverliteracies:multimodalcompositionandgamingliteracy
Inthisliteraturereview,Ipresenttheconnectionsbetweenout-of-schoolandin-schoolmultimodalcomposition focusing ongame-basedliteracy.LiteracyscholarsPamelaTakayoshiandCynthia L.Selfe(2007)havestatedthat thedefinitionof multimodalcompositionneedstoreflectstudents’literacypracticesinnewdigitalcommunication environments.Usingcomputergamesandgamingparatextsinmultimodalcomposition,instructorscanbuildbridges acrossstudents’interest-drivenliteracypracticesandhelpthemdevelopscholarlypurposesofwriting(Arduini,2016, 2018;Daniel-Wariya,2016;DeLuca,2018;Fry,2012;Gee,2007;Jewitt,2008;Sabatino,2014).Accordingtoliteracy researcherTinaArduini(2016,p.10),gamingliteracyrefersto“theliteratepracticesofgamersandtheirabilitiesto effectivelyplayvideogamesandutilize gaming-relatedparatexts.”Even thesimpleactof gamingrequiresfluency inimageinterpretation,textconsumption,andaudioawareness,andArduini (2018)hasarguedthat engagingwith paratexts (e.g., game-related websites, walkthroughs, Internet forums, etc.) creates spaces for gamers to develop importantmultimodalcompositionskills.KatherineDeLuca’sresearchonaffinityspacesandfandomshassuggested that integratingelementsfromtheseonline communitiesintothecompositionclassroomcanenableinstructors“to build apedagogy that begins where the students alreadyare through the distribution, circulation, anddiscussion of multimodalcompositions”(2018,p. 76).Examiningtherelationsbetweencomputer gamesandwriting,Joshua Daniel-Wariya(2016)hashighlightedtheelementofplayingaming,aswellasplayasameaning-makingresource.
Inherdissertationongamewalkthroughsasinstructionaltexts,MeganFry(2012)haspositedthatwalkthroughscould be pathstoprofessionalwriting. Thisrelates toLindsaySabatino’s (2014)workon Facebookgaming,whereshe
hasdemonstratedhowthepublicnatureofwritingingame-basedcommunitiescreatesopportunitiesfordeveloping audienceawareness.Anoverviewarticleof93Nordicstudiesonstudents’out-of-schoolliteracypracticesidentifies thatmostofthestudiesexaminewhetherthesepracticescrossovertoclassroompractice(Juuhl&Michelsen,2020).
Researchoncrossoverliteracieshasshownthatstudentssearchfor,andreuse,semioticresourcesinre-mixingprocesses withineducationalsettings(Erstad,Gilje,&deLange,2007;Strømman,2020).ArecentFinnishstudyofstudents’
gamingliteracyidentifiescreatingandsharingasimportantmetagameactivities(Kahilaetal.,2020),findingsrelevant formyownresearch.
Whilenewmediascholarshaveidentifiedimportanttransferrableskillsthatstudents’everydaydigitalliteracies canprovide,severalscholarshavealsopointedtoapedagogicaldigitaldisconnectbetweenhomeandschoolliteracy practices (DePalma&Alexander, 2015;Walsh, 2010;Williams, 2014;Wohlwend, 2017). Thisdisconnectcanbe thoughtofasasiteofcontestationwhereunequaldiscoursesonliteracyintersectandinfluencehowliteracyistaught, whogainsaccess,andwhatcountsasliteracy(Wohlwend,2017).Researchershaveidentifiedgapsinstudents’rhetorical comprehensionofthevariousnewmediainwhichtheyengageandgapsbetweenmultimodalandprint-basedcomposing tasksregardingthestudents’notionsoftheaudienceandtheprocessesofinventingmeaning(DePalma&Alexander, 2015).Someinstructorsalsofaceagapbetweentheconventionalteachingofcompositionandmoderntextcultures.
Wheninstructorsseektoincludedigitalgamesandotherelementsofpopularcultureintheirteaching,thiscanentail engaginginpracticeswithwhichtheyhavelittleaffinityorexpertise(Walsh,2010;Williams,2014).
2.2. ResearchingPokémonGo
Releasedin2016,PGis anaugmentedrealitygame wherethePokémon figuresappearonamapinrealtime.
PlayerscollectPokémonsandprogressthroughthegame’slevelsbythrowingPokéballstowardthecharactersasthey encounterthem.Thecharacterscanbeattractedthroughitemscollectedduringthegame,andPokémonscanbattlein gyms–destinationswhereopposingteamsfaceeachother(Niantic,2016).AsPGservesasanaccessibletextforclass purposes,ithasbeenusedanddiscussedbybothpractitionersandliteracyscholars(Blevins,2018).ResearchonPG includesstudyingthegameinrelationtoaffinityspaces(Tran,2018),navigatingaugmentedreality(Blevins,2018;
Wohlwend,2017),thecharacteristicsofthePokémonfandom(Assunc¸ão,Brown&Workman,2017),andconceptsof multimodality(Howell,2017).AccordingtoEmilyHowell(2017),PGisanexampleofhowamultimodaldigitaltext hasitsownaffordances(e.g.,byemployingimagesandmapping)andhowitconnectswithgamingtutorialsemploying multiplemodes(e.g.,text,font,color,hyperlinks,stillimagesandvideos),requiringstudentstounderstandinformation invariousmodes.MuchoftheteachingonhowtoplayPGisoffloadedfromthegameanddistributedacrosssitesand resourcesprovidedbyotherplayers(Tran,2018).Thesesitesandresourcesincludewalkthroughs,wikis,glossaries, andfanartsharedonsocialnetworks,andtheliteracypracticesinthePGaffinityspacehaveamultimodalnature.
3. Theoreticalperspectives
Several,sometimesoverlapping,theoreticalframeworksandconceptsrelatetointerest-based,technologicallymedi- atedlearning,especiallyamongyouths.Here,Idrawontheoriesfromtwointerconnectedfields:1)NewLiteracyStudies (NLS),whichexploreissuesofidentityandagencyinrelationtoliteracypracticesacrossvariouscontexts,suchas in-schoolandout-of-school(Barton,2007;Barton&Lee,2013;Gee2004,2007;Gee&Hayes,2012;Maybin,2007), and2)multimodaltheory,whichisorientedtowardtheobservationandtheanalysisofsemioticproductionindigitally mediatedmultimodaltexts(Kress,2010;Kress&vanLeeuwen,2006;Jewitt,2008,2017).ThethirdtheorythatIuse concernsLevManovich’s(2013)viewsonnewmediaandthecultureofremixability.
3.1. Literacyeventsandaffinityspaces
ScholarswithinNLSregardliteracyasasetofsocialpracticesthatareobservableineventsmediatedbywritten texts.InNLS,thenotionofaliteracyeventbecomestheempiricalphenomenon,providingastartingpointforanalysis, while literacy practicesare more theoretical,providing patterns abstracted from events(Barton &Papen, 2010).
According to ethnographerShirley Brice Heath, aliteracy eventis “any occasion in whicha pieceof writing is integraltothenatureoftheparticipants’interactionsandtheirinterpretativeprocessesandstrategies”(1983,p.74).
Toreflectmorecontemporarynotionsofliteracy,NLSscholarDavidBarton(2007)haselaboratedontheideaofa
literacyevent,suggestingthatrelationsbetweeneventsmaybeserialandchained,embedded,orsubordinated.When researchingstancetakinginpubliconlinespaces,DavidBartonandCarmenLee(2013)havefoundthatpeoplemove seamlesslyinandoutofdifferentliteracyeventsanddomains.AspointedoutbyBarton(2007,p.177),thismovement isalsothe caseinschools:“(...)schooledliteracyisnottheonlyformof literacygoingoninschools.Theseare otherliteracieswhicharerenderedinvisible.”ThisalignswithJanetMaybin’s(2007)argumentthatthehome/school binary embedded in the NLS theoreticalframework has been associated withan abstract conceptionof schooled literacythatdoesnot considerstudents’actualeverydayexperiencesinclassrooms.Inherethnographicclassroom research,Maybinhasuncoveredarange ofunofficialliteracyeventsamongthestudents,referredtoas “underthe desk literacies”(2007,p. 517).Although theseliteracyeventsappeared tobeoff taskactivitiesthat strayed from institutionalnorms,theyinvolvedarangeofdifferentkindsofengagementwithtextsthatcouldhelpbuildliteracy (Maybin,2007).
AsimilarspiritthatisopentoeverydayliteracypracticesisevidentinJamesPaulGee’s(2004)researchonvideo gamesandaffinityspaces.TheaffinityspacetheoryisanchoredonGee’sobservationsonhowindividualsinteract inonlinegames,forumsandwebsites.Asplayersjoin digitalgames,theyinteractwith“amultimodalliteracypar excellence”(Gee,2007,p.18),andthesegamescancomprise“afruitfulprecursordomainformasteringothersemiotic domainstiedtocomputersandrelatedtechnologies”(p.40).Basedonstudiesongame-basedfansites,JamesPaul GeeandElisabethHayes(2012)haveidentifiedfeaturesofnurturingaffinityspacesthataresupportiveoflearning.
Affinityspaceshonorcraftknowledgeandfacilitateidentityplayandreflectionsonaudience,genre,andprocess.In onlineaffinityspaces,peoplecanmake,commenton,andshareculturalartifacts(e.g.,images,videosandtexts),and bydoingso,adaptandaddtotheseartifacts’multimodaldesigns.Affinityspacesaretypicallylinkedtoout-of-school practices(Gee,2004).Inmystudy,manyofthestudents’interest-drivenpractices(photoandfilmediting,gaming, producinggamewalkthroughs,etc.)areexamplesofcrossoverskills,bothpresentinthestudents’everydayliteracies andintegraltoschoolassignments.
3.2. Multimodalrepresentations
Thetheoreticalfieldofmultimodalityplacestextmakinginatraditionofsocialsemioticsandunderstandssigns asmultimodal,intentional,andculturallyshaped(Kress,2010).AccordingtoGuntherKressandTheovanLeeuwen (2006),socialsemioticsisorientedtotheobservationandtheanalysisofthecomplexityofsemioticproduction,aswell astothediscoveryofnewsemioticresourcesandnovelwaysofusingexistingones.Here,Kress’(2010)conceptof themotivatedsignisrelevant.Inbrief,thisconceptplacesagencyonthesignmakeranddrawsattentiontotheinterests andthe intentionsthat motivatethe students’choicesof semioticresources. Multimodalcomposition involvesthe abilitytodesignmultimodalrepresentations,andaccordingtoKress(2010),theprocessofrepresentationisidentical totheshapingofknowledge.FollowingKress,Iviewmultimodalrepresentationsasmaterializationsofthestudents’
semioticandinterpretivework.
Currentmultimodalcompositionresearchhasdrawnonapproachesbeyondthelinguisticmodelofsocialsemiotics, including film, music, andgamingtheories. Asmultimodal compositionscholar Carey Jewitt (2008)has posited, gamingandgamingparatextsfacilitatethemovementofimagesandideasacrossgeographicalandsocialspacesin waysthataffecthowyouthsinteractandlearn.Jewitthasalsohighlightedthatdigitaldevicesconstituteakeyarea ofmultimodalinvestigationbecausetheymakeawiderangeofrepresentationalmodesavailable,oftenin“newinter- semioticrelationships”(2017,p.452).MediaresearcherLevManovich’s remixmetaphorisusefulinthiscontext, capturingnotonlyhowdifferentmediatypesgetremixedbutalsohowwaysofrepresentationandexpressioncross overandcombine(2013,p.176).
4. Researchdesign,methods,anddata 4.1. Multimodalethnography
Mystudydrawsonmultimodalethnography,aresearchframeworkthatusesthesocialsemiotictheoryinethno- graphicresearchtoproduce accountsof situated artifactsandinteractionsandthe relationsbetweenthem (Jewitt, Bezemer,&O’Halloran,2016).Multimodalethnographicapproachesexaminepeople’spracticesinsociallysituated signprocesses,andtheresearcherisimmersedintheparticipants’practices,languageuses,andcultures,taking“an
emicperspective”(Purcell-Gates,2011,p.142).Ispent18monthswithfifty11–12-year-oldstudentsandtheirteacher, limitedtothecompositionclasses(usuallytwiceperweek),andutilizingakeymethodologyofmodernethnographic approaches–familiaritywiththecontextinwhichthedataarecollected,participantobservation,interviews,andfield notes(Purcell-Gates,2011).Toelicitinsiderinformationandexploretheliteracyeventsindetail,fourfocalstudents wereselected.Thesestudentswerechosenbecausetheydemonstratedawillingnesstotalkabouttheirwritingsand texts.Thisarticle featuresthetwofocalstudentswhomIwasabletoobserveinthePGcompositionphases.2The ethnographicworkinvolvedmonitoringthetwostudents’useofiPadsincomposingthePGtexts,recordingtheirtalks aroundtheirtexts,andanalyzingthemultimodalrepresentationstheyproduced.
4.2. Participants
The12-year-oldstudentsJonandMartinwereskilledproducersofonlinecontentandhadsimilarinvestmentsin technology.Bothweregamers,producersofvideogamewalkthroughsandfanart,andparticipantsof theschool’s game-basedlearningplatformMinecraftEducationEdition.Theyalsobelongedtovariousaffinitygroupswherethey interactedintermsofcommoninterests.Joncreatedinstructionalgamingvideos,applyingmanyiPadapps,andhad hisownYouTubechannel.Usingseveralapps,Martinproducedjinglesandothermusicalpieces,frequentlyincluding them,togetherwithsoundeffectsanddigitaldrawings,inschoolassignments.
4.3. Talksaroundtexts
Myinterviewswiththefocalstudentsrangedfrominformalconversationstomorestructuredinterviews.Aform ofsemi-structuredinterviewingiscalled“talkaroundtexts”,usingtextsorartifactsasstartingpoints(Page,Barton, Unger,&Zappavigna,2014,p.120).Inthesesessions,thetextsandthemultimodalrepresentationsproducedbythe studentswereused asthe fociof thetalks.JonandMartinalsoengagedinconversationswhile theywerewriting (e.g.,inpeer-responseactivities), andthesetalkswererecorded.Literacyeventsareconstitutedthrough dialogues, oftenrelatingtotextsorimages,whichgiveorallanguageanimportantrole.Thetalk-around-textmethodprovided cluesabouttheparticipants’perceptionsonthecontextofwritinginschool,theirpartakinginaffinityspaces,their multimodalchoices,andthusabouttheunderlyingculturalpractices.
4.4. ThePGwritingtask
TheobjectivesofthePGwritingtaskwereto“describethemobilegamePokémonGotoapersonwhohasnever playedit,”“explainthedifferentstagesofthegame,”and“sharetipsandtricksofthegame.”Furthermore,thestudents wereaskedtoevaluatethegameandgamingbylistingboth“positiveandnegativeaspects.”Inadditiontodepicting theirideasinwords,thestudentswereaskedto“insertimagesand/orscreenshots”(e.g.,fromtheirmobilephones).
Theassignmentpositionedthestudentsas“gameexperts”bywritingadescriptiveandinformativetextaddressing alessinformedreader.Infact,notallthestudentswereexperts;somedescribedthemselvesas“beginners.”Forthe initialphase of thewriting process,theteacher thereforeprovidedthe studentswithpossibilities toengageinthe game,usingtheirmobilephonesinschool,andassignedtheexpertstheirrolesasinstructors.Asallthestudentswere familiarwiththePokémoncharactersandmanytalkedabouttheirfavoritePokémon,anadd-ontotheassignmentwas to“writeaparagraphaboutyourfavoritePokémoncharacter,whereyoudescribewhatitlookslike.”3Exceptforthe callforinsertionofimagesinthewritingtask,theillustrationofthetextswasnotatopicinthelessons,leavingthe representationalchoicesuptothestudentsthemselves.Theassignmentservedthepurposeofconnectingwritingto authenticpurposesandpossiblytakingacriticalstancetowardthegameandthegamingactivity.Finally,thewriting taskfunctionedasapromptformultimodalcomposition(Kress,2010).
2 Thestudentsworkedinpairs,andIchosetomonitortheonlypairthatcomprisedtwoofmyfocalstudents.
3 TranslatedexcerptsfromthewrittenassignmentaboutPG.
Table1
DatafromthePGwritingproject.
Datasources Useinthestudy Meansofanalysis
Writtentexts:Twostudents’PGtext writteninPages
Usedforanalysisofmultimodal representations
Socialsemioticmultimodalanalysis (SSMA),theory-guided(deductive)text analyses
Talkaroundtexts:Videotaped peer-responseactivitybetween twofocalstudents
Allowedanunderstandingofthe students’multimodalchoicesand participationsinaffinityspaces
Inductivecodingbasedonthestudents’own wordsandphrases,renderedasexcerptsin theanalysissection
Screenshots:PGheading,logo,and fanart
Usedforanalysisofgame-related artwork
SSMA,renderedintheanalysissection (Figs.1–6.)
Teacher’shandout:PGwritingtask Usedtohelpframetheenvironment oftextcomposition
Renderedintheintroductionsection Fieldnotes(sevenlessons) Providedadditionalinsightsintohow
thewritingtaskwaspresentedand whatthestudents’writingprocesses andmultimodalchoices“looked like”tomeasaresearcher
Underlinedfeaturesofthewritingprocesses, andmultimodalchoicesusedastalking pointsintalks-around-texts
4.5. Data
Thenotionofaliteracyeventwasusedtofocusdatacollection.Inthispaper,thefocalliteracyeventsinformingthe analysisincludeapeer-responsesessionbetweenthefocalstudentsJonandMartin;thetwoboys’workonheadings, logosandfanart;andtheirfinalPGtexts.AccordingtoNLSresearchers,thissetofelementsrepresentsalinkedchain ofliteracyevents(Barton,2007).Iemployedacombinationofwritingfieldnotesandaudiotapingandvideotaping fast-movingevents,suchasthepeer-responsesituationandtalks-around-texts.Theaudiotapedandvideotapeddata weretranscribedandtranslatedintoEnglish.Videotapingalsoallowedscreenshotsoftextualelements,suchasvarious multimodalrepresentationsinthemaking.Finally,thestudents’texts4werecollected,alongwiththeteacher’swriting task.Thethree-leveldescriptionofthedata(Table1)showsthedifferenttypesofdatacollectedinrelationtothePG assignment.
4.6. Analyticalapproach
Asethnographicapproachestoliteracyseektoexplain,describe,andprovideinsightsintoactivitiesintheircontexts (Purcell-Gates,2011),myanalysesaredirectedtowardthecharacteristicsofthestudents’multimodalrepresentations producedinsitu.Theearlycodingofthedatabeganintheclassroomwhendifferentinstancesofmultimodalitywere beginningtoemergefromthestudents’conceptmapsandPGtexts.Thenotesandthequestionsfromthehandwritten fieldnotesweremarked,labeled,andusedastalkingpointsintheinformaltalk-around-textsessions.Thesecondphase ofthedataanalysisinvolvedcodingandcategorizing,bothinductivelyanddeductively.Intheinductiveor“bottom- up”coding(Purcell-Gates,2011,p.150),thecodingdecisionswerebasedonthestudents’ownwordsandphrases.
Forexample,whenwritingthePGtexts,severalstudentsspenttimeexploringandchoosingamongthebuilt-infonts inPages.Inthepeer-responsesession,JonandMartindiscussed“therealgamingtext,”“thelanguageofgaming,”
“lettersthatgrown-upsdon’tunderstand,”andthedesireto“createone’sown‘texttype’”(sic).Allthesecodesrelate tothefontas asemioticresource,acategoryof interesttothe researchfocus andacentralconceptinmultimodal theory.
SSMAemphasizessituatedactionandmeaning.Severalresearchershavedescribedsocialsemioticprocessesand differentdimensionsofrepresentation(Jewitt,2017:Kress,2010;Kress&vanLeeuwen,2006).SSMA’sdescriptive framework has been used as a tool for the deductive, or rather, theory-guided,analysis of the multimodal repre- sentations.In my analysis,I relatekey analytical pointstocentralSSMAconcepts: mode, transformation,modal affordances, aptness, functionalload, andsalience. Mode isasocially andculturally givenresourcefor meaning
4 ExcerptsfromthePGtextsareparaphrasedintheanalysissection.
making.Writing,image,andlayoutare thedominant modesinthe PGtexts.According toKress(2010),adesign processthatinvolvesstayingwithinthesamemodes,semioticcategories,andlogicsiscalledatransformation.The differingaffordances(possibilitiesandlimitations)of modesenablespecificsemioticworktodrawontheseaffor- dances.Animportantpointhereisthatmodalaffordancesareconstantlyreshaped,dependingontheneedofthose whomake meanings (what theyconsiderapt),their rhetorical assumptionsinthe environment of communication (whattheydecideshouldhaveafunctionalload),andwhatisforegroundedinthetexts(whattheychoosetomake salient).
5. Analysisandfindings
5.1. Designoftheheading:salienceandaptness
Thissectionfocuseson theprocessof designingaPG-headinganddraws onavideotapedsession ofthe focal studentsMartinandJoninapeer-responseactivity.Inthissetting,theboysweresupposedtoreadandcommenton eachother’stexts;instead,theydriftedintoexploringafreeonlinelogomakerprogram,Vistaprint.no.Thiswebsite gavethemaccesstoarangeofimages,fonts,andcolorsfordesigningtheirownlogos.Inthefollowingexcerpt,Jon showsme(E)howtheprogramworksanddesignshisownPGlogo,whichMartinevaluates.Theitalicizedtextsin parenthesesdescribehowtheboysinteractwiththedigitalinterface.
E:WheredidyoufindthiswebsiteVistaprint?
J:Ijustgoogled“Howtomakeyour ownlogo.”OK,nowI’mgoingtomakemyownPokémon Gologo.Let’s see...(searchesforfigures).Theseare...notcool.(...)Thisonemightfit!
E:Whatmakesitfit?
J:‘CauseitlooksabitlikeaPokéball.AndthenIchoose...Ineedtofixatext(types).Thisworks!
E:Whydoyouthinkthisfontworks?
J:‘Causeit’sbold.E:Doesthegamelogohaveaboldfont?J:Yeah,andthisoneiscool.IfI’mabletochangethe colorofit(explorescoloroptions).Noredcolor,though.Well.Done!So,thisismynewPokémonGologo!(shows mehisiPad)(Fig.1).
Fig.1.Jon’sPokémonGologoinVistaprint.no.
J:AndnowIcanusethislogoasthetitleformytext.Let’sseewhatthatlookslike(replacestitlewiththenewly madePGlogo).
E:Andwhatdoyouthinkaboutthat?
J:Ahhh,Ithinkitlookscool.Whatdoyouthink,Martin?
M:PokémonGo!(readsJon’stitle).Butitdoesn’tlooklikeaPokéball.(pointsattheiPadscreen).
J:No,Iknow.SomaybeI’lljustundoit(deletesthelogoandreplacesitwithadifferenttitle)(Fig.2).
Fig.2.JonmakeshissecondPG-title.
Extractingmultimodalrepresentations,suchasthemakingofthislogo,forexaminationdirectsanalyticalattention tothedesigndetails.Fonts,images,andthelayoutarechosenandrepresentthefocusoftheindividual’sinterest(Kress, 2010).Jonisagameratlevel22inPG;hetakesanaffectivestancetowarditbydescribingitas“afunmobilegame for childrenfromtheageof5 to80.”Jon’schoiceofdesignelementsisbasedonhisinterestinthegameandthe brandname’sfeatures.Inthisrepresentation,hechoosesaboldfontandthemostappropriatefigureandcolorscheme toresembletheclassicPGball,basedonwhatheconsiders themostapt image/textdesign.Here,thenewlymade signcomplexdoesnotinvolveshiftsinmodes(cf.transformation).ThePokéballisrepresentedasaball-likefigure, andthegame’snameremainstobeinwriting.AccordingtoKressandvanLeeuwen(2006),theleft-rightplacement createsagiven-newstructure.Inthiscomposition,theballisplacedtotheleftandthuspresentedassomethingthat thereader/vieweralreadyknows.
Asshownintheprecedingdialogueexcerpts,Jon’sprocessofcreatingandthen(promptedbyMartin’scomment) deletinghisPGlogorepresentsanearly-phasedesignthathelaterrejects.InsteadoftheballmadeinVistaprint,he endsupusingtheheadingshowninFig.3,wheretheofficialPokéballimageservesastheletter“o”in“Go.”Thistype oftextualintegrationgivestheimageitsfunctionalload.Theplacementandthecoloroftheballalsomakeitstand outfromitssurroundings;thiseye-catchingfeatureattractstheviewer’sattention.Inmultimodaltheory,thistypeof visualweightisreferredtoassalience(Kress&vanLeeuwen,2006).
Fig.3.Jon’sthirdandfinalheading.
Martinchoosesadifferentsemioticsolutionforhisheading,namelyapictorialwrittenlanguageimportedfromthe PGwebsite,theso-calledPokémonalphabet.Thisheading(Fig.4)consistsofcurlingletters,withthePokéballatthe centerofeachletter,givingaresemblanceofeyes.Here,theheading’sessentialpurposeremainsintact,communicating thetext’stopic.Furthermore,itisanexampleofmultimodalstance-taking,reflectingMartinasagamer,possiblya PGexpert.Finally,hisuseofrecognizablevisual-linguisticcuesaddressesandincludesfellowPGplayers(Sabatino,
2014).Thevisualcharacterofwrittentexthasalwaysbeenpresentintypography,butasthesetwoexamplesshow,the fontasaphenomenonisofinteresttostudentsandincreasinglycontributestothewaysinwhichtheymakemeanings.
Fig.4.Martin’sheadingusingthePokémonalphabet.
Allthestudentsinmystudypaidattentiontonon-verbalaspects,suchastypographyandlayout,andtothedesign featuresofweb-basedtexts,suchasimagery,color,heading,andsoon.Manyofthestudentschoseaplainorcolored PGheading (19of42students)or insertedan imageof theofficiallogo(23).Twostudentsmadecollages,which included theofficiallogoandtheir choiceoftheir favoritePokémon.Whilemost studentschoseacopy-and-paste practicefortheirheadings,reusingreadymadematerial,Jon’sandMartin’sdifferentdesignsolutionsdemonstrated theirfamiliaritywiththesemioticfeaturesofPG,aswellastheirstatusasgamers.JonandMartinalsoknewhowto selectanddigitallyadaptsemioticresources,resultinginaptandinnovativedesigns.
5.2. Affinityspacesandfanart
Manygames,includingPG,allowplayerstomodifythembyusingthedesignsoftwarethatcomeswitheachgame.
ThisprocessisclosetohowJonnavigatedthewebsitewhenheredesignedthelogoandisanexampleofacrossover betweengamingandwriting.PGalsohasafancommunity,afandom,devotedtomasteringthegameandcreating artwork(fanart).ForJonandothergamers,suchcommunitiesareaffinityspacescenteredonproducingthingsand honoringcraftknowledge.Thefollowingexcerpt(fromthesamepeer-responseactivityastheprecedingone)shows howJonandMartincreateanddiscusscharactercollages,aswellaspositionstheboysasPGfans:
E:Whatdoyoucallthese?(pointingatJon’siPadscreen[Fig.5]) J:Fanart.
E:Fanart.Isthatthenameofawebsiteyouvisitorisitatypeoftext?
J:It’satypeof...No,it’sathingyoucreate
M:Sayifyou’redrawingsomethingforsomethingyoulike,thenyouareafan.Youknowwhatthatis?Andart, thatisthedrawing.
E:So,itislikedrawingorwritingonanimage?
J:Yes.Likethisone(pointsathisscreen).
E:Whataretheyfor?Doyousharethemwithothers?
M:Yes.
Fig.5.Pokémonfanart.
InFig.5,allthePokémonsarewearingsunglasses,aprominentsemioticfeature.Theretroglassesaresalientfan artaccessoriesandcanbedownloadedasfreetemplatesfromclipartwebsites.Thiscollagetechniquesupportsthe visualizationofselectedsegmentsandrequiresfamiliaritywithphotoediting,layering,andmultimodalcomposition.
In contrasttome,theboysseemconfidentwiththenameof thekindof artworktheyare creating(fanart)andits purpose(sharingwithothers).Game-relatedaffinityspaces,suchasthePGfandom,arecommonlyorganizedtoallow
Fig.6.ThuglifeCharmanderasascreensaver.
andencourageanyonetolearntobuild,design,remix,andshare.Gamesthatemphasizetheinvolvementofplayers asdesignersandencouragethedevelopmentofcreativegamedesigncommunitiesare“particularlygoodforfostering skillswithtechnologyanddesignandlearninginthe21stcentury”(Gee&Hayes,2012,p.4).
Thenextfanartexample(Fig.6)showsJon’siPadinthescreensavermode.Here,thePokémonCharmanderis portrayedaswearingsunglassesandlightingacigarette(orajoint?)withitsburningtail.Intheexcerptbelow,Jon showsMartinandmehislatestpieceoffanart,“Thuglife,”andexplainshisdesign.
J:Imadethispiecetoday,actually!
(showsusthescreensaverimage“Thuglife”).
E:Sunglassesandacigarette?
J:Yes(laughs).
E.Whydidyouincludethosethings?
M:(interrupts)Becauseit’scool!
J:Becausehelightsthecigarettewithhisburningtail.
E:AnddoyouknowwhatThuglifemeans?
J:Eh...no.Butitlookscoolwhenothersuseit.
InthePGgame,Charmander’stailburnsfiercelywhenhebecomesenragedorishappy.However,thecigarette, whichhastheshapeofajoint,andtheheading“ThugLife”areassociatedwithundergroundfanart,typicallyshared onsocialmedia(e.g.,YouTube,Facebook)andotherphoto-sharingplatforms(Pinterest,Deviantart,etc.).Jon’schosen fontisaGothicblackletterfontavailableinPagesandresemblestheThugLifelogorenderedonsocialmedia.5
Shiftingtechnologicallandscapeshaveallowedincreasedproductionandcirculationofmultimodalfancompositions suchasthese.Thoughchangesintechnologymayshifttheplaceswheremultimodalcompositioniscreatedbyand forfandoms,thepurposesofthesetextsremainconsistent–tobecirculatedamongindividualswithsharedinterests inordertomaintainacollectiveidentityacross time,space,andtechnologies(DeLuca,2018;Kahilaetal.,2020).
The boys’reactionstomy questionsindicate their awarenessthat thispieceof fanartstretches theboundariesof
“schooled”multimodalcomposition.Jondoesnotknowthemeaningof“Thuglife”6buthasadoptedakindoftaboo visualpracticecommoninsomefandoms.WhenJonhandsinhisfinishedtext,“ThuglifeCharmander”isreplacedby amoreinnocent-lookingCharmandertakenfromtheofficialPGwebsitegallery.Inotherwords,“Thuglife”remainsin thisundergroundorhiddenliteracypractice,thusservingasanexampleofMaybin’s(2007)underthedeskliteracies.
5.3. Jon’sPGtext:redesigningandrepresentingtheabstract
Theassignment wastowriteadescriptivetextaboutPG,including adiscussionwherethestudentswereasked tolistpositiveandnegativeaspectsofplayingthegame.Thishybridwritingtaskopenedforstancetaking.Jonhas
5 Forexample,seedafont.comandpinterest.com.
6 ThugLifewasarapgroupformedby2Pac.AccordingtotheUrbandictionary,itisanacronymforTheHateUGiveLittleInfantsFucks Everyone(https://www.urbandictionary.com/define.php?term=Thug%20Life).
Fig.7.PositiveaspectsofplayingPG.
chosen“Positive”and“Negative”tosignalthetwoparagraphsthatanswerthediscussionpartofthetask.Accordingto him,thepositiveaspectsofplayingPGrevolvearoundthefactthatitisanoutdoorsocialactivity:“Whengamersare playingPokémonGo,onespendsmoretimeoutdoorsinsteadofjuststayingathomegaming.Beingout,onecanteam upwithnewfriendstohuntforPokémons.”ThisparagraphtapsintotheverynarrativeofPokémon(i.e.,travelling theworldtocatchcreatures),andJonillustratesthispointinacollagecombiningtwoimages.Thethree-dimensional illustrationofthreefriendsistakenfromCanStockPhoto(Fig.7).Jonhasadjustedtheoriginalimagebyrotating it180degreesclockwiseandhasattachedamobilephonedisplayingthePGgame.Thephoneattractstheviewer’s attentionduetothecontrastincolor,establishingsalience.ThisexampleshowshowJonhasusedexistingmaterialto makeanewsignandinthisremixprocess,hastransformedorredesignedthesemioticresourcestofitthenewcontext.
Positivt
Nårspillereerutemedpokémon go,ermanmeruteennhvisman baresitterinneåspilleronline.Og nårmanerute,kanmanfånye venneråmøteoppsammen senerepåpokémonjakt.
Theimagepresentstheunfoldinggameactivityamongfriends,correspondingtothetopicofJon’swrittentext.The mobilephonethatJonhasplacedinthehandofoneofthegame-playingfriendsdisplaysascenefromthePGgame.
Inotherwords,thegamingprocessisrepresentedbothlinguisticallyandvisually,creatingmultimodalcohesion.The imagerestatestheinformationinthewrittentext,aformofinformationlinkingreferredtoaselaboration(Kress&
vanLeeuwen,2006).
Representationssuchastheseshowhowstudentsrelatetothesubjectandwhattheythinkisworthwhiletocom- municatetoareader. InJon’stext andillustration,the focalpositiveaspect ofplayingPG isbeingoutdoors with friends, whichpresumablyishow Joninterprets hisown PGgamingexperiences.In linewithManovich’s (2013) remixmetaphor,Jon’srepresentationsalsoreflecthisreceptionofthePGtechnology.Inthiscase,Jonhasmanagedto representabstractconcepts,suchasfriendshiporsociability,tiedtoanotherabstractphenomenon,thepositiveaspectof gaming,acombinationofentitiesthatwouldbedifficulttorepresentwithaliteralimage.Masteringabstractconcepts suchastheseisaprocessofmakinganalogies,comprehendingtheminconcreteterms,andfinallyrepresentingthese ideasusingthemostaptsemioticresources.Here,thenewlymadecollagerestsonormaterializesthesignmaker’s interest,anditisamotivatedsignthatpossiblyservestopersuadelessinformedreaderstoengageinthegame.The playfultext/imagerelationalsoindicatesaudienceawareness.
Thenextexample(Fig.8)dealswiththenegativeaspectsofplayingPG.Theopeningparagraphcorrespondstothe previousone:“Whenyouareout(gaming)withfriends,accidentseasilyhappen.Example:traffic.Someonebrokea
Fig.8.NegativeaspectsofplayingPG.
leginFrognerparkeninOslo,soaccidentsmighthappen. .”7Here,theuseofthegrimacingfaceemojiisanexample of howthetext-messagegenrepermeatestheiconographyofcontemporarychildhoodandsubsequently,children’s schoolwork.FivemorestudentshaveusedvariousemojisintheirPGtexts,andtwostudentshaveintroducedanew paragraphwiththegraphicelements ˆ-ˆ,representingthefaceofPikachu,themostpopularcharacterinthePokémon series.
Negativt
Nårmanerutemedvennerkan detfortskjeulykkereks:trafikk.
Noenbrakkbenetsittnårdevari
FrognerparkeniOslo,sådetkanskjeuhell.
Thefocalnarrativeelementintheprecedingexcerptistheaccident,asemioticpatternestablishedtextually(“neg- ative,”“accidents,”“brokealeg”),intheuseoftheemoji,andintheillustrationofthecarhittingthePG-gamer.This combinationcreatesmultimodalcohesion.Thisillustration(Fig.8)fromPixabayhasavector,comprisinglinessigni- fyingthespeedandtheimpact.Thecontrastbetweenthewhitedriverandtheblackpedestrianprovidesdirectionality.
TheimageofthemobilephoneisreusedasthesalientfeatureremindingtheviewersthatthisisPGgaming.
Sixof thestudentshavenotdiscussed thepositiveandthenegativeaspectsofplayingPG,andonlythreehave chosentoillustratethispartofthetext.WhileJonhasarrangedmetaphoricalcollages,twootherstudentshaveinserted literal images (i.e.,photographsof a cardriver holdingamobilephone, leavingit up tothe audienceto imagine that the driveris playingPG onhis phone). Thisexample showsthat when readymadeimages torepresentJon’s ideasareunavailable,hemakesnewimagesbycombiningthem.Thevisualrepresentationof positiveandnegative aspectsisaresultof mappingpertinentfeatures,whereJonassociates“positive”with“friendship”and“negative”
with“accidents.”Mostofhisclassmates(40)havediscussedPGgaminginrelationtobeingwithfriendsanddoing physicalexerciseoutdoors,whereasthenegativeaspectsof playinghavebeenlinkedprimarilytotrafficaccidents (38).Othernegativeaspectslistedinthetextsaregamingaddiction(29)andthetimeandthemoneyspentongaming (10).
5.4. Martin’sPGtext:thewalkthroughgenre,audience-awarenessandtheinsiderperspective
“This isadescriptivetext aboutPokémon Go written forapersonwho hasneverplayedthegame.”Thisfirst sentenceinMartin’sintroductionsetsthetoneofvoicefortherestofhistext,whereheinstructsthereaderonhowto downloadandplayPG.
7 TheincidentreferredtobyJonandhisclassmatesoccurredataPG-LurePartyinOsloin2016andwasfeaturedinthenews.
Fig.9.ExcerptfromMartin’sPGtext.
Thefollowingexcerpt(Fig.9)showshowMartinpresentssomeofthemostimportantPGterms(Gym,Pokéstop, andPokédex)inseparateindexedparagraphs.Thiscreatesaproceduralreadingpaththatsimplifiesthereader’swork.
Inthefirstparagraph,MartinpresentsthethreePGteams(Mystic,Valor,andInstinct)intheirrespectivecolorsand instructsthereadertoselectateam.Furthermore,heinstructsthereaderonhowtobattleagym:“Youneedtobeat level5tocompete.Whenyouwinthegym,youpressOKandyou’llgetXP(...).”Martinhasleftoutthefactthat therearetwokindsofgyms(rivalandfriendly)andthatrivalgymbattlesentailloweringanopponent’sprestigelevel.
However,hetellsthereaderhowtoclaimanewPokémon:“IfyoucatchanewPokémon,akindoffilmappearsandit says‘registeredtopokédex’(...).”Healsosharesaninsidertiponwheretohunt:“OsloisthebestplaceforPokemon huntingandistheplacewiththemostPokestops,inNorway.”
Martin’stextsharesthesamelogicsandcontentasthoseofmanyofthePGparatexts.Thestructureandthelayout aresimilartothoseofthePGglossariesofcommonlyusedterms,andthereader-orientedinstructionalstyleresembles theonefoundingamewalkthroughs.Martinaddressesthereaderdirectlyusingthepronoun“you,”atechniquethat pulls usersintothe informationandmakesit relevant tothem.Furthermore, hehas chosen towritethePG lingo (e.g.,level,items,registeredtopokédex)inEnglish,followedbytheNorwegiantermsinparentheses.Theseexamples indicatethat Martinunderstandsthe walkthroughtextasagenreandpurposefullyusesthispriorgenreknowledge tocomposehisowntext.Furthermore,theinsiderinformationthatMartinsharesistypicalofPGstrategyguides,as thisgamedoesnotofferatutorialonhowtoplayit.Anotherconsiderationregardingthegameisthatplayersneed informationonPokémons’location(Tran,2018).Martin’stextattendstoboththeseaspects.Hepositionshimselfas aPGexpert,an“insider”whopresentstheterminologyandsimplifiesthegamingprocedurestothelevel ofaless informedreader/player(Gee,2007).Inthiscase,theinsiderperspectivemightbethereasonwhyMartinhasforgotten toincludethepointofjoininggymbattles.Asfightingforhonorandterritoryisacommonnarrativefeatureofmost games,this“goeswithoutsaying”inMartin’stext.Inotherwords,thiselementistacitknowledgeingamingliteracy (Arduini,2018;Gee,2004).
6. Discussion
Asshownintheanalysis,JonandMartindemonstrate purposefulandinnovativeusesofsemioticresources in self-representation,stance taking,andaudienceawareness.The analysisalsoindicatesthe useof theirpriorgenre knowledgeofthePGparatexts(fandomsandwalkthroughs).TheboysareparticipantsofamakerculturearoundPG, andtheirgame-basedliteracypracticesbothinformandinspiretheirtexts.Theydemonstratedigitalliteracyskillsand exploredesignsthatarespecifictogamers,which,inturn,crossovertothedigitalenvironmentoftextcompositionin school.Inthefollowingparagraphs,thekeydidacticimplicationsofsuchcrossoversarediscussed.
The first step of the PG assignment was to play the game inthe schoolyard. Focusing on the actual gaming experience allowsthe studentsto presentthe game fromthat experience andbe agentiveandengagedwriters, as
reflectedinJon’sandMartin’stexts.TheexamplesfromJon’stextsshowthathegivesdesignedprominencetothe images(e.g.,logo,fanart,andcollages).Hecomposesmultimodalrepresentations invariousdesignprogramsand createsmultiplelayerstoassemblehis finalimages.Martinalsopresentshimselfasagamerandpositionshimself asaninsider/expert.Inthetext,hetakesonthetutorrole,mimickingthelayoutpracticesingame-relatedparatexts.
ThevisualandlinguisticconnectionsthattheboysmakewiththePG-makercultureconstituteanimportantdriving forceintheiractiveself-representationas gamersandsignmakers.Invitinggaminginto theclassroomisoneway of connectingschoolliteracywiththestudents’ownlife-worlds, shapingopportunitiesfor identityexpressionand play(Daniel-Wariya,2016;DeLuca,2018).Furthermore,applyinggamingtocompositionclassrooms couldfoster students’abilitiestocollaborate,analyze,andrecognizevisualcues(Sabatino,2014).Nonetheless,notallthestudents arePGplayersorinvolvedingame-relatedaffinityspaces.Thefindingsalsoillustratetheneedtoprovidestudentswith richandwide-rangingexamplesofmultimodaltextsthattheymightmodel.Extendingliteracyeducationbeyondthe schoolmeansincludingandaddressingthelinguistic,semiotic,anddigitalassetsthatthestudentspossessinvarious affinityspaces.Thisapproachalignswithothermultimodalstudies’callforacurriculumthatconnectswithstudents’
out-of-schooldigitalandmultimodalrepertoires(Jewitt,2008).
Intheselectedmultimodalrepresentations,differentsemioticmodes(imagesandwords)worktogetherinstance- taking acts,broadlydefined asaposition takenby thewriters inrelationtothe contentandthe audience(Barton
&Lee,2013,p.31).Theimagesareeditedinwaysthat indexacertainkindofpositionof thestancetakers(e.g., gamers,producersoftaboofanart,etc.).Stancetakersmayalsoopttoprojecttheirsenseofselfthroughaparticular styleofwriting,suchasMartin’suseofanexpertvoicewhendescribingthegame.Discussingagameorgamingin writingallowstheexplicitteachingofmultimodalstancetaking.Instructor-leddialoguescouldfocusonthesemiotic resources available to students,their appliedvisual andlinguisticmarkers of subjectivity andaffectivity, andthe rhetoricaltechniquesthatcanbeusedtoaccomplishtheirgoals.Thesedialoguescouldinturnhelpstudentsrealize thattheirrepresentationalchoicesareactsofpositioningandexpandtheirliteracyoptions.Cultivatingmeta-awareness aboutwriting,language,andrhetoricalstrategiesmightpositivelyimpactthewaysthatstudentsassessandadaptto writingdemandsacrossdisciplines(DePalma&Alexander,2015).
DePalma and Alexander have also described the challenges related to audience adaptation that graduate and undergraduatestudentsfaceincarryingoutmultimodalcompositionactivities,primarilybecausethestudentsview multimodalcompositionasopposedto“academic”writingandperceivethe“public”asanill-definedmass(2015,p.
186).Incontrast,theaudienceinthePGassignmentiswell-defined,andthestudentsareabletoaddresstheirreaders.
Writing adescriptivetextto“apersonwhohasneverplayedPG”assignsexpertrolestothestudentsandrequires textsthatareadaptedtoassistthelessinformedreader.AspointedoutbyDePalmaandAlexander(2015),writingas experts,studentscanfocusonthepurpose,audience,andcontentofinstructionalwriting,insteadofaimlesslylisting thestepsofarandomprocess.Focusingonaspecificaudience,mostofthestudentsarealsoabletoselectamong modalitiesandadaptsemioticresourcestosuittheobjectiveofthePGtask.
AspositedbyHowell(2017),PGisamultimodaldigitaltextwithitsownaffordances,whichmeansthatitcanbeused tointroducekeytermsinmultimodalcomposition.Applyingnewtermstoafamiliargenrecouldhelpstudentsassess thepossibleusesofdifferentmodalities(functionalspecialization)anddevelopamorenuancedunderstandingofthe uniqueaffordancesofmodesandresources(Kress&vanLeeuwen,2006).InherresearchonPGasaugmentedreality, BrentaBlevins(2018)hasemphasizedthatPGmakesmeaningthroughthelayersofmultiplemodesandhassuggested thetextuallayerasacomposingconcept.Identifyingthelayeredmodes(linguistic,visual,auditory,gestural/haptic, sound, andcombinations)availableingamescould helpstudentsgainproficiencyinanalysis andcomposelayers ontheirown(Blevins,2018).BuildingonManovich’s(2013)views,Blevins’approachencompassesteachingabout composingconceptsandgivingexplicitsoftwareinstructioninpreparingstudentstobecomecommunicatorscapable ofcomposinginmultiplemedia.
NLSscholarshaveexaminedcompositionasanactthatisperformedacrossmultiplemodes,andiPadsrepresent the latestinaseriesof technology-basedlearningaidsthat facilitatesuch multimodalcomposition.Students,here representedbyJonandMartin,embracethemultimodalpossibilitiesaffordedbyiPads.First,theythinkaboutnotonly thecontentoftheirtextsbutalsohowthetextisdisplayedonscreen,andtheymaketheirrepresentationalchoices accordingly.Second,theliteracyappsprovidedbytheschool,suchasthevideoapps,Pages,andPhotoGenearealso employedintheproductionof posters,fanart,andwalkthroughs.Finally,theboys’actionsthrough theiriPadsare transformative;thedigitaltechniquesandtheworkmethodstheyapplyinvariousaffinityspacesaresimilartothose thattheyuseinmultimodalcomposition.Theboys’familiaritywithcommunicatinginvariousdigitalspacesandtheir
transfersoftechniquesandwaysofrepresentationbetweenthesespacesalignswithManovich’scharacterizationof
“prosumers”(2013,p.205).Byrecognizingthesekindsoftransferrableprosumerpractices,instructorsofmultimodal compositioncanbetterpreparestudentsforthecommunicationpracticesofthetwenty-firstcentury(Arduini,2018;
Erstadetal.,2007).
TheclassroomwhereIhaveconductedmyresearchhasarangeofunofficialliteracyactivitiesthatarerelatedto,but notpartof,thePGwritingtask.Thetwoboys’playfulcreationsoflogos,headings,andfanartareinthemselvesacts ofstancetaking,demonstratingtheiraffiliationwiththePGuniverse.Theseactivitiesruninparalleltotheteacher-led compositionclassanddonotfindtheirwaytotheteacher’sradarscreen.FollowingMaybin(2007),Ihavechosento presenttheseactivitiesasfocalliteracyevents.Thereasonisnotbecausethattheyaremoreinterestingthantheofficial activityofcomposingdescriptivePGtextsbutthattheyhelpmakevisiblethemultimodaldesignsinwhichthestudents areengaged,aswellasthrowintosharprelieftheskillsthatmightotherwisebehiddentopractitioners(Blevins,2018;
Maybin,2007;Walsh,2010).Foregroundingthese“off-task”activitiescouldalsofuelthediscussionsonwhatcounts asliteracy(Juuhl&Michelsen,2020;Wohlwend,2017),andthusinfluenceandalterconceptionsaboutschoolliteracy practices.
7. Limitations
Thissmall-scalestudy’sfindingscannotbegeneralized.Nonetheless,it providesawindowintothe educational value of atechnology such as PGandhow iPads canfacilitate crossoverskillsinmultimodal composition.Most multimodalstudiesanalyze(oftenindetail)selectedsmallfragmentsorsmallsetsofmultimodaltexts(Jewittetal., 2016).Foregroundingtwostudents’practicesandtextsenablesmetogiveduerecognitiontothecharacteristicsof multimodaldesignandtotheirinterestsassignmakersbutpreventsathoroughpresentationofthesemioticcomplexity andrichnessofthedata.Asmyresearchdesignprivilegesthestudents’voicesandtheirmultimodalrepresentations, Ihavenotexploredtheexplicitteachingofmultimodalityindigitalenvironmentsortherelationshipbetweenteach- ers’pedagogicalbeliefsandiPaduseinmultimodalcomposition.Thisunderscorestheneedforfurtherresearchon multimodalityanddigitalliteracy.
ElinStrømmanisanAssociateProfessorandadoctoralstudentinNorwegiandidacticsinTheDepartmentofTeacherEducationattheNorwegian UniversityofScienceandTechnology(NTNU).HerongoingPhDproject,‘WritingwithiPads,’focusesondigitalandmultimodalliteracypractices.
Herresearchasksquestionsrelatedtotechnology,writing,andpedagogy,andshehaspublishedworkinNordicJournalofDigitalLiteracyand NordicJournalofLiteracyResearch.
References
Arduini,Tina.(2016).Toolsofplay:DevelopingapedagogicalframeworkforgamingliteracyinthemultimodalcompositionclassroomDissertation submittedtotheGraduateCollegeofBowlingGreenStateUniversity.https://etd.ohiolink.edu/.(Accessed19May2020).
Arduini,Tina.(2018).Cyborggamers:Exploringtheeffectsofdigitalgamingonmultimodalcomposition.ComputersandComposition,48,89–102.
https://doi.org/10.1016/j.compcom.2018.03.006
Assunc¸ão, Carina, Brown, Michelle, & Workman, Ross. (2017). Pokémon is evolving! An investigation into the development of the Pokémon community and expectationsfor the future of the franchise. Press Start, 4(1), 17–35. https://pdfs.semanticscholar.org/57a5/
cb6f77f00f49499877da8cd5e5740d7f448b.pdf
Barton,David.(2007).Literacy:Anintroductiontotheecologyofwrittenlanguage.NewJersey,USA:Wiley-Blackwell.
Barton,David,&Lee,Carmen.(2013).Languageonline.Investigatingdigitaltextsandpractices.USAandCanada:Routledge.
Barton,David,&Papen,Una.(2010).Whatistheanthropologyofwriting?InD.Barton,&U.Papen(Eds.),Theanthropologyofwriting.
Understandingtextuallymediatedworlds(pp.3–32).London:Continuum.
Blevins,Brenta.(2018).Teachingdigitalliteracycomposingconcepts:Focusingonthelayersofaugmentedrealityinaneraofchangingtechnology.
ComputersandComposition,50,21–38.https://doi.org/10.1016/j.compcom.2018.07.003
Daniel-Wariya, Joshua. (2016). A language of play: New media’s possibility spaces. Computers and Composition, 40, 32–47.
https://doi.org/10.1016/j.compcom.2016.03.011
DeLuca,Katherine.(2018).Sharedpassions,sharedcompositions:Onlinefandomcommunitiesandaffinitygroupsassitesforpublicwriting pedagogy.ComputersandComposition,47,75–92.
DePalma,Michael-John,&Alexander,KaraPoe.(2015).Abagfullofsnakes:Negotiatingthechallengesofmultimodalcomposition.Computers andComposition,37,182–200.
Erstad,Ola,Gilje,Øystein,&deLange,Thomas.(2007).Re-mixingmultimodalresources.MultiliteraciesanddigitalproductioninNorwegian mediaeducation,Learning.MediaandTechnology,32,183–198.https://doi.org/10.1080/1743988070134339
