Achieving agility and quality in product development -an empirical study of hardware startups

(1)

ContentslistsavailableatScienceDirect

The Journal of Systems and Software

journalhomepage:www.elsevier.com/locate/jss

Achieving agility and quality in product development - an empirical study of hardware startups

Vebjørn Berg

^a^,^∗

, Jørgen Birkeland

^a

, Anh Nguyen-Duc

^b

, Ilias O. Pappas

^a^,^c

, Letizia Jaccheri

^a

aDepartment of Computer Science, Norwegian University of Science and Technology, Sem Sælands vei 9, Trondheim 7034, Norway

bDepartment of Business and IT, University of South-Eastern Norway, Lærerskoleveien 40, Notodden 3679, Norway

cDepartment of Information Systems, University of Agder, Universitetsveien 25, Kristiansand 4630, Norway

a rt i c l e i n f o

Article history:

Received 6 January 2019 Revised 16 March 2020 Accepted 8 April 2020 Available online 22 April 2020 Keywords:

Startup Hardware startup Software engineering Product development Empirical research

a b s t r a c t

Context:Startupsaimatscalingtheirbusiness,oftenbydevelopinginnovativeproductswithlimitedhu- manandﬁnancialresources.Thedevelopmentofsoftwareproductsinthestartupcontextisknownas opportunistic,agility-driven,andwithhightolerancefortechnicaldebt.Thespecialcontextofhardware startupscallsforabetterunderstandingofstate-of-the-practiceofhardwarestartups’activities.Objec- tive:Thisstudyaimedto identifywhetherand howstartupscanachieve productqualitywhilemain- tainingfocusonagility.Method:Weconductedanexploratorystudywith13hardwarestartups,collect- ingdatathroughsemi-structuredinterviewsandanalysisofdocumentation.Weproposedanintegrative modelofagilityandqualityinhardwarestartups.Results:Agilityinhardwarestartupsiscomplexandnot achievedthroughadoptionoffast-paceddevelopmentpracticesalone.Hardwarestartupsfollowaquality- drivenapproachfordevelopmentofcorecomponents,wherefrequentusertestingisameasureforearly debtmanagement.Hardwarestartupsoftenlackmindsetandstrategiesforachievinglong-termqualityin earlystages.Conclusions:Hardwarestartupsneedattentiontohardwarequalitytoallowforevolutionary prototypingandspeed.Futureresearchshouldfocusondeﬁningquality-drivenpracticesthatcontribute toagility,andstrategiesandmindsetstosupportlong-termqualityinthehardwarestartupcontext.

ThisisanopenaccessarticleundertheCCBYlicense.(http://creativecommons.org/licenses/by/4.0/)

1. Introduction

Startups, newly created companies producing cutting-edge technology,areanimportantsourceoftechnologyinnovation,and have a signiﬁcant impact on the wave of digital transformation (Jacobsonetal.,2017).Despitestories ofsuccessfulstartups,most ofthemfail,primarily duetoself-destructionratherthancompe- tition (Crowne, 2002;Marmeretal., 2011). Without previousop- erationalexperience,startupsoftenneedtolearnhowtoestablish newroles,newconnectionstoexternalstakeholders,andnewpro- cessesandpractices(Stinchcombe,2000;Abatecolaetal.,2012).In astartupcompanydevelopinghigh-techproducts,besidespersonal traitofstartupfoundersandﬁnancialandmarketfactors(Giardino etal., 2015;AldrichandAuster,1986;Van Gelderenetal., 2005), product development is also a key factor characterizing the development ofthe startup(Unterkalmsteiner etal., 2016;Giardino

∗ Corresponding author.

E-mail addresses: [email protected] (V. Berg), jorgen.birkeland1@gmail.

com (J. Birkeland), [email protected] (A. Nguyen-Duc), [email protected] , [email protected] (I.O. Pappas), [email protected] (L. Jaccheri).

et al., 2015; Tripathi et al., 2016; Giardino et al., 2016; 2014a).

For instance, software research has shown interest in achieving effective Minimum Viable Products (Nguyen-Duc and Abrahams- son,2016) andmanagingtechnicaldebt(Giardinoetal., 2016) in thestartupcontext.Eventhoughtheobstaclestosuccessgradually become known and aware to entrepreneurs, the startup context posesseveraluniquechallengestotraditionalproductdevelopment and innovation methods (Unterkalmsteiner et al., 2016; Nguyen- Ducetal.,2016).

Thepartofstartupecosystemsthat isrelativelylittleexplored in research is hardware startups. They include startup companies developing products and services with a value proposition based on an integral solution of software and hardwarecomponents(DiResta et al., 2015; Jacobson et al., 2017). Hardware is a physical,tangible part of a system, ora systemof systems (e.g., sensors,gateways,connectivitycomponents,wearabledevices,mo- bile phones), while software is a code-based, intangible part of thesystem(e.g.,operating systems,server-sidescripts, databases, algorithms). A typical example for a modern hardware system is a connected house, where the hardware part is implemented to measure, collect and transmit data, and the software part is https://doi.org/10.1016/j.jss.2020.110599

(2)

used to coordinate the operations of hardware, store and pro- cessthecollecteddata.Thebarriers forstartingahardwarecom- panyhaveneverbeenlower,aresultoftheadvanceddevelopment ofhardwaretechnology. Rapidprototyping,decreased component costs, small-batch manufacturing, andfundraisingplatforms have renewedthe interest for hardwarestartups (DiResta et al., 2015;

Wei,2017).

Hardware startupsadd additional complexitytosoftwarestar- tupsas they need to handlethe development andintegration of hardwarepartsintotheofferedproducts(Nguyen-Ducetal.,2018).

Hardware products usually need to be secured and safe, which putsa focuson ensuring quality attributesofdelivered products.

Moreover, the quality ofthe whole product relieson the quality ofits integrated components, both software and hardware mod- ules.Whileitisknownthatsoftwarestartupsfocusonspeedand agility,remaininglowpriorityonqualityassurance,itisnotknown ifthesame practicesoccur in hardware-relatedproduct development.Whileknowledge fromdevelopmentofembeddedproducts inestablishedcompaniescanberelevant(Kaistietal.,2013;Albu- querqueetal., 2012),the”newnessandsmallness” natureofstar- tupscallsforaninvestigationandfurther,an adaptionofexisting methodologies andpractices that are suitable to startup context (Bosch,2016).

Softwarestartups areknownforfast-paceddevelopment, with ability to handle uncertainty, react to changes in product and business development, and introduce ﬂexibility in the process (Garbajosaetal.,2017).Theconceptofagilityinhardwarestartups mightbe differentfrompure softwaredevelopment, ashardware developmenttypically involvesa longdevelopmentcycleandde- pendsonalarger setofthird-partyvendors.The relationshipbe- tweenagility andqualitymight bemorecritical insome circum- stances,forinstancestartupcompanieswhodeliverquality-driven products. For example, the Norwegian startup Prediktor Medical AS develops a glucose smartwatch that measures glucose level withoutpenetratingpeople’s skin.The productwasquality-driven andhasbeendevelopedunderamarket-pressurewithapromised launch time. A recent industry survey also calls for systematic adoptionofproductdevelopmentmethodologiesinhardwarestar- tups(Nguyen-Ducetal.,2018).

To thisend, we seekto createa betterunderstanding of work- practicesin hardwarestartupsbyinvestigating the roleof engineer- ing activities,from idea conceptualization to a launched product. In particular, we will investigate factors inﬂuencing agility and qual- ity, and explore commonalities and challenges. As mentioned by Jacobsonet al.(2017), literature regardingmethods for hardware product developmentis scarce. We aim atexploring how agility andquality are managedin practice. Thishas motivatedthe fol- lowingresearchquestion:

RQ Howdohardwarestartupsachievebothagility andproduct qualityduringproductdevelopment?

Thispaperpresentstheresultsfromaqualitativesurveyinves- tigating13early-stageEuropeanhardwarestartups.Theworkcon- tributestostartupengineeringresearchby focusingonhardware- intensiveproductdevelopment.Theresearchprovidesearlyempir- icalevidence to agility in hardwarestartups,and simplequality- aware practices in the context of restricted resources. The work also builds the foundation for researchers and practitioners to further explore hardware startup engineering, which is still in a nascentstage.

The remainderofthispaperproceedsasfollows:Section 2in- troduces the background of the study and relevant theoretical frameworks.Section 3 presents the research method undertaken andpotential threatsto thevalidity.Section 4reportsthe results ofthestudy,includingtranscribedcitationsfromtheparticipants.

Section5discussestheresultsinrelationtotheresearchquestions.

Section6concludesthepaperbyansweringtheresearchquestions andproposingdirectionsforfuturework.

2. Background

2.1. Thecontextofhigh-techstartupcompanies

The term “startup” has been deﬁned differently across various principles (Steininger, 2019;Sutton, 2000; Ghezzi, 2018; Un- terkalmsteiner et al., 2016; Crowne, 2002). From the recurrent themes on startups, high-tech startups share common charac- teristics of organizations focusing on the creation of software- intensive products, with little or no operating history, aiming to growbyaggressivelyscalingtheirbusinessinhighlyscalablemar- kets (Giardinoetal., 2016). Thecontextofstartupsislongunder- stood as a special organizational state. New companies generally involvenewroles,andthe“coordinationofstrangers” scenarioof- tenleadtolowqualityofperformance(Stinchcombe,2000;Abate- colaetal.,2012).Sommeretal.(2009)highlightedthatnewcom- paniesoftendonotcorrectlyforeseerealopportunitiesorthebest ways of addressing them, and so are forced to adapt and mod- ifytheir approachovertime.Giardinoetal.(2014b) revealedthat startupsoftenfail toachievetheproblem-solutionﬁtduringtheir execution. Fromearly-stage, startupsincreasetheir learningcurve andfostertheestablishmentofsurvivaldeterminants(i.e.,success- fulpracticesandprocedures)(Abatecolaetal., 2012;Hodgsonand Knudsen,2004).

2.2. Softwareproductdevelopmentinstartupcompanies

Startups generally develop products in high-potential target marketswithoutnecessarilyknowingwhatcustomerswant(Blank, 2013b; Raﬁq et al., 2017). Increasingly more industries experience that new technologies become available to all players at the same time, hence the beneﬁts of technology-driven innova- tions decrease. This has led companies to prioritize customer- drivendevelopment,whichinvolvesidentifyingnewandunknown customer needs as well as meeting known needs (Bosch, 2016).

This relates to market-driven software development, where re- quirementstend tobe (1)inventedby thesoftwarecompany, (2) rarely documented (Karlsson etal., 2002), and(3) validatedonly aftertheproductisreleasedinthemarket(Carmel,1994;Dahlst- edt, 2003; Keil and Carmel, 1995; Raﬁq et al., 2017). Products not meeting customer needs are common, resulting in failure of newproduct releases (Alves et al., 2006). There existseveral en- trepreneurialtheoriesandframeworksthatcanguidepractitioners intheirpursuittolastingbusinessgrowth,including“Effectuation Theory” (Sarasvathy, 2001), “DiscoveryandCreation” (Alvarezand Barney,2007),thecustomer developmentapproachintroduced by Blank(2013a),andtheLeanStartup(Ries,2011).

Research on software engineering depicts that startup companies prefer to prioritize time and cost over product quality (Yau and Murphy, 2013), neglecting traditional process activities like formal project management, documentation, andtesting (Giardino etal.,2016). Shortcutstakenin productquality,design, or infrastructure can inhibit validated learning (Ries, 2011), in a contextwherecustomizeddevelopmentpracticesarenecessaryto manage thechallengesposedbycustomer developmentmethods.

Inadequate useof softwareengineering practicesmight be a sig- niﬁcantfactorleadingtothehighfailureratesofsoftwarestartups (Klotinsetal.,2015).

Entrepreneurs are in general aware of the signiﬁcance of how their products are built. Even though studies have found that startups are either reluctant to introducing process (Coleman and O’Connor, 2008), or that they use their own mix

(3)

of Agileand ad-hoc methods (Giardino etal., 2014a), manystar- tupsemphasizedtheimportanceofhavinggoodpracticesinbuild- ing their products (Sutton, 2000; Giardino et al., 2014a). Small early-stagesoftwarestartupsdon’texperiencethesamechallenges aslarger, moreexperienced companies, andthe costandtime of implementing a rigorousAgile methodologymay not providebig enoughbeneﬁts(YauandMurphy,2013).

2.3. Agilityinproductdevelopment

Agility as a concept is multi-facet and in many cases refers to the ability of an organization, a team, or a project to react to changes occurred to them (Conboy, 2009). In a general sense, agilitycan bedeﬁnedas”the capabilitytoreact andadapt to expected andunexpected changes within a dynamic environ- ment constantly and quickly,and to usethose changes (if possible)asanadvantage” (BohmerandLindemann,2015). Insoftware development, Agile methods have proven to be a powerful tool when the goalis tobuild a successful, proﬁtablebusiness model (Cunninghametal.,2001).When acompanyneedstoquicklyad- dress market and customer needs, Agile processes have proven to be much more effective than traditional high-ceremony processes (Wasserman, 2016). Since the birth ofthe Agile Manifesto (2001), withstatedprinciplesandpractices ofAgilemethodology (Becketal.,2001),ithasbecomeapopularsetofpracticesinthe softwareindustry toreplacetraditional,rigid, andheavysoftware developmentprocesses.

Duringthelastdecades,Agileinsoftwareengineeringhasbeen an extensive research area with an enormous amount of literature(Dybå andDingsøyr,2008;Conboy,2009;Abrahamssonetal., 2010;Díazetal.,2011;Misraetal.,2012;JalaliandWohlin,2010;

DaSilvaetal.,2011).Existingstudiesprovidetheintroductionand adoption ofAgilemethods andtheir variance indifferentorgani- zational settings.Theydo not agreeon a unifiedview of current practices, butoffer a broad pictureof experience andsome con- tradictory findings (Dybå and Dingsøyr,2008). Benefits were re- ported in the following areas: customer collaboration, work pro- cessesforhandlingdefects,learninginpairprogramming,thinking ahead for management, focusing on current work for engineers, and estimation(Dybå andDingsøyr, 2008). Arecurring theme in studiesonAgiledevelopmentishumanfactors(e.g.,teamdynam- ics, team coordination, customer involvement,etc.) and their in- fluence on Agile development. Much research reports experience ofcombiningAgiledevelopmentwithother SoftwareEngineering paradigms, such as distributed teams (Jalali and Wohlin, 2010), product line development (Misra et al., 2012), anduser-centered design(Díazetal.,2011).Thecombinationofproductlinedevelop- ment,withthefocusonupfrontinvestments,planning,design,and Agile methods,with thehighlight ofrapidand frequentchanges, attention to the design is found challenging (Misra et al., 2012).

Several practices are investigated in the fusion of Agile methods intomorerigidprocesses,includingreleaseplanning(Hanssenand Fægri, 2008)andthebottom-upapplicationdriven approachwith automatedacceptancetests(GhanamandMaurer,2010).

ResearchsuggeststhatAgilemethods aresuitable forsoftware startups, as iterative development approaches are adaptive, with shortleadtime (Pantiuchina etal., 2017;Paternosteretal., 2014).

TheadoptionofformalsetsofAgilepracticesandmethodsinstar- tups is limited, often due to an excessive amount ofuncertainty andhightime-pressure(Giardinoetal.,2014a).Startupsoftenuse a tailoredversionof Agiledevelopment,inmanycases,thequick combinationofAgileandothermethodologies.

2.4.Engineeringprocessesforembeddedsystemdevelopment Research on development processes in hardware startups is rare, where exploration of state-of-practice is limited to a few studies(Nguyen-Ducetal., 2018). Theprocessesandpracticesfor developinghardware-relevantproductshavebeenreportedinliter- atureaboutembeddedsystemengineering, whichconcernsabout application-speciﬁccomputingdevicesconsistingofhardwareand software components (Ronkainen et al., 2002). Current knowledge on development processes of hardware-related products in established companies is rarely transferred to hardware startups’

product development, as the startup context is unique and special(Nguyen-Ducetal.,2018;RonkainenandAbrahamsson,2003).

In the embedded domain, hardware sets strict requirements to software. Development ofhardware-intensive systems require si- multaneousdevelopmentofhardware-components andhardware- relatedsoftware(Ronkainenetal.,2002).Sincesoftwareallowsfor frequentupdatesandreleases,thesystemarchitectureoftenseeks to separate hardwarefrom software to allow for two largely in- dependent release processes (Bosch, 2016). This is illustrated in Fig.1wherehardwareandrelatedsoftwaredevelopmentare dis- tinct processes requiring constant communication and intercon- nectedtestingandveriﬁcation.

Ronkainenetal.(2002)foundfourmaincharacteristicsofhard- ware and relatedsoftware development, including (1)hard real- time requirements,(2)experimental work, (3)documentation requirements,and(4)testing.

1.Hard real-timerequirements(e.g.,datathroughputrates,cycle counts, orfunction call latency)mean thatifsoftware doesn’t meet requirements, further system operation may be at risk.

Hardware simulations can help determine the precise opera- tionofhardwarewithoutproducinganexpensiveprototypeand evenenabletestingofthehardware-softwareco-operation.

2.Hardware-oriented software development is experimental by nature,anddevelopersneedtounderstandthewholesystemto deal withall uncertaintiesrelatedtochanges inhardwareand howsoftwareaffectsthewholesystem.Everyrequirementcan- notbe knownandeverydecisioncannotbemadebeforewrit- ing software. Developers should utilize an iterative develop- mentapproachtodealwithallambiguitiesofhardware-related softwaredevelopment.

3.The communicationamonghardwareandsoftware developers must worktoimplementthehardware-software interfaceeﬃ- ciently. Informationhastobeexplicitandreliesheavilyonex- act documentationto minimizeinformationlossbetweeniter- ations.However,duetothevastamountofexperimentalwork, toomuchdocumentationisnotfeasibleinearlystagesofprod- uctdevelopment.

4.Testingisanessentialactivitybothduetoreliabilityanddevice autonomy requirements, andregression tests to ensure paral- lel developmentdoesn’tdrift.Inaddition toindependentsoft- ware and hardware tests, checking the right interaction be- tweenhardwareandsoftware(i.e.,co-veriﬁcation)isimportant toensurethesystemworksasintended.

RecentadvancementinhardwaretechnologysuggestthatAgile practicesalsocouldbeusedintheembeddeddomain(Kaistietal., 2013). AlthoughAgilemethods andpractices mayhaveapositive impact(e.g.,decreaseddevelopmenttimeandreducederrorrates) onproductdevelopment,theuseofAgileintheembeddeddomain isnotwidespread(Albuquerqueetal.,2012).Thereisaneedfora coherentunderstandingofhowAgilemethodologiesbestﬁttoem- beddedsystemsdevelopmentinthestartupcontext,andhowsuch practicescanreducecostsandeffortsindifferentphasesofthede- velopmentprocess(i.e.,requirementmanagement,design,andar- chitecture).

(4)

Fig. 1. Hardware-software co-design process ( Ronkainen et al., 2002 ).

Fig. 2. Research process.

3. Researchmethodology

Softwarestartupengineeringresearch istoagreatextentcon- cernedwithinvestigating thedevelopment, operation, andmain- tenanceofsoftwareandhardwareproducts instartupcompanies.

Inordertogatherandtointerpretevidenceforansweringourre- search questions, we devised a qualitative approach. The goal of qualitative research is to investigate and understand phenomena withintheirreal-lifecontext(Robson,2002).Dependentonthein- depthknowledgeinacase,thequalitativeresearchcanhaveanar- rowfocusonafewcasestudies,orabroaderscopeasaqualitative survey (Robson, 2002; Andersson and Runeson, 2002; Walsham, 1995). As the study’s overall goal is to characterize current sta- tusofadoptingagilityandquality-drivenpracticesinapopulation ofhardwarestartups, aqualitative survey appears to be suitable, especially when there is a limited capacity for capturing insight datafromanumberofcompaniesinashorttime(Anderssonand Runeson,2002).Robsonclassiﬁedfourtypesofresearchpurposes (Robson,2002):

• Exploratory - understanding what is happening; to seek new insights.

• Descriptive-portrayingasituationorphenomenon.

• Explanatory -seekingan explanationofa situationoraprob- lem,mostlybutnotnecessaryintheformofacausalrelation- ship.

• Improving- trying to improvea certain aspect ofthe studied phenomenon.

In linewiththe non-deterministicnature of product develop- mentinthestartupcontext(Nguyen-Ducetal.,2015)(i.e.,contexts for product development are highly inﬂuenced by team, ﬁnan- cial,market situations andentrepreneurial approaches),andwith the exploratorynature ofour research question,we exploratively investigate multiple startups.Klein and Myers differentiate three typesof research perspectives, positivist, critical,andinterpretive (Klein andMyers,1999; Walsham,1995). Positiviststudies search evidence fortesting hypotheses, drawing inferences froma sample; criticalstudies identify socialcritique, and interpretivestud- iesattempttounderstandphenomenathrough participants’inter- pretationof their context. Inthisresearch, we investigate aphe- nomenonthat integrateshumanfactorsandengineeringconcepts.

Hence, weadopted theinterpretiveview andcollecteddatafrom semi-structuredinterviews.

Thereareseveralpossiblelevelsofanalysis(e.g.,individual,ar- tifact,team,projectandcompany).Wechoseprojectasasuitable levelofanalysis,asthisstudyconcernsaboutproductdevelopment activitiesandprocesses,withcertainexpectationsabouttheinter- actions betweenthe products andtheir contextual environments.

The focusof ourinterviews isstartups’ singleprojectsthat leads to thelaunching oftheir coreproducts.Fig. 2illustrates all steps oftheresearchprocess.

(5)

Table 1 Startup channels.

Channel Description Link

Innovation Center Gløshaugen

The center is located at campus Gløshaugen, and houses various early-stage high-tech startups, mainly to support innovative students.

www.ntnu.no/ig NTNU Accel and FAKTRY NTNU Accel is a uni-based accelerator for promising startups. FAKTRY is an

incubator which is part of Accel, and houses various hardware startups. www.ntnuaccel.no , www.faktry.no Our professional networks Italian companies (S13), Spanish and Dutch companies (S11)

OsloTech and StartupLab OsloTech manage Oslo Science Park, including incubator StartupLab which has supported more than 200 startups since 2012.

www.oslotech.no www.startuplab.no The Hub The Hub is a community platform which gives an overview of Norwegian

and Nordic startups. Via the platform, startups can get assistance with recruitment and connection with investors.

www.hub.no

3.1. Companiesandsubjectsselection

Our research relies on theoretical sampling: purposive, non- probabilistic sampleswhich aretypically small,asa singleobser- vation issuﬃcientforinclusioninthecodingsystem. Researchers identify key participant, for instance, CEO, CTO or key engineers who has access to important information. To select appropri- ate participants, we chose criteria, assuggested by Runeson and Höst (2009). Startupswere relevant for inclusion in the study if theymetthefollowingcriteria:

• Thestartuphasatleasttwomembers,soproductdevelopment isnotanindividualactivity.

• The startup has been active for at least six months, so their experiencecanberelevant.

• The startup develops products or services that include both hardwareandsoftwareparts.

• The startup has a ﬁrst running prototype so it’s engineering practicesarerelevant.

Our sample in the survey is comprised of 13 hardware companies.Theyrepresentadiverseselectionofapplicationdomains, product typesandcompanycharacteristics,although theyare not systematicallysampledfromanylargerdistribution.

Peoplefromtherelevantstartupswereeligibleforparticipation if they had experience and/or knowledge about software and/or hardware development. If the candidate met the criteria, he/she wasregardedasqualiﬁedforcontributingtotheresearchstudy.

Viaprofessionalnetworksofco-authorsofthiswork,weiden- tified severalpotential sources of contacts,whichare co-working spaces,incubatorprograms, andtechnologyparks. Thefivediffer- entchannelsusedtofindrelevantstartupsare(1)InnovationCen- ter Gløshaugen, (2) NTNU Accel and FAKTRY, (3) our co-authors’

professional networks, (4) OsloTech and StartupLab, and (5) The Hub.Table1providesanoverviewofthedifferentcommunication channelsandcan helpother researcherstoﬁnd andcontactstar- tups(Table2).

There wasa mixofstartupsoriginatedfromacademica(7 out of13companies),entrepreneurs (5out of13companies),andin- dustryspin-off (1 out of 13 companies).The investigated startup foundershavevariedindustrialexperience,rangingfrom1tomore than 10years.Regarding entrepreneurial experience,ﬁvestartups are ﬁrsttime startups.The other eightstartups haveexperienced somefailurebefore.Regardingthebackgroundoftheinterviewees, themajority(12outof13companies)havetechnicalbackgrounds thatarerelevantfordevelopingproducts(Table3).

3.2. Datacollectionprocedure

Our chosen data collection method was interviews, identiﬁed asaneﬃcientmethodforanswering researchquestionsinexplo- rativestudies(Oates,2005).Thesemi-structuredapproachenabled discoveryofunforeseeninformationasintervieweescouldexpress

Table 2

Interviewee descriptions.

Company Role Background Gender

Startup 1 (S1) CEO Industrial Engineer M

Startup 2 (S2) CTO Informatics M

Startup 3 (S3) CTO Computer Science M

Startup 4 (S4) Hardware developer Cybernetics M

Startup 5 (S5) CTO Electronics M

Startup 6 (S6) Software developer Informatics M

Startup 7 (S7) CEO Electronics M

Startup 8 (S8) CEO Mechanical Engineer M

Startup 9 (S9) CEO Entrepreneurship M

Startup 10 (S10) Software developer Computer Science M

Startup 11 (S11) CEO Computer Science M

Startup 12 (S12) CEO Electronics M

Startup 13 (S13) Software developer Computer Science M

themselvesmorefreely,andﬁttedboththetimeconstraintsofthe project andthe availability ofstartup companies. We followed a questionnaireguidingthedatacollectionprocess.

• Section1:Warm-up

1. Tellusaboutyourcompanyatthecurrentstage 2. Whatwastheoriginalideas?

• Section2:AgilityandAgilepractices

1. Have you heardabout, orused any of the methodologies:

Agile,LeanStartup?

2. Howisthemethodologyimplemented?

3. How do external dependencies inﬂuence product development?

4. Howdoyoubalancehardwareandsoftwaredevelopment?

5. Howdoyoumanagedocumentation?

• Section3:QualityandQualityassurance 1. Howdoyoumanageproductquality?

2. Whendidyoulastrefactorthecodebase?

3. Towhat extent doyou reuse components ofearlierproto- types?

4. Howdoyouperformhardwareandsoftwaretesting?

5. Whendoyoustartwritingtests?

• Section4:Closing-up

1. What would you do differently withthe product development?

2. Anyﬁnalinterestingremarks?

Thefirstandsecondresearcherwereindirectcontactwiththe subjects,hence the data collection process can be regarded asa firstdegree datacollectiontechnique. First degree datacollection requiresa significant effort,butallowed both researchersto con- trolwhat data wascollected, ensuring that all pre-definedinter- view questionswere answered sufficientlyandexploring newdi- rectionsby askingfollow-upquestions(RunesonandHöst,2009).

Boththeﬁrst andsecond authorattendedall interviewsto avoid one single interpretation ofthe respondent’s perspective and in-

(6)

Table 3

Startup description.

Company Product Current Stage Founded Location # of employees

Startup 1 (S1) Smart gloves Startup 2016 Norway 18

Startup 2 (S2) Medtech biosensor Startup 2017 Norway 5

Startup 3 (S3) Physical exercise game Stabilization 2016 Norway 5 Startup 4 (S4) Unmanned aircraft system Startup 2016 Norway 7 Startup 5 (S5) Advanced noise cancellation Startup 2017 Norway 5 Startup 6 (S6) Medtech hydration monitoring Startup 2016 Norway 10 Startup 7 (S7) LPG management system Stabilization 2016 Norway 8 Startup 8 (S8) Cable cam system Stabilization 2016 Norway 10

Startup 9 (S9) Digital piggy bank Startup 2017 Norway 5

Startup 10 (S10) Collaborative camera Growth 2014 Norway 50 Startup 11 (S11) Interactive children’s toy Startup 2015 Netherlands 8 Startup 12 (S12) 3D-printer control board Growth 2009 Norway 1

Startup 13 (S13) Sensors for IoT Growth 2007 Italy 25

Fig. 3. Thematic Synthesis Process ( Cruzes and Dybå, 2011 ).

sightontopics,asqualitativedataoftencanberichandbroad,but lessprecise.

Beforetheinterviews,welookedintotheparticipants’business background,eitherthroughtheir companywebsitesorother rele- vantincubator oracceleratorwebsites. Additionally,mostpartici- pantsanswered a simplequestionnaireprior tointerviews where they filled out basicinformation aboutthemselves andthe company(AppendixB).Thesemeasures allowedformoreefficientin- terviews as the first and second author possessed more knowledge aboutthe interviewee and could use less time on formali- ties.Initial company analysisallowed fora holisticunderstanding ofeach company andprovided stronger evidencefor theconclu- sions drawn from the interviews. Each interview lasted between 40minand1h.Table3presentskey factsabouttheinvestigated companies.The sizeofthecompany provides insightintothe re- quired need for development process and managerial overhead.

The“Currentstage” isadapted fromthepaper(Crowne,2002),as appliedinthesystematicmappingstudybyBergetal.(2018),rep- resentingthestage of thestartups atthe time ofthe interviews.

Thestartupstagereferstotheperiodbetweenproductconception andtheﬁrstsale.Thestabilizationphasestartswhentheﬁrstcus- tomerreceives theproduct,whilethe growthphasebeginswhen aproduct is delivered toa newcustomer withoutdisturbingthe developmentteam.

3.3.Analysisprocedure

We applied the thematic synthesis process which is a codes- to-theorymodelforqualitative research (Cruzes andDybå,2011).

The objectiveof our thematic synthesis process was to both an- swertheresearchquestionsandcomeupwithamodelofhigher- orderthemesdescribingdevelopmentstrategiesinhardwarestar- tups,focusing onaspects that are uniquefromsoftware startups.

ThemainstepsoftheprocessareillustratedinFig.3. 3.3.1. Initialreading

The ﬁrst stepof theanalysisprocess wasto read through the transcribedinterviews to generate initial ideas andidentify pos-

siblepatternsinthe data.Allinterviewswere transcribedshortly afterthey wereconductedto ensuretheactual meaningofinter- viewees’answers. Allauthorsdiscussedthe interviews,creatinga mindmapofcentralconceptsrelevanttohardwarestartups.

3.3.2. Codingprocess

Togenerateinitialcodes,thefirstandsecondauthorapplieda descriptivecodingtechnique(Saldaña,2015),toidentifyinteresting concepts, categories,orother findings ina systematicwayacross thedataset.Descriptivecodinghelpedorganizeandgroupsimilar dataintocategories,whichisthefirststeptowardsthecreationof themes.

The coding process followed an integrated approach (Saldaña, 2015). This allowed us to avoid coding data out of context, while at the same time identifying what the text was sayingratherthanwhatwewantedtosee.Weappliedaniterative coding process, to allow for simultaneous data collection and analysis(RunesonandHöst,2009). Thecodingprocessresultedin atotalof49codesand734referencesfrom13interviews.Thefirst iterationinvolvedcodingthedatafromthefourfirstinterviews.A total of29 codes were generatedfrom 416references. The codes were examined by the first, second, and third author. Lessons fromthe evaluationwere implemented in thenext interviews to generate relevant codes. For the second iteration, we classified textintothecodesfromthefirstiteration,whileatthesametime generatingnewcodesinaninductivemanner.

3.3.3. Codesintothemes

Athemecan beseenasawayofgroupinginitial codesintoa smallernumberofsets,to createa meaningfulwholeofunstruc- turedcodes(CruzesandDybå,2011).Wedividedrelatedcodesinto categories andconcepts (Strauss andCorbin, 1998). All interview transcripts were analyzed separately to ensure that themes were inlinewiththeassociatedcontext.

3.3.4. Modelofhigher-orderthemes

Thegeneratedthemeswerefurtherexploredandinterpretedto createamodel ofhigher-orderthemes(AppendixA).The higher- orderthemeswereprototyping anddevelopment,qualityassurance, andenabling factors.Inaddition,weidentiﬁed patternsmoregen- eral to the startup context. The 14 themesin the thematic map were extracted to address management of Agility (as shown in Table5)andQuality(asshowninTable6).

3.4. Validityprocedure

Inqualitativeresearch,thevaliditymustbeaddressedtoenable replicationofresearchandtoensureﬁndingsaretrustworthy(Yin, 2003;CruzesandDybå,2011;RunesonandHöst,2009).Toensure

(7)

thevalidityofthisstudy,wefollowedthevalidityguidelinesfrom RunesonandHöst(2009).

Construct validity ensures that the operational measures that are studied really represent what the researcher have in mind and what is investigated according to the research questions.To assure that the interview questions (Section 3.2) were suitable foranswering ourresearchquestions,we deﬁnedinterviewques- tionsthroughatop-downapproachusingtheGoalQuestionMetric method.Intervieweeswere either CEOsorengineerswithinsight intobusiness-andtechnical-relatedaspects. Sinceitisdiﬃcultto understandastartupanditsdimensionswithinatime-spanof30 to60min,wecollecteddataaboutthestartupsthroughincubator andcompanywebsitespriortointerviews.Toimprovethereliabil- ityofthestudy,allparticipatingstartupswereincludedinthepro- cessofwritingcompanydescriptionstoensuretheirconformance withreality.

Externalvalidity refersto theextenttowhichtheﬁndings are generalizable beyond the context studied. For qualitative studies, the intentionis to enableanalytical generalization where there- sultsareextendedtocompanieswhichhavecommoncharacteris- tics.Ourstartupswere mostlylocatedinNorway, mainlyconsist- ing ofearly-stage small-size entrepreneurial teams.Theyare also mostlyself-funded andacquiringsome key competencefromthe start.Hence,itwouldbesafetorelytheﬁndingstostartupswith similarcharacteristics(i.e.,early-stageEuropeanstartups).Startups from other American countries or startups already in a growth stage,mightnotbeobservedwithsimilarfeatures.

Reliability refers to the extent that data and the analysis are dependent on the specific researchers.We have definedandval- idated interview protocols with colleagues. To decrease the risk of biasedinterpretations, authorone and two attended all interviews.SomeinterviewswereinNorwegian,hencetranscriptswere not always verbatim to preserve the actual meaning of respon- dents.Recordings weretranscribedshortlyaftereachinterviewto mitigate bias.Additionally,we compared findings torelatedliter- ature (Giardino etal., 2016; Nguyen-Duc et al., 2018; Ronkainen andAbrahamsson,2003),examiningsimilarities,contrasts,andex- planations.Suchcomparisonshaveproventoenhanceinternalva- lidityandthequalityoffindings(Eisenhardt,1989).

4. Results-Howdohardwarestartupsachievebothagilityand productqualityduringproductdevelopment?

4.1. Anintegrativeviewonagilityandproductqualityinhardware startupdevelopment

The integrative model of agility andquality in hardwarestartups is presented in Fig. 4.We have grouped the main concepts according to two dimensions (1) agility-driven or quality-driven, (2) projectactivities (i.e., prototypeand productdevelopment, or quality assurance). Eachconcept describes a common foundation in hardware startups that manage agility or product quality. We classiﬁedtheemergingconceptsintothreecategories:

• Mindset (represented by green boxes in Fig. 4): a belief, an opinion,orawayofthinkingtowardsatopic

• Practice(representedbypinkboxesinFig.4):theactualappli- cationofanidea,abelief,oramethodtosolveaspeciﬁctask

• Strategy (represented by yellowboxes in Fig. 4): a highlevel plan that might include a set of practices or processes to achieveagoal

As can be seen from Fig. 4, the integrative model of agility andqualityinhardwarestartupsfocusesonfourquadrantsontwo axes.Theverticalaxisshowstwomajoractivityareas(1)prototyping andproductdevelopmentand(2)quality assurance.The hor- izontalaxisshowsthearea ofAgilityorQuality.Byputtingthem

together,weofferanintegrativeoverviewofhowagilityandqual- ityare managedin bothproduct developmentand quality assur- anceactivities. Theﬁnal sectioninthemodelrepresentsenabling factors that apply to both quality and agility concepts. As seen fromthemodel,hardwarestartupsachieveagilityatbothmindset, strategy, andpractice levelinthe prototypingandproductdevel- opment phase. Hardware startups include developmentpractices duringthequality assurance phase that provideshort-term gains inquality.However, itbecomesclear thathardwarestartups lack both strategies andmindsets forachieving the long-termquality oftheproductduringtheprototypinganddevelopmentphases.

Themodelalsoillustratesthelackofpracticesduringthequal- ityassurancephasethatsupportthevitalneedforagilityinhard- warestartups.Inother words,therearenonequality-drivenactiv- itiesadoptedbyhardwarestartupsthatcontribute totheiragility.

Thisimpedestheadoption andfocusonqualityinhardwarestar- tups.

The commonality among hardware startups performing these approaches are (1) customized iterative practices, (2) suﬃcient competenceinteam,and(3)collaborativetechnicaldecisionmak- ing. These appear as key mindsets and strategies for startups to perform both agility and quality-driven product development. In the following sub-sections, we present detailed insights related to the common enabling factors, agility and quality aspects in prototyping,productdevelopment,andqualityassuranceactivities (Table4).

4.2.Enablingfactorsforachievingagilityandquality

Customizediterativepractices. Hardware andhardware-oriented product developmentinvolve a lotof experimental work, and so developersareencouragedtofollowaniterativedevelopmentap- proach(Ronkainen andAbrahamsson,2003). Among thestartups, fivepracticedsimplisticversionsofScrum,sevenusedad-hocAgile practices,whileone startupdidnot followadefinedAgiledevel- opmentprocess.Insomestartupstherewasnotidentified aneed toimplementspecificdevelopmentmethods,onereasonbeingthe smallsizeof thedevelopmentteam.Thiswasespecially thecase inearlystageswhentechteamswereco-locatedandintroduction offormal communication processeswould inhibitthe agility and freedomoftheteam.Inthestartupwherethedevelopmentteam onlyconsistedofoneperson,thedegreeofprocesswasalmostab- sent.

S5-“Sincetheteamissosmall,communicationiseasy.Wehave notseenaneedtoimplementany speciﬁcAgilemethodsorother leanpractices.”

S13-“Idon’tthinkAgilepracticesareapplicabletohardwarede- velopment,forexampleyoucannotfrequentlyre-designaportas itinvolvesgreatcosts.”

S8- “In hardware, thevariance oftasks andinterrelated depen- denciesmakeitmorecomplexthanwhatcurrentScrumtoolslike Jiraaresuitedfor.”

S4- “StrictScrumis probablyeasierto implement forpure soft- waredevelopment,soweuseasimpliﬁedversionofit.”

Due to different team sizes, product offerings, and other fi- nancial,managerial, and humanfactors, Agile practices were implemented differentlyamong thehardware startups.Sprintdura- tionusuallylastedbetween1–2weeks,andgoalsweredefinedin weekly meetings. Sincedevelopmentof physicalproducts usually takes longer time than implementation of software, the startups focusedondefiningmeasurablesub-goalsthatwerepartofalong- termplan.Moststartups hadthesame Sprintsforthe respective hardwareandsoftwaredevelopment.However, onestartupdiffer-

(8)

Fig. 4. An integrative view on quality and agility.

Table 4

Enabling factors for achieving agility and quality.

Terms Deﬁnitions Impacting factors Category

Customized iterative practices Self-deﬁned versions of Agile with Sprints where customers or potential users can give feedback. Tailored set of Agile practices (e.g., product backlogs, Sprint reviews) might be adopted

Team competence, team size, third-party vendors, market feedback

Strategy

Suﬃcient competence in team Acquiring in-house software and hardware engineers to perform design, implementation, and product testing

Funding, professional networks, recruitment strategy

Strategy

Collaborative technical decision making

Achieving harmony between hardware and software integration with a ﬂat-team structure that supports quick decisions regarding both implementation and testing

Team competence, leadership, coordination and

communication mechanisms

Mindset

Table 5

Achieving agility in hardware startups.

Partial laboratory-prototyping Production of simple and low-ﬁdelity prototypes, representing a part of the ﬁnal products and services, software and hardware prototyped separately

Complexity of hardware, funding, available tools, third-party vendors

Practice

Adoption of tools and components Utilizing commercial-off-the-shelf or open source components and tools to speed up the prototyping

Funding, component-driven experts in team, third-party vendors

Practice Optimizing manufacturing and logistic

process Managing third-party risks and maintaining ﬂexibility in

development process to achieve performance Team competence, communication

skills, risk management Strategy Combining documentation with Agile

methods

Spend less time on documents, make it as a task in the Sprint backlog

Expert availability Mindset

Accepting technical debt as an intrinsic attribute

Allow amount of technical debt that does not block product demonstration

Product nature, team competence, traceability of issues

Mindset Outsourced manual testing Outsourcing none important, manual testing tasks to

third-party vendors Communication skills, quality of

outsourced partners, task deﬁnition mechanisms

Strategy

(9)

Table 6

Quality aspects of hardware startups.

Quality-driven prototyping for core components Test-driven prototyping, early focus on non-functional attributes

Testing capabilities within team Practice

Towards more frequent user testing Early verify customer value before thorough testing

Communication skills, task deﬁnition Strategy

Partly automated testing Team members individually

test new functionality

Team competence, product nature Practice Simulation as an aid for unit and component testing Ability to predict product

behaviour before physical production

Available tools, product nature Practice

Regulation and standard to guide quality assurance Market regulations and standards may infer strict development guidelines

Quality of outsourced partners, available tools Practice

entiatedbetweenhardwareandsoftware Sprintstobetterhandle contingenciesofhardwareandsoftwaredevelopment.

S1-“Weworkona weeklybasiswhere wedeﬁnegoalsforeach week. Thesearepartofa maingoalofwhatwe wanttoachieve duringthesemester.”

S10-“Softwaredevelopmentfollowstwo-weekSprintswhilehard- wareSprintslast1–2months.”

Suﬃcient competence in team. Although contracting is a com- monapproach,startupsmentionthatinternaldevelopmentwould be the best wayto achieve agility.Hardware startups needteam membersthatarededicatedtoallaspectsofthedevelopmentpro- cess. As hardware startups have to deal with many factors be- sides software there are higher demands to expertise and experience of team members. Team members of hardware startups willpreferablyneedknowledgeaboutapplicationdomain,systematic development, softwareandhardwaredevelopment, mechanical engineering,andexperienceofworkingwiththird-party companies. Particularly, achieving a good collaborationbetweensoft- wareengineersandhardwareengineersintheteamwouldaccel- eratetheprocessofprototyping.However,thisisonlyobservedin onestartup.Mostofthestartupshadchallengesofachievingright setsofcompetencefromthebeginning:

S6 -“Finding talentedpeople is hard.Since we area startupwe cannotgiveverygoodsalary.Thisiswhy wetrytoattractpeople whoseethattheproductmayprovidegreatvalueinthefuture.”

Eventhough external resources can substitute forthemissing competence,thiswouldnotbe sustainableinthe longrun.Many startupsincludepart-timeteammembers,whoaretypicallymore task-basedorientedthanco-founders. Dependingonthesepeople mightreduce theagility ofproductionduetotheavailability and commitmentissues.

Collaborative technical decision making. Hardware startups are found with technology-driven processes of iterating their products. The teams are typically flat structured, probablydueto the factthat startupsoftenhaveasmallnumberofmembersatearly stages.Membersare motivatedandvoluntaryintakingtasks and responsibilities. In our study, startups seem to lack governance mechanisms oflegal rules andstrict regulation. Typically,technical decisionsare madebyengineersthemselves.All decisionsare madeonteam-basis.Wealsoobservethatstartupsallowforflexi- bilityinworkingtimeandplace,aseveryoneisresponsibleforthe requirements needed for their area of responsibility. For a small team, teammembers could probablyplay multiple roles.Overall, team members trust other’s competence.The team is flexible in organizingandreorganizing(i.e.,addingnewmembersandcollab- oratingwithvendors)toreacttochangesfromenvironments.

4.3.Agilityaspectsofhardwarestartups

Partial laboratory-prototyping. Almost all startups immediately builtaphysicalprototypetoelicit requirementsandachieverapid business experimentation. They usually followed an evolutionary approach,performing incrementalimprovementsonanearlylow- resolutionprototype.Rapidprototypingisimportanttoobtaincus- tomer feedback, however it can be problematic in the hardware context.Hardwarestartupsusuallyhaveasigniﬁcantfocusonnon- functionalrequirementsbecauseofthemanychallengesandregu- lationsassociatedwithcomplexsystemsdevelopmentandthegen- eralhardwareecosystem. Commonto theinvestigatedstartups is their priorityof modularity both atsoftware andhardwarelevel, muchsotoachievefrequentusertesting.

S10 - “We made a physical prototype immediately. It looks like today’sproduct,butwithmanyshortcutsandlowerquality.”

S8 -“We candevelop many low-resolutionprototypes using our ownequipment,butifwewanthigh-qualityprototypeswemight havetoorder10differentpartsfrom2to3suppliers.”

Todeal withtheir inability to quicklydevelop prototypes, the startupstriedtobeﬂexibleonthesoftwaresideoftheir products.

Sincesoftwarecanbefrequentlyupdatedandtestedbycustomers, they focusedon developinga simpleinterface betweenhardware andthesoftwaredirectlyaccessingthehardware.Inthiswaythey couldachievemoreparallelandindependentdevelopmentofhard- wareandsoftware.Theymainly tried toreusesoftware, ashardware components were easierto reuse withmore reﬁned prototypes.

S4-“Wehavedevelopedasimpleinterfacebetweenhardwareand softwaresothatthedevelopmentcanhappenindividually.”

S3-“Whenweoutsourcedsoftwaredevelopment,changestooka lotoftime...Insoftwareweneedtomakechangesweekly.Inhard- wareitisokaythatthingstakeabitmoretime.”

S2- “Weprefer makingchanges in thesoftwareor ﬁrmware.To facilitatethis,wehaveaclearlydeﬁnedinterfacebetweensoftware andhardware.”

Adoptionoftools andcomponents.Among theinvestigatedstar- tupstherewasamoreextensivereuseofsoftwarethanhardware.

Hardware andmechanicalcomponents were easierto reusewith morereﬁned prototypesthanearlylow-resolutionprototypes.The startupsmadelittleuseofmock-uptools,andsothrowawayproto- typesseemtotakelittlepartoftheprototypingstageofhardware startups.

S2-“Wetrytoreuseasmuchaspossiblefromeachprototype.We dividethe codeinto differentmodules,so thatif we replaceany

(10)

hardwarecomponentweonlyneedto changethatspeciﬁcpartof thecode.”

S7 - “We tried to reuse the electronics, but it was harder than expected. So the physicalcomponents are usuallysubstituted for eachprototype...Thesoftwareismoreeasilyreusable.”

Having accessto prototyping equipment willbe an important asset,reducingbothdevelopmenttimeandprototypingcost.With 3D-printers startups can do a lotof the prototyping themselves, andmakerapidchangesbasedoncustomerfeedback.Thisenables fasterproblemspacetesting.

S2-“Witha3D-printerwecanmakeproductsthatlookandfeel real. Thisis a hugeadvantage. Wecanliterally do almost every- thing apart from the electronics production ourselves and put it togetheralmostforfree.”

S9-“Wehavedonealotof3D-printing.Withoutaccesstouseful equipmentprototypingwouldhavebeenveryexpensiveandtaken moretime.”

Optimizing manufacturing and logistic process. The most time- consumingprocessofhardwareprototypingisthelongproduction andshipping times,as production usually is located in China or other countries in southeast Asia. This means that not only will thedeliverytime ofnecessaryparts dependonthevendor’sown schedule,butalsotheshippingmethodused.Severaloftheinves- tigatedstartupsspentasigniﬁcant amountofmoneyonspeeding upproductionandshippingtimeofmanufacturedcomponents.

S8 - “All parts of the prototypes must be ordered, mostly from China,withlongdeliverytimes. Wespendalotofmoneymaking deliverytimesshorter.”

Severalstartups experienced qualityissues workingwiththeir externalpartners.Manufacturingdefectsofcrucialprototypecom- ponentscausedextradelays,whichiscriticalconsideringthevalu- abletime alreadyspent waitingfor thecomponents. Cooperating withprofessionalactorscandecreasetheriskofqualityissues,and enhancecommunication.

S4-“Wehaveoutsourcedproductionofmechanicalpartsandcir- cuit boards. Some of thecomponents we havereceived from the manufacturerhavebeeninbadconditionandwithsigniﬁcantde- fects.”

As high-tech prototyping is a time demanding process, there mightgoseveralyearsfromthe startupisfoundedtothe timea ﬁnalizedproduct isready to be releasedto the market.This im- pliesthat vendors’dependability also is of importance. Choosing componentsthat withcertaintywillbeavailable theentireproto- typingstageiscrucial.

S12-“Theﬁrstversionofthescreenwentoutofproduction.This wasthemostimportantcomponentandittookalotoftimetoﬁx theproblem.”

To achieve speed, product quality often gets low priority in startups. However, because of the vendor dependency of hardware startups, hardware development should receive higher focuson quality.Making shortcuts inhardwaredesign,andnot as- suringthat the design is of sufficient quality before sending the specificationsforproductionmight be costly. Initialfindings sug- gestthat hardwarestartupsfocusonensuring thequality ofcore hardwarecomponents,aslow-cost solutions mayinhibitprogress inthelong-run.FindingsfromS12andS1indicatethat hardware startupsshould putgreat effortintoensuringthequalityofhard- warecomponents,aslow-cost solutionscaninhibitthelong-term evolutionoftheirprototypes.

S1-“Wespent morethan$500ona singlecomponentwecould not use. Inaddition we hadto spend more timeredesigningthe board,andwaitforittobeproduced.”

Because ofpressured ﬁnancial resources andsmallproduction batches it can be hard for startupsto receive commitmentfrom professionalmanufacturers.Workingwithvendorsproducingcom- ponentsofhighqualityatanaffordablecostwillbeanadvantage.

Thebiggeographical distance,andthedifference inlanguageand culturemayalso challengethe communicationskillsoftheteam, aseffectivecommunicationisimportanttoreceiveserviceaspaid forandmaintainproductdevelopmentspeed.

S10-“Aswehavegrown,wehavebeenableto workwithbetter suppliersproducingathigherquality,whichinturnhashelpedus prototypefaster.”

S2 - “We are building on networks from earlier startup experi- ence... Previously, we chose the cheapest suppliers, but then we alsogotcomponentsin badcondition, therewerecommunication problems,anditusuallytookmorethan4weeksto gettheprod- ucts.”

Combining documentation with Agile methods. On the software sideoftheproduct,thecommonperceptionisthatsincethedevel- opers work on thecode-baseevery day, documentationactivities lead to additional overhead.Tacit knowledge seem to be a com- monpracticeinhardwarestartups.

S3 - “Wespend less timeon documentation to speed things up, development isourmain focus.Itisalsobecausesoftwaredevel- opmentisin-house.Weworkonitdailyandunderstandthecode.”

High-techproducts includea lot of differentsub-systems and technologies, and so product complexity increases fast. This im- plies that documentation of components should receive a bigger attentioninhardwarestartups. Inworst case, lackof qualityand documentationcanputalldevelopmentonhold.

S2 - “Instead of updating documentation and quality, we did things as fast as possible, which eventuallyled to a lotof extra work.”

Theprototypingstageinhardwaredevelopmentisoftensignif- icantly longer than that of software development. Since it might take years before hardware startups have a functioning product ready for the market, there’s a great probability of people quit- ting the project before it is ﬁnished. As to this there should be increasedfocus ondocumentation inhardwarestartups, since knowledgeoftenaccompaniesthepersonquitting.

S4-“Sometimesitbecomeschallengingtokeeptheknowledgeof people who quit, the knowledge often accompanies that person.

Thisleadstoextracostsandeffort.”

The choice of outsourcing companies can greatly impact the amount of documentation. Good partners usually provide well- documented solutions and components, which can help manage technicaldebt.

S3 -“We receivedan 80-page usermanual fromthe consultants whodevelopedthehardware.”

To help startups perform documentation, there exist multiple toolsloweringthebarriersforwritingdocumentation.Examplesof toolsincludeWikis,GoogleSpreadsheets,andConﬂuence.Utilizing toolscanhelpdecreasetheamountofreworkinthelongrun.Also thoroughdocumentationcanallowformoreeﬃcientintegrationof newemployeesinthedevelopmentprocess.

S2-“Previouslywehavespentalotofextratimeduetoalackof documentation. Insteadofstopping,we didthingsas fastaspos-

(11)

siblewithoutperformingdocumentation.Thiseventuallyleadtoa lotofextrawork.”

S4 -“Wehavea wikiforinternaldocumentation. Itis quitelow efforttowritesomething.”

Accepting technicaldebt as an intrinsic attribute.Technical debt has been illustrated by Brown et al. (2010), stating that “developers sometimes accept compromisesin asystem inone dimension (e.g., modularity)to meet an urgent demand insome other dimension (e.g.,a deadline), andthat such compromises incur a

“debt” on which“interest” hasto be paidandwhich the“princi- pal” should be repaid atsome point forthe long-term healthof the project”. Finding the correct balance between learning goals andqualityisthereforeimportantinordertominimizewasteand tomanagetechnicaldebt(Terhoetal.,2016).

Byacceptingthattimetomarketismoreimportantthanprod- uct quality, hardware startups incur intentional technical debt.

Business experimentation to build new features is performed in small iterative cycles with minimal effort on product quality to receive fast customer feedback. Corresponding to software star- tupstechnicaldebtmainlymanifestsitselfonthesoftwaresideof the productin hardwarestartups.Sincesoftware canbe changed quickly, shortcuts andworkaroundsare moreeasily taken onthe software side thanon the hardwareside ofthe product.The development team prioritizesimplementing new functionality over improvingthequalityofthecodebase.

S2-“Softwarechangesallthetime...Tomakethingsworkstraight away,we’drathertake ashortcutandﬁxitlater.Weknowwe’re building uptechnical debt,but it’son purpose to be ableto test theproductoncustomersasquicklyaspossible.”

Hardware startups do not accumulate technicaldebt for their hardwarecomponentssimilarlyasfortheirsoftwarecomponents.

As the concept of technical debt is built on erosion of systems fromfrequentlow-qualitychanges,thisisnotaseasilymanifested in hardwarecomponents. Refactoringdelivered or released hard- wareisadiﬃcultandrarelyperformedendeavour.However, poor hardwaredesignmighteventuallyleadtothehardwareneedingto beredesigned.Ashardwarecomponentsoftenarereusedbetween low-resolution prototypes,baddesign mightimplythat thehardware needs redesign on an earlier iteration of the product than intended.Earlylifetimedesigndecisionsmightpropagatethrough- out the lifetimeofthe product,andmay eventually becomepart oftheﬁnalproduct.Thesepoorlymadedesigndecisionswillthen bediscoveredaftertheproductisreleased.Hence,temporarylow- quality solutions in both hardware and software will eventually leadtoaccumulationoftechnicaldebtinhardwarestartups

Outsourcedmanual testing. Outsourcingincludes thechoicesof both local consultant companies andaboard contractingvendors.

Hardwaredevelopmentrequiresasigniﬁcantamountoftestingto ensureproductquality.Thisappliesalreadyattheprototypestage, and for demonstration. Among the companies, some outsourced their manualtesting(i.e.,testingtheﬁnalreleaseatdifferentexe- cution environments, and testing the integration between devel- oped components and known services or products). Outsourcing manual testingcan save time andeffortfor startups tofocus on innovationandcorevaluecreationactivities.

S10 - “In softwarewe havea greatfocus on testing.When soft- wareismodiﬁed,werunautomaticteststoensurethateverything works...Inhardwarewetestthattheproductfunctionsindifferent climates,andperformvariousmechanicaltests...Wehavealsoout- sourcedmuchmanualtestingtoacompanytocheckmorepartsof theproduct.”

4.4.Qualityaspectofhardwarestartups

Quality-driven prototyping for core components. Testing is cen- tralto hardware startups.High quality in hardwaredevelopment is important both because of the cost associated with mistakes fromproduction,butalso asquality greatly affects theperceived functionalityoftheproduct.Incontrasttosoftwareproducts,itis challengingtoimplementchangesandmakeimprovementstothe qualityafterthe producthasbeenproduced andassembled.Asa consequence,focusonnon-functionalattributesattheprototyping stageisessential.Weobservedmanystartupsthatimplementeda test-drivenapproach fordevelopingthecorecomponentsoftheir prototypes.

S4-“Wehaveatestsetuptoensurethatthesubsystemswork asintended,andthatallowsustoanalyzedifferentmetricsand data.Forthemostcriticalcomponentsandfeaturesweusually deﬁnedetailedtestplansinadvanceofdevelopment.”

Towards more frequent user testing. To achieve quick development speed in early stages, low-level testing activities generally receivelittlefocus inhardwarestartups.Beforea featureis guar- anteedtobepartoftheﬁnalproduct,itismoreimportanttover- ify that the feature adds value to the customers.Until then, the timespent on testingactivitiesis minimized. Thisisalsoevident insoftwarestartups,wheredevelopersavoidwastingtimeonim- provementsofnot-validatedfunctionalities(Giardinoetal.,2016).

S2-“Wepreferto workfast,aswritingtestscandouble thede- velopmenttime...Ifpartsaretobereplaced,thenwethinkthere’s nopointinspendingtimeontesting.”

InS3andS6,theCEOhighlightedtheimportanceofhavingfre- quentfeedbackfromtheir customersandusersontheprototypes.

Thiswouldbecriticalforthedesignanddevelopmentofaproduct thatlaterissoughttoamassmarket.

Severalstartupsfaced thechallengeoftestingtheir productin realistic environments becauseoflegal restrictionsrelated topri- vacyandpublicsafety.Simulationsanddummy-datacanbealter- nativestoearlytesting.

S4-“Settingupafoundationfordoingrobusttestsisachallenge.

When developingdrones itis not easy to perform testing, it re- quiresspeciﬁcexperienceandknowledge.”

Lackofﬁnancialresourcesandlongdeliverytimesmakeitchal- lengingto test the product on a broader spectrum ofcustomers.

Physicalprototypesare resource-intensivetodevelop,andincon- trast to pure software products, one cannot necessarily delivera new digital software update to customers. The investigated star- tupsreliedonasmallsetofcustomersforfrequentfeedback.

Partlyautomated testing.The hardwarestartups reliedoneach individual developer to test features as they were implemented.

Inthatwaythepersonresponsible forthecode wasalsorespon- siblefor its quality and functioningwith the rest ofthe system.

Afrequentlyusedtestingactivityamongthestartupswasmanual smoketests(i.e.,ensuringthatmajorfunctionalityworkbeforeun- dertakingmoreformal testingprocedures).Prototypeswereman- uallytestedbyinternalemployeestoidentifythemostprominent defectsbeforetestingprototypeswithearlyadoptersorcustomers.

S8-“Wetest thesubsystemsourselves,butdo nothavea struc- turedsystemfortesting...Thepersonresponsiblefordeliveryisalso responsiblefortestingthefeaturetomakesureitworks.”

S1-“People insidethestartupwhohaveexperiencewithsimilar solutionstesttheproductbeforeitistestedwithpilotcustomers.”

Softwareengineerstendstooptimizetheintegrationandopera- tionofsoftwarecomponentsbyadoptingautomatedtesting.Thisis