morphology of the foot on a one-legged pneumatic robot
Using evolutionary algorithms for online-optimization of gait speed and
morphology
Claes-Jonas Gill
Thesis submitted for the degree of Master in Robotics and Intelligent systems
60 credits
Department of Informatics
Faculty of mathematics and natural sciences
UNIVERSITY OF OSLO
morphology of the foot on a one-legged pneumatic robot
Using evolutionary algorithms for online-optimization of gait speed and
morphology
Claes-Jonas Gill
Optimizing gait speed and morphology of the foot on a one-legged pneumatic robot
http://www.duo.uio.no/
Printed: Reprosentralen, University of Oslo
For humans, walking and adapting in various terrains and environments is a natural task. However, strange as it sounds, teaching a robot to take its first steps offers great challenges. Robot adaptation and control is an endless challenge that a substantial amount of researchers strive to solve.
The evolutionary robotic research field aims to solve such problems using natural evolution as inspiration.
This thesis explores a one-legged pneumatic jumping robot’s ability to au- tomatically adapt to the environment using an evolutionary algorithm. The purpose of this study is to discover whether the robot can automatically adapt to the real world environment by finding optimal gait and morphol- ogy.
In order to discover whether a robot can adapt to the real world environment, a testing environment and two 3D printed robots were made. The first robot laid the groundwork for the second robot, and was designed to only optimize its gait, in which it successfully achieved using an evolutionary algorithm. The second robot was partly able to achieve its design purposes in terms of optimizing gait and morphology.
Several experiments were carried out on the two robots. An early finding was that it is possible to use the evolutionary algorithm to optimize the first robot’s gait. To make the second robot adapt to the natural environment, however, was a more complicated matter as it encountered various challenges leaving the investigation imperfect. On the other hand, the evolutionary algorithm itself performed quite well and the robots was ultimately fit to handle the major stress inflicted by the explosive pneumatic system.
I would like to express my deepest gratitude to my thesis advisor, Associate Professor Mats Høvin, for continuous guidance and invaluable input throughout the thesis. He allowed the paper to be my own work but steered me in the right directions when it was needed. I would also like to thank, Head Engineer Vegard Søyseth, for helping with equipment and tools. My grateful thanks are also extended to Lecturer Yngve Hafting for productive discussions, suggestions, and advice.
I would also like to thank my fellow students for making the University of Oslo, Department of Informatics a great environment both academically and socially.
Finally, my special thanks are extended to my family and my better half, Camilla Hellum, for productive feedback and their endless support throughout the thesis.
1 Introduction 1
1.1 Motivation . . . 1
1.2 Goal of the thesis . . . 2
1.2.1 Real world challenges . . . 3
1.3 Implementation . . . 4
1.4 Outline of the thesis . . . 5
2 Theoretical background 7 2.1 Human or robot? . . . 7
2.2 Physical robot . . . 8
2.2.1 Robot types . . . 8
2.2.2 Robot foot and ankle . . . 8
2.2.3 Robot activation . . . 9
2.3 Evolutionary computing . . . 10
2.3.1 Biology inspiration . . . 11
2.3.2 Why use evolutionary computing? . . . 12
2.4 Evolutionary algorithms . . . 12
2.4.1 Evolutionary components . . . 13
2.4.2 Genetic algorithm . . . 16
2.4.3 Evolutionary strategies . . . 17
2.4.4 Evolutionary programming . . . 18
2.4.5 Genetic programming . . . 18
2.4.6 Optimization and search . . . 19
2.4.7 Exploration and exploitation . . . 20
2.4.8 Evaluating performance of EA . . . 20
2.5 Evolutionary robotics . . . 22
2.5.1 Offline and online evolution of robots . . . 22
2.5.2 Problems in ER . . . 23
3 Tools and engineering process 25 3.1 Design tools . . . 26
3.1.1 Fusion 360 . . . 26
3.2 3D Printing . . . 27
3.2.1 Model preparation . . . 27
3.2.2 Printing process . . . 28
3.3 Electronics and mechanics . . . 29
3.3.1 Microcontroller . . . 30
3.4 Testing hub . . . 32
3.5 Programming and software . . . 34
3.5.1 Python . . . 34
3.5.2 Arduino IDE . . . 34
4 Robot implementation 37 4.1 Hoppfully - The first robot . . . 37
4.1.1 Design and simulation . . . 39
4.1.2 Testing and assembly . . . 41
4.2 RoboHopp - The second robot . . . 42
4.2.1 Design and simulation . . . 43
4.2.2 Testing and assembly . . . 45
5 Implementation and experimental setup 47 5.1 Experimental setup . . . 47
5.1.1 Controlling the robot . . . 48
5.1.2 Fitness score . . . 49
5.2 Reducing noise . . . 50
5.2.1 Increasing friction . . . 50
5.2.2 Gait pattern . . . 52
5.3 Evolutionary algorithm . . . 52
6 Experiments and results: HoppFully - the first robot 57 6.1 Experiments and tests . . . 57
6.1.1 Test 1 . . . 58
6.1.2 Test 2 . . . 60
6.1.3 Comparing results . . . 61
6.2 Summary . . . 62
7 Experiments and results: RoboHopp - the second robot 63 7.1 Experiments and tests . . . 64
7.1.1 Test 1: Optimizing morphology . . . 64
7.1.2 Test 2: Optimizing interval parameters . . . 66
7.2 Summary . . . 69
8 Issues and investigation 71 8.1 Addressing issues . . . 71
8.2 Investigating noise . . . 73
8.2.1 Test 1: Adjusting the step size . . . 74
8.2.2 Test 2: Excluding steps . . . 75
8.2.3 Test 3: Head start . . . 76
8.2.4 Discoveries from tests . . . 77
8.3 Summary . . . 78
9 Discussion 79 9.1 General discussion . . . 79
9.1.1 Experimental setup . . . 79
9.1.4 Investigation . . . 82 9.2 Conclusion . . . 82 9.3 Future work . . . 83
Bibliography 85
A Previous work 93
A.1 Previous work according to the thesis . . . 93
B Code 99
B.1 Python code for optimization search and data acquisition . . 99 B.2 Arduino code to run the population on the robot . . . 108
C Schemes 111
C.1 Arduino, encoder and valve . . . 111 C.2 Testing hub scheme . . . 113
2.1 Examples of different ankle joints.(a)Robot with ankle joint with one degree of freedom. Image taken from [73]. (b) Bipedal robot using a servo motor to achieve one degree of freedom. Image taken from [26]. (c) Robot foot able to achieve pronation and flexion of the foot. Image taken from
[33]. . . 9
2.2 Overview of a chromosome containing N genes. . . 12
2.3 A flowchart of an general evolutionary algorithm. . . 13
2.4 Illustration of a population containing N individuals. . . 14
2.5 Hill climbing algorithm concept illustration. The points are randomly placed and use the greedy approach to reach the optima. Here, point A is going towards a local optima while B to the global optima, which shows its vulnerability to noise and other uncertainties in data. . . 20
2.6 Explanation of a box plot displaying whiskers and outliers. . 22
2.7 Workflow of robot design distinguishing offline and online evolution, where they are separated by the moment of deployment. . . 23
3.1 The figure shows two renders and one image of the part used for attaching the robot to the test hub. . . 29
3.2 The figure shows a rendering and an image of the part used for attaching the robot to the test hub. . . 30
3.3 Image of the Arduino Mega 2560. . . 30
3.4 Dynamixel with associated adapters. (a) is a image of the MX-64 servo, (b) the USB2Dynamixel adapter for serial connection between the servo and computer, while (c) is the SMPS2Dynamixel adapter for external power supply. . . 31
3.5 Image of the pneumatic cylinder showing both extended (a) and the retracted (b) states. . . 32
3.6 Image of the solenoid valve. . . 32
3.7 Image of the in-house testing hub. Attaching the robot to the 1 meter long balancing rod makes it able to walk in two dimensions counter clockwise around the hub. . . 33
4.1 Concept of the structure of Hoppfully. Containing a pneumatic cylinder to create movement. . . 38
lift-off and landing it are fully retracted. . . 38 4.3 The CAD model and the 3D printed femur. (a) is a rendered
image from Fusion 360 and (b) a photo of the femur 3D printed. 39 4.4 The CAD model and the 3D printed foot. (a) is a render from
Fusion 360 and (b) is an photo of the foot 3D printed. . . 39 4.5 The foot connected to the ankle joint of the tibia. Sequence of
the limited ankle movement going from extension to flexion. 40 4.6 The CAD model and the 3D printed tibia. (a) is a render from
Fusion 360 and (b) a photo of the 3D print. . . 40 4.7 FEM simulation result for the tibia. . . 41 4.8 Image of Hoppfully completely assembled with reinforce-
ment plates. Lock nuts are used to attach all parts. . . 42 4.9 The concept of the structure of RoboHopp. Containing a
pneumatic cylinder for movement and a servo for morphol- ogy adjustments. . . 43 4.10 The various angles the robot foot can be in. . . 43 4.11 Image (a) and (b) are both rendered in Fusion 360, where (a)
show the gearing system and (b) the reserve solution using a rigid connection between the servo and foot. The tibia is transparent to better show the system while the mount bracket are displayed in black. . . 44 4.12 Image of the 3D printed tibia from its left side. Printed on
the Mark X7 3D printer. . . 45 4.13 Images from the motion link simulation in Fusion 360, where
the tibia is transparent to observe the full movement. . . 45 4.14 FEM simulation result for the tibia. . . 46 4.15 Image of RoboHopp complete assembled using the MX-64
servo motor and gearing solution. . . 46 5.1 A overview of the experimental setup. Note that the stapled
line are the connection between the computer and servo, and are not in use on the first robot. . . 47 5.2 The process of taking one step. The cylinder can either be
high or low while the pause lengths are given in milliseconds. 48 5.3 The process of changing the foot angle before the step intervals. 49 5.4 Illustration of the robot taking three steps in the forward
direction. Here, the encoder measures the positions, p, and the distancesare the length of each step. The start time and end time are described ast0andt1. . . 50 5.5 Image of the moulded silicone sole on Hoppfully. . . 51 5.6 Images of the sandpaper attached to the foot in three
different views. . . 52 5.7 Illustration of the robot three step gait pattern. Walking
direction is from left to right, counter clockwise on the testing hub rod system. . . 52 5.8 Flow chart of the evolutionary algorithm. . . 53
6.2 Graph of the average fitness from all three runs in test1. . . . 59 6.3 Graph of the average results from all runs in test2. . . 61 6.4 Comparison of both experiments. . . 62 7.1 Image of RoboHopp during a run in test2. . . 63 7.2 Minimum (−40°) and maximum (40°) angles of the robot
foot using gears. Minimum are shown to the left, flat in the middle and maximum to the left. . . 64 7.3 Graph showing the results from RoboHopp, optimizing its
morphology. . . 66 7.4 The nine morphologies of the robot foot going from−40° to
40° using the solid part instead of gears. . . 67 7.5 Results of all the runs showing the mean of the measure-
ments, where the y-axis represents the measured speed in m/s and x-axis the number of generations. Organized in the same order as the foot angles in figure 7.4. . . 69 8.1 Image of the foot using neoprene in between the foot and
sandpaper before the last flap is secured. . . 73 8.2 Box plot of the results from test1. Median are the orange line
in the box, while outliers are displayed with red dots. . . 75 8.3 Box plot showing the results from the three different meth-
ods in test2. Number 1 shows the test excluding the start step, 2 the end step and 3 excluding both the start and end step. Median is the orange line in the box, while outliers are displayed with red dots. . . 76 8.4 Box plot showing the results from test3, where the robot
are given a head start to build up speed for a more precise measurement. Median are the orange line in the box, while outliers are displayed with red dots. . . 77 8.5 Box plot of all the previous experiments. Results from test1
(1.1, 1.2, 1.3) are shown in violet, test2 (2.1, 2.2, 2.3) are shown in orange and test3 (3.1, 3.2) in blue. . . 78
2.1 The corresponding terminology of natural evolution and
problem solving. . . 11
2.2 Sketch of the simple genetic algorithm. . . 17
2.3 Sketch of evolutionary strategies. . . 17
2.4 Sketch of evolutionary programming. . . 18
2.5 Sketch of genetic programming. . . 19
3.1 A list of the different hardware and equipment used in the thesis. . . 25
3.2 A list of the different software tools used in the thesis. . . 26
4.1 The robot design and system requirements. . . 39
4.2 Measurements and weight of Hoppfully. Note that the simulated weight is much lower due to cylinder, screws and washers are not included in the weight measure. . . 42
4.3 Measurements and weight of RoboHopp. Note that the simulated weight is much lower due to cylinder, screws and washers are not included in the weight measure. . . 46
5.1 The table shows the approximated average friction coeffi- cients from the tested materials. Each tested three times on the different surfaces. . . 51
6.1 The table showing the parameters used in test 1. Pause parameters and mutation step size are given in milliseconds. 58 6.2 Showing the average results from the three tests. The average result from the tests are separated with a double line. All results regarding speed are in m/s, time are in hours and the interval pairs are in milliseconds. Best represent the best interval from all runs based on the maximum speed. . . 59
6.3 The table showing the parameters used in test 1. Pause parameters and mutation step size are given in milliseconds. 60 6.4 Showing the average results from the three tests. The average result from the tests are separated with a double line. All results regarding speed are in m/s, time are in hours and the interval pairs are in milliseconds. Best represent the best interval from all runs based on the maximum speed. . . 60
the best of all three runs from each experiment and are given in milliseconds. . . 61 7.1 The different mutation step size parameters tested. Param-
eters that have a interval such as the one in number 2 are random picked from 20 to 30 with a step of 1. . . 65 7.2 The table showing the parameters used in optimizing the
foot angles of RoboHopp. The angles and mutation param- eter are given in units, and the pause lengths are given in milliseconds. . . 65 7.3 Results from one run of the algorithm on RoboHopp. All
results regarding speed are in m/s, while time are given i hours. Best angle are found based on the maximum speed. . 65 7.4 The table showing the parameters used in optimizing the
foot angles of RoboHopp. . . 67 7.5 Results from all test using different static angles on the robot.
All speed measurements are given in m/s. Best interval are found based on the maximum speed and given in milliseconds. 68 8.1 Results and percent deviation of the first noise test. Using
a fixed foot angle of 0° and pause length of 110 and 100 milliseconds. . . 74 8.2 Results and percent deviation of the different measurement
points where the measurement excluded the first step, the last step and first and last step. . . 76 8.3 Results and percent deviation from a noise test where
measurements was done continuous. . . 77
2D Two dimensional 3D Three dimensional
AES Average number of evaluations to a solution CAD Computer-aided design
CFF Continuous filament fabrication COF Coefficient of friction
EA Evolutionary algorithms EC Evolutinary computing ER Evolutionary robotics EP Evolutionary programming ES Evolutionary strategies FDM Fused deposition modeling FEA Finite element analysis FEM Finite element model FFF Fused filament fabrication FOS Factor of safety
GA Genetic algorithm GP Genetic programming HC Hill climbing
MBF Mean best fitness SA Simulated annealing SGA Simple genetic algorithm SI International System of Units SR Success rate
Introduction
1.1 Motivation
Robots have become a big part of our everyday life. They are used in hospitals, for package delivery, in our homes and serve as companions in various software services. Even humans can have a robotic touch to them when using prosthesis replacing missing limbs, or exoskeletons to enhance mobility. However, one does not often see robots walking in the streets or going out on a hike.
For humans, walking and adapting in various terrains and environments is a natural task. However, strange as it sounds, teaching a robot to take its first steps offers great challenges in the development of real robots.
Challenges start in the interdisciplinary field of real robotics, as it requires knowledge in mechanics, electronics, informatics, and computer science.
Combined they are used in designing, constructing and controlling a robot.
Robot adaptation and control are endless challenges that a substantial amount of researchers strive to solve. The research field evolutionary robotics is inspired by the natural evolution in nature, and intends to solve tasks such as robot adaption and control.
The field of evolutionary computing, and more specifically the subfield of evolutionary robotics (ER) is slowly gaining more popularity. The study in [14], shows growth in publications in evolutionary robotic research over the past 20 years. The field aims to evolve controllers for real and simulated autonomous robots [71], using evolutionary algorithms. Most of the ER research over the years and today have been experiments using simplified physics simulators [49]. A common problem that follows the use of physics simulators is the leap in performance between simulation and the real world, referred to as the reality gap. One solution for closing the reality gap is done by switching from simulation to evolve online on the hardware of the system. A system like this would yield more realistic results and display the same noise and unpredictability as in the physical environment [19]. In such systems lies further investigations such as co-evolution of
controller and morphology.
Among others, there are examples of evolution of controller and morphol- ogy experiments in both simulation [63] and servo-driven robots perform- ing in the real world [54, 55]. However, a rough estimate done by Nichele [52] indicates that more than 95% of evolutionary robotics research is evo- lution of controllers, less than 4% evolution of morphology in simulations, while the remaining research involves evolving both morphology and con- troller. While these estimates are probably not accurate, this along with the previously mentioned reasons motivates for further research into the exciting field of ER. Where the absence of pneumatic robots is a motiva- tion on its own. The motivational goal of the thesis is to contribute to the field running evolution of controller and morphology on a real pneumatic robot performing in a real world environment. The purpose is to investi- gate if automatically change in morphology, and optimizing gait can help increase the robot’s performance and adaption to the environments. Be- sides, an ambition is addressing challenges that target real robotics in a way that can enlighten the field of ER.
1.2 Goal of the thesis
The ultimate goal in the thesis is to investigate if the robot can adapt to its environment using an evolutionary algorithm for optimizing morphology and gait parameters. The performance determines how well the robot adapts, and is measured by the robot’s forward walking speed of a gait pattern, where the latter contain one or multiple steps. The hypothesis claims that the evolutionary algorithm is capable of finding the best optimized morphology and gait parameters that enable the robot to adapt to the environment. To further investigate the hypothesis, the ultimate goal is divided into two goals.
The first goal involves creating a one-legged pneumatic robot with the ability to jump continuously in a given gait pattern. The goal is to optimize the parameters within the gait pattern using an evolutionary algorithm to find the robot’s best performance. As this goal lays the groundwork for the second goal, it is important that the implemented robot and evolutionary algorithm perform satisfactorily. Summarizing the first goal:
1. Design and implement a one-legged pneumatic jumping robot for optimizing parameters in the gait pattern to find the best performance.
The second goal involves creating a second one-legged pneumatic robot.
It is required to function identical to the first robot and also capable of automatically change the morphology of its ankle. The morphology can be controlled by either a soft robotic sole or a servo motor. The goal is to optimize the parameters in the gait pattern and morphology using the pre-developed evolutionary algorithm from the first goal. Importantly, the
second robot needs to perform satisfactory and credible to be able to prove the hypothesis. Summarizing the second goal:
2. Design and implement a second one-legged pneumatic jumping robot for optimizing parameters in the gait pattern and automatically adjust morphology for testing the hypothesis.
Accomplishing these goals will achieve the ultimate goal of the thesis and be able to prove or disprove the hypothesis.
1.2.1 Real world challenges
Running robots in the real world one have to consider the time it takes to perform an action in the environment. Having a large set of parameters increases the search space, and can be a very time-consuming process.
It can be avoided by limiting the number of parameters or running the optimization search on one or more parameters at the time. In an attempt to reduce the search space there are proposed four methods of running the optimization search of the evolutionary algorithm. If time left in the thesis the fifth method is suggested for comparing run time and results. As the first robot only need to optimize the parameters of its gait, the proposed methods only apply to the second robot. Using the following methods would additionally determine the robot’s performance and ability to adapt to its environment:
1. Optimize the morphology parameters using fixed, good performing parameters in the gait pattern.
2. Optimize the parameter of the gait pattern using several fixed morphological parameters and compare performance.
3. Divide the optimization search into two steps. First optimizing the parameters of the gait pattern, and secondly use the best parameters of the gait pattern for further optimize the morphology parameter.
4. Attempt the third proposal in reverse order. Optimizing morphology parameter first, and then optimize the parameters of the gait pattern.
5. Search for all parameters simultaneously.
In addition to being time-consuming, there are multiple challenges to encounter when running robots in the real world. One is associated with online evolutionary adaptation, where one needs to consider the various elements that build the robot. In which can cause both loosenesses in between joins and broken parts. A second is friction. It has many effects on the testing environment and can have a major impact on the results. Having any uncertainties can infect important measurements in the evolutionary optimization search and contaminate the result it appears as noise in data and is a common problem with running robots in the real world. The paper in [23] points out side effects in the physical environment such as highly unpredictable friction due to vibrations,
varying pneumatic air pressure and parts that wear out. Notably, noise sources are found in sensors, actuators and external stress from the real world environment.
Simulations are known to be efficient, but inaccurate, as simulated sensors and actuators are ideal with the possibilities to add noise to get a better approach to reality. However, it can be a challenge when simulating a pneumatic robot, as it is explosive and can cause high unpredictability in the physical environment. The struggles of accurately simulate the real world are a well known problem in robotics [42] and are hence not used in this thesis. Instead, there are proposed some approaches that can help reducing noise during the evolutionary optimization search:
• Design the robots robust and durable to prevent them from breaking during experiments.
• Eliminate looseness between joints using lock nuts.
• Increase friction between the robot foot and the floor to create traction to maintain good grip when walking.
• Using a gait pattern that contains multiple steps to provide more consistent speed measurements.
1.3 Implementation
The following was done to meet the goal of the thesis:
• Building two one-legged robots:
– Hoppfully, the first robot, made for optimizing only the param- eters of the gait.
– RoboHopp, the second robot, containing the ability to optimize parameters for both gait and morphology.
• An testing environment was constructed to experiment with the robots and testing the hypothesis.
• Multiple experiments1 were done on each robot using an evolution- ary algorithm for optimizing parameters.
• Investigate and resolve the issues that contaminated data and prevented further experiments.
1A rough estimate of the robots total run time was 1 week and 3 days, based on the amount of generated log-files.
1.4 Outline of the thesis
The thesis is divided into nine chapters: introduction, theoretical back- ground, tools and engineer process, robot implementation, implementa- tion and experimental setup, experiments and results on the first robot, experiments and results on the second robot, issues and investigation, and discussion.
Chapter 2: Theoretical background Attempts to give an introduction to the theory relevant to the thesis.
Chapter 3: Tools and engineering process Containing an overview of the tools used for creating and making the robots along with other practical work.
Chapter 4: Robot Implementation Describes the implementation and design process of both robots.
Chapter 5: Implementation and experimental setup Describes the implementation of the system and the experimental setup.
Chapter 6: Experiments and results: HoppFully - the first robot Explains the experiments and the related results on the first robot, Hoppfully.
Chapter 7: Experiments and results: RoboHopp - the second robot Explains the experiments and the related results on the second robot, RoboHopp.
Chapter 8: Issues and investigation The investigation attempts to inves- tigate various issues occurred during experiments on RoboHopp.
Chapter 9: Discussion Contain a general discussion of the results and closing out with a conclusion of the work in the thesis and possible future work.
Theoretical background
This chapter attempts to give an overview of the theoretical background relevant to this thesis. It includes an overview of early work relevant to the thesis and the real robot, followed by an introduction to the field of evolutionary computing.
2.1 Human or robot?
Early stages in the thesis, explored the possibilities of solving it with regard to human or robot. A paper, written in the subject INF5207 – Advanced Specialization in Design of Information Systems, dealt with early work associated with the thesis (appendix A.1). The paper describes the creation of a soft robotics sole strong enough to lift a human. The sole is made out of two silicon molded parts with inflatable chambers. The chambers are air- inflated using pneumatic activators to create a lifting motion in the front and back of the sole. In the experiments, the paper investigates the sole’s ability to lift a test-subject in both flexion and extension states. The sole was able to lift the subject 1 cm in the flexion state and 10 cm in the extension state. Conclusively the paper suggests implementing various sensors and improvements as future work. However, in later experiments and research, the sole displayed major issues in durability and robustness. It was also experimented with an elastic sensor that performed poorly when used in the silicone sole. As the ultimate goal of creating a sole was to be able to adapt in terrain to increase human performance, the previous problems contradicted its purpose. Since the sole already presented problems in such early stages of the thesis, the proposal got compromised. The decision is based on predictions that more problems would arise and create an overcomplicated task unable to be accomplished in the limited time given in a master thesis. The compromise was instead creating a solution for a robot containing the ability to adapt to its environment thus the ultimate goal of the thesis.
2.2 Physical robot
When optimizing a robot in the real world, there is an essential factor that should be taken into consideration regarding time and search space, being the complexity of the robot. A very complex robot can provide a huge search space, and could take a great amount of time optimizing in an evolutionary search. For this reason, robots are typically run in a simulated environment to reduce cost in terms of time. Although several simulation tools exists [78], they will, perhaps newer, be able to behave as if in the real world. As there are no robot simulations in this thesis, the complexity of the robot should be reduced considering both testing time and search space. Further suggestions are made along the way in the upcoming sections.
2.2.1 Robot types
Robots exist in various shapes and forms depending on their usage. Some intend to replicate humans, while others draw inspiration from nature and animals. Humanoid robots are mostly bipedal, meaning they have two legs meant for doing typical human tasks [37]. Other bipedal robots are navigating through rough terrain [27], while others explore more advanced topics such as human spaceflight [59]. Robots inspired by animals are often multi-legged since most animals are. Some inspired by four-legged creatures such as the cheetah created for fast trotting [64]. Others is inspired by six-legged robots to recreate a spider used for motion planning [10] or self-adaptation [15].
Another type is the one-legged robots which is used in many applications.
For instance, they are used to navigate in uneven terrains both on land [3] and underwater [11], or explore the effects of artificial muscles by mimic the muscular and skeletal system of a human leg [75]. They are commonly used as a prototype for testing feasibility of a proposed method or hypothesis in the real world. It is mainly due to the complexity that comes with multi-legged systems where fewer legs can help reduce the overall complexity. For this particular reason, the proposal in the thesis is to use a one-legged robot to reduce complexity for better test the hypothesis.
2.2.2 Robot foot and ankle
The robot foot can be divided into two parts; foot and ankle joint. The foot covers a given surface area used for grip, while the ankle controls the angle of the foot. Compared to a human foot, robot feet are usually less complex and more simplified. Robot feet mostly consist of a single rigid plate, but others might have a toe joint [56] or both toe and heel joints [29]. These joints are usually used to gain extra balance abilities for steady
walking. Ankle joints can be based on the complexity of the human foot and are created to be both dynamic and flexible with multiple degrees of freedom [1, 51]. Some are more static with only one degree of freedom [73] and some have no feet or ankle at all [62]. The foot plays a vital role in a walking robot, and some say that it is the most important part of a robot [57]. In most applications, the foot and ankle joint are used to maintain balance, whereas in this thesis it used to change the morphology of the robot to increase performance and adapt to the environment. In changing the morphology, the suggestion was to use a soft robotics sole or an electrical servo motor. Given the early work in the thesis and the electric servomotor seems more appropriate to use, the soft robotic sole are excluded.
(a) (b) (c)
Figure 2.1: Examples of different ankle joints.(a)Robot with ankle joint with one degree of freedom. Image taken from [73].(b)Bipedal robot using a servo motor to achieve one degree of freedom. Image taken from [26].(c)Robot foot able to achieve pronation and flexion of the foot. Image taken from [33].
To achieve this, the robot in the thesis needs to be able to change the angle of its foot using at least one degree of freedom. The foot should be a simple design for two reasons. Firstly, it does not need any balance properties as the one-legged robot uses a hub to maintain its balance. Secondly, one should consider that using a toe or heel joint is necessary or should be avoided since it may be a source of undesired noise during testing.
2.2.3 Robot activation
Being able to create a durable robot requires not only a good design, but also durable actuators. The activation of a robot is important in creating movement, and there are a number of different actuators to chose from.
Activators exists as electrical servo motors, as well as in hydraulics and pneumatic systems [72]. A commonly used actuator is the electrical servo motor. It is known to provide high precision, easy to control and applicable in creating almost any type of robot [74], as well as walking creatures [28].
Despite their high precision and ease of use, they are fragile, expensive and
has limited power to weight ratio. In contrast, hydraulic and pneumatic systems are known to be robust, durable, have a high power to weight ratio, and the ability to handle impact [38]. Common actuators in such systems are cylinders, and are seen in many walking robots [23, 32, 47, 65]. In addition to the previously mentioned features, they are easy to use, easily replaceable, and have very explosive behavior. Also worth mentioning are artificial muscles. Often inspired by natural creatures musculoskeletal system to replicate or mimic their movement behavior found in both walking [70] and running [53] robots. However, because of the various difficulties in the early work of the thesis experimenting with soft robotics, section 2.1, this type of actuator is avoided.
As the robot should be one-legged for better testing the hypothesis, it should be durable and able to make rapid explosive jumps to create fast gait. Considering the previously mentioned activators, the cylinder con- tains the properties to fulfill the acquirement regarding gait. Furthermore, one has to choose between hydraulic or pneumatic systems. A pneumatic system is favored as the maintenance level in a pneumatic system is less than for a hydraulic, because of the use of compressed air instead of liquid fluid. Also, if leakage would occur, a hydraulic system makes the higher danger of harming equipment and the surroundings.
In regard of automatically adjusting the morphology of the robot, it should be considered using an electrical servo motor as it is precise and this robotic feature does not require the same amount of force as the gait.
2.3 Evolutionary computing
Within computer science, there is a research field called evolutionary computing (EC) [20, p. 13]. The name draws inspiration from the natures abilities to evolve, known as the natural evolution. The natural evolution is an inspiration source because of the natures power of deriving diverse species, where each species are fit to survive in its environment. The evolutionary computing metaphor is related to the nature’s evolution of problem-solving, also known as trial-and-error.
To understand the idea of a natural evolution in computer science a simple explanations is as follows. Starting with a given environment filled up with a population of individuals, where the objective of the individuals are to survive and reproduce. Each individual has a fitness tied up against them, which describe how well they perform. The fitness is determined by the environment and represents the individuals chance for survival and reproduction. Solving problems by stochastic trial-and-error is starting with a collection of candidate solutions. In the context of trail-and-error, candidate solutions are judged by its quality, meaning their ability to solve the problem. It determines which candidates are to keep and used to seed in the further candidate solutions. For a slightly clearer understanding between the two analogies, see table 2.1.
Natural evolution Problem solving
Environment ←→ Problem
Individual ←→ Candidate solution
Fitness ←→ Quality
Table 2.1: The corresponding terminology of natural evolution and problem solving.
2.3.1 Biology inspiration
Darwinian evolution Darwin’s theory of evolution [16] offers an expla- nation of evolution in biology and how it works. The theory also state that natural selection plays a central role in evolution. With a given environ- ment hosting only a limited number of individuals, where all individuals have the basic instinct to reproduce, one need a selection to avoid the pop- ulation size to grow exponentially. Natural selection favors the individuals that fit the environmental conditions the best. This is known as survival of the fittest [20, p. 16], and is one of two cornerstones of evolutionary progress. The other type is phenotypic traits and comes from Darwin’s results from phenotypic variations among members of the population. The phenotypic traits describe the individual behavioral and physical features that affect its response to the environment and other individuals. The traits are evaluated by the environment, and used to determine the fitness of an individual, where each individual has its own unique combination. When evaluating the combination and seeing it is a favorable solution, it is a higher chance of reproduction of the best individuals. If not favorable the individual would be discarded without reproduction resulting in no off- spring. One thing to keep in mind is that some traits are heritable and may be propagated into the individual offspring. Through small and random variations in the phenotypic traits during reproduction, Darwin’s idea was that when this was happening from generation to generation, it would pro- duce new combinations and variations from the best individuals resulting in progressive evolution.
Genetics Looking at molecular genetics can give a broader understand- ing of natural evolution. It is a level below the phenotypic features and is related to heredity. Each individual has an outside and an inside. On the outside, we find the phenotype, where the inside represents the phe- notype on a genetic level, named genotype. Here, a gene could inherit the encoding of the genotype. For example, a gene could determine the visual properties like the eye color, which could be either blue, brown or green. A gene is made up of alleles, where an allele is one of the possible values a gene could have. Figure 2.2 shows the relationship between the genes and alleles witch together builds up a chromosome.
Using the same example as before, the alleles value could contain the eye color and for example, represent the eye color blue. However, in natural
Chromosome 1 0 1 0 ... N
Gene
Allele
Figure 2.2: Overview of a chromosome containing N genes.
systems, the genetic encoding might contain more phenotypic traits and be determined by more than one gene and are not one-to-one. The mutation and recombination of genes cause the difference in the genotype which will always cause a difference in the phenotype.
2.3.2 Why use evolutionary computing?
Developing automated problem solvers plays a central part in computer science. Drawing inspiration from the most powerful natural problem solvers known, the human brain and the evolutionary process. The former leads to the field of neurocomputing, witch Hecht-Nielsen [31] defines as the engineering discipline concerned with neural networks that develop associations between objects in response to their environment, and the latter is the basis for evolutionary computing [20, p, 20].
Since the computerization in the second half of the 20th century, it has been an increasing demand for automated problem-solving. While the growth of problems has increased, the research and development field has not been able to keep the pace. Simultaneously the complexity of new problems has increased, which implies an urgent need for robust algorithms with satisfactory performance. It leads to evolutionary algorithms (section 2.4), which is a tool for creating automated problem solvers that are applicable to a wide range of problems. For specific problems, they do not need much modification to deliver good solutions within a reasonable time frame and are applicable to a large number of problems despite its complexity.
2.4 Evolutionary algorithms
There are many different variants of evolutionary algorithms (EA). They are built up by different components, however the underlying idea of all techniques remains the same. They start by initializing a population of individuals within an environment. Then the population going through a parent selection to select candidates for recombination based on its fitness.
Two or more different candidates are then selected for recombination to generate offspring, while other candidates can be mutated to generate one new candidate. Here, both recombination and mutation are used to derive
even better results. Furthermore, the new candidates and offspring are going through a survival selection where a fitness function determines which individual would be brought to the next generation. During all generations, the same fitness function is applied to test the new generation to further compare with the old generation. This process loops, and terminates when either a stop criteria or a maximum limit of the iteration are reached. A flowchart of the general EA is shown in figure 2.3.
Initialize population
Parent selection Evaluation of
population
Recombination / mutation
Survivor selection Terminate?
Yes No
Figure 2.3: A flowchart of an general evolutionary algorithm.
2.4.1 Evolutionary components
To better understand the EA this section discuss, in more detail, the most important components that build up a particular EA.
Representation Creating a bridge of the real world problem to fit into the EA world is the first step in defining an EA. It usually involves simplifying some aspects of the real world problems to create a well-defined problem.
Furthermore, one should determine how possible solutions should be specified and stored for further manipulation by a computer. These are
the phenotype and genotype, where the representation specifies a mapping from the phenotype onto a set of genotypes. For example, an integer with value 120 could be seen as a phenotype, and its genotype could represent it as a binary number with the encoded value 1111000. It is important to note that the evolutionary search takes place in the genotype space.
Fitness function The fitness function represents the quality of the solutions or individuals in a population, where it returns a fitness score based on how good or bad a solution is. The importance of a fitness function is that it provides an accurate evaluation of solutions. Baresel et al.
[8] state that a well-constructed fitness function may substantially increase the chance of finding a solution and reaching higher coverage. However, for some cases in the real world, a fitness function does not exist or could be too computationally expensive. In these cases, it is necessary to construct an approximation to the fitness function [34].
Population A set of individuals are called a population. Each individual is a representation of a possible solution to the problem, where the individual has its form of a chromosome build up of genes. Depending on the problem, defining a population could be as simple as deciding the population size, which describes how many individuals are in it. To create competition and limit resources, it is common in most EA applications to set the population size as a constant during the evolutionary search.
Since the individuals are defined as a static object [20, p. 30], it is the population that adapts in the form of selection operators. Figure 2.4 illustrates a population of individuals.
Individual 1 Individual 2 Individual 3
Individual N . . . .
1 0 1 0 0 0 1 1
1 1 1 1 1 1 1 0
1 0 0 0 0 0 1 0
1 1 1 1 1 0 1 1
Population
Figure 2.4: Illustration of a population containing N individuals.
Parent and survivor selection There are two main types of selection, parent selection and survivor selection. Parent selection is a process
of selecting parents to mate and reproduce offsprings for the next generation. The selection is typically probabilistic and gives a higher chance for selection of a high-quality individual than a low-quality individual. Allowing low-quality individuals to sometimes participate prevents extremely high-quality solution from taking over the population.
This is known as premature convergence which leads to getting stuck in a local optimum and is an undesirable condition.
The survivor selection process determines which individuals to discard or keep. This is usually based on their age or fitness score, where balancing diversity with high-quality individuals is important. Although in some cases one wants to keep one or more of the best individuals to prevent losing good solutions. This is referred to as elitism.
Variation operators When speaking of variation operators in EA, one speak of two different operators,mutationandrecombination. These are used to generate new individuals from old ones.
The mutation operator is stochastic and takes one input to produce one new individual. In general, it is meant to make a small change in the genotype of an individual. It should be noted that mutation is used differently in various evolutionary algorithms. For example in evolutionary programming, it is used to generate new individuals, while in genetic programming it is not used. Depending on the representation of the real values, it exists various mutation techniques like the ones listed below:
• Nonuniform mutation- Has a fixed distribution which is used on real- valued or floating-point representation. In the early stages of the algorithm, it searches the space uniformly, whereas in later stages, is does a very locally search. This results in a wider search by initialization, and a better locally finely tuned search towards the end [77].
• Uniform mutation - Selects a random gene and replaces its value to a uniform random value within a lower and upper bound. Used in integer and float-point representations.
• Creep mutation - Used on integer representation. Selects a random gene to change its value with a random value between a lower and upper bound.
• Random reset mutation - Regardless of representation, one or more alleles can be added to a new random value, usually at a low probability.
The recombination operator is often referred to as a crossover operator.
It uses two or more individuals as parents to produce one or two new individuals, which is called offspring. The main idea behind this operator is that the produced offspring will use qualities from both parents and be
better than them. The new offspring is created by the crossover operator selecting genes from both parents and passing them to the offspring.
Similar to mutation operators there are a number of different crossover techniques available, and the most common are listed below:
• One-point crossover- Selects a random crossover point used on both parents, copying the first part of the first parent and the second part of the second parent, passing it on to the offspring.
• Two-point crossover- Similar to the one-point crossover, but selects two points instead of one.
• Uniform crossover - Mixing the genes of the parents with a mixing ratio. For example, a mixing ratio of 0.5 would result in half the genes coming from parent 1 and the second half coming from parent 2.
Initialization In most EA applications the individuals in the population are randomly generated. However, a problem-specific heuristic can be used to gain a population with higher fitness [20, p. 34].
Termination condition The termination condition is used to determine when an EA run must end. Usually, the terminating condition is chosen to achieve the most optimal result possible. One reason for this is that some EA’s tends to converge in the later stages with minor improvements. There are several ways of a termination condition:
• When a maximum number of generations is reached.
• When a specified fitness value is reached.
• If there are no improvements in the population for a number of iterations.
• When the diversity of a population is below a certain threshold.
2.4.2 Genetic algorithm
One of the most known subset of evolutionary algorithms is the genetic algorithm (GA). The discovery of the GA was made by Holland while studying adaptive behavior [20, p. 99]. The GAs has been considered as function optimization methods until the work in De Jong’s thesis [17]
helped define the GA, although De Joung has disagreed [35]. Now it is referred to as the simple genetic algorithm (SGA). The algorithm has a binary representation, low probability of mutation and uses a roulette wheel selection. Table 2.2 contains a summarize of the SGA.
While many new features to the GAs have been developed over the years, there is one obvious missing from the SGA, that being elitism.
Representation Bit-strings Recombination 1-Point crossover Mutation Bit-flip
Parent selection Fitness proportional - implemented by Roulette Wheel
Survival selection Generational
Table 2.2: Sketch of the simple genetic algorithm.
2.4.3 Evolutionary strategies
Rachenberg and Schwefel were the ones who invented the evolutionary strategies (ES). The invention was made in the 1960s while they were working on shape optimization application in Berlin [20, p. 101]. ES, in the early days, worked in a vector space and was a one plus one two- membered algorithm, denoted as (1+1). Generation of offsprings was done by adding a random number to each of the elements of a parent vector.
The random number is the mutation step size drawn from a Gaussian distribution, where the mean is zero. If the offspring is fitter than its parent, it would be accepted. Another alternative to the (1+1) ES, is the one comma one (1, 1). Here, the offspring always replaces its parent, in which results in forgetting previous solutions.
The introduction of a multi-membered ES concept came in the 1970s. The concept came with a new notation that was used to describe the individuals asµand the offsprings asσ, where the ES would be described as(µ+σ) and (µ, σ). This lead to a more sophisticated form of step-size control, where the very useful feature, self-adaptation of strategy parameters, came to life in evolutionary computing [20, p. 101]. Self-adaptation of strategy parameters means that during a run, some parameters of the algorithm are changed in a specific manner, which means that the parameters are evolving along with solutions within the chromosomes. A chromosome in ES can be expressed ashx,pi, where xis a vector of the object function and p is the strategy parameters of the algorithm. Since the discovery in 1977, most ES have been self-adaptive, and have commonly been adopted by other EAs [20, p. 101].
Representation Real-valued vectors Recombination Discrete or intermediary Mutation Gaussian perturbation Parent selection Uniform random
Survival selection Deterministic elitist replacement by (µ,λ) or (µ+ λ)
Specialty Self-adaptation of mutation step sizes Table 2.3: Sketch of evolutionary strategies.
2.4.4 Evolutionary programming
Evolutionary programming (EP) came to life in the 1960s and was developed by Fogel et al. to simulate evolution, where the aim was to generate artificial intelligence [20, p. 103]. At this time, intelligence was viewed as a systems ability to adapt its behavior to meet specified goals within different environments. The capability of adaptive behavior and prediction on its environment was considered a requirement.
These days the EP uses real-valued representations, while in the past they used finite state machines as individuals. The EP has its similarities to the ES, but there are some differences. The differences are the selection mechanisms and the non-existing recombination. In ES the selection of parents is done stochastic, where the selection is done deterministically of theµbest from the (µ+λ)union. On the other hand, EP generates only one offspring where the parents and offsprings compete against each other for survival in a stochastic round-robin tournament. The table 2.4 shows a version of a standard EP.
Representation Real-valued vectors Recombination None
Mutation Gaussian perturbation
Parent selection Deterministic (each parent creates one offspring via mutation)
Survival selection Probabilistic(µ,µ)
Specialty Self-adaptation of mutation step sizes (in meta-EP) Table 2.4: Sketch of evolutionary programming.
2.4.5 Genetic programming
A young member of the evolutionary algorithm family is genetic program- ming (GP). It is different from other EAs, both in its application and rep- resentation as its chromosome are represented as a tree structure. As typi- cally evolutionary algorithms are applied to optimization problems, the GP could be applied in machine learning. While EAs are looking for maximum payoff, is the GP looking to find models with the maximum fit. In GP there are various ways of initializing trees, where the so-called ramped half-and- half method is the most used method. Initially, a maximum depth of trees is chosen, where each member of the initial population is generated using one of two methods: the full method or the grow method. In the former does each branch have a maximum depth, while in the latter can branches have different depth up to a maximum depth. Offspring in GP are typi- cally generated by either recombination or mutation, whereas other EAs use recombination followed by mutation. Early research done by Koza [39]
suggests that GP works without mutation, while in later research Banzaf et al. [6] recommended 5% mutation rate. However, a small mutation rate
is recommended. Although for most real world problems there is no best mutation rate.
Representation Tree structures Recombination Exchange of subtrees Mutation Random change in trees Parent selection Fitness proportional Survival selection Generational replacement
Table 2.5: Sketch of genetic programming.
2.4.6 Optimization and search
Exhaustive search The objective of an exhaustive search is to examine all possible solutions and pick the best one. It is guaranteed to find the global optimum, but by checking each solution makes it computationally slow and bad for large size problems or problems with some or more complexity.
Greedy search The greedy algorithm passes through the data only one time, making the locally optimal choice at each stage with the purpose of finding a global optimum. This makes the algorithm computational cheap, but it is not guaranteed to find the optimal solution nor a good solution.
Hill climbing In a hill climbing algorithm (HC), the main idea is to perform a local search around a current solution. Here, it progresses by choosing the enhanced solution until there are no better solutions. The initial solution is chosen randomly and can have a large impact on the end result. The HC is terminated with pre-defined parameters, typically after a number of iterations or if there are no improvements after a length of time. Since the HC has a potential of finding the global maximum, there is no guarantee of it. This is because the algorithm can be stuck at a local optimum, where there are several potential solutions. This is illustrated in figure 2.5.
Simulated annealing Simulated annealing (SA) is a method that is based on the way a physical system, in the real world, can be brought into very stable, low energy states [41, p. 207]. By heating the system material and making sure there is plenty of energy around it, results in randomly effective behavior. To cool down the system material, an annealing schedule is applied, which makes the system return to its low energy state and alter the materials physical properties.
Local optima
Global optima
A
B
Figure 2.5: Hill climbing algorithm concept illustration. The points are randomly placed and use the greedy approach to reach the optima. Here, point A is going towards a local optima while B to the global optima, which shows its vulnerability to noise and other uncertainties in data.
In practice, the SA method starts with an arbitrary high temperature variable. A new point is randomly chosen at each iteration. The distance between the current point and the new point is based on a probability distribution with a scale proportional to the temperature [44]. All the new points that improve the solution are accepted, where also a new point that are worse than the current solution has a probability of being selected. In the early stages of the algorithm, this allows the SA of exploring global solutions without danger of being trapped in a local optimum, where the temperature systematically decreases as the algorithm goes on.
In connection with the thesis, one look for the algorithmic convergence instead of temperature decrease.
2.4.7 Exploration and exploitation
An optimization or search algorithm can be separated into methods that perform explorationor exploitation. Exploration involves testing new solutions of the whole search space, like the exhaustive search. Exploitation is testing out local variations of a current best solution. An example of exploitation is the hill climb algorithm.
2.4.8 Evaluating performance of EA
Since the EAs nature is stochastic, the evaluation relies on the performance of multiple runs of the same EA, using the same parameters. This leads to
the three most commonly used methods to measure performance in EAs [20, p. 126], given as follows:
• Success rate (SR)
• Mean best fitness (MBF)
• Average number of evaluations to a solution (AES)
SR can be defined as the percentage of runs and could be used to measure the required quality of a solution. It can not be used in problems where the optimal solutions are not possible to find or if a criterion (lower/upper) is not accessible. In an EA, the problem using a fitness measure can be defined by the MBF. Recording the best fitness of an individual each iteration, and taking the average of these values is the MBF. It could also be used to record the worst-ever or best-ever fitness to study different scenarios. Since the SR are not able to be defined for some problems and the MBF are applicable for all, one should note that there is no general rule of which one to use for measuring the different EAs.
Additional one can measure the efficiency or speed to determine the performance of an EA. The measures depend on various things, for example, hardware and operating system to mention a few, but most commonly is the speed measured by the CPU time or user time. As of testing experiments on different machines could lead to various results, a workaround is to count the number of points visited in the search space. The measure of EAs is always based on the number of independent runs, because of its stochastic nature. For this is the AES used, where it only calculates the average of successful runs. As of this, the AES can give a good comparison of the algorithm speed, but if run on different hardware can give misleading results because it can produce extra computational efforts that increase performance or a large difference in evaluation time.
Visualization of data The results from an EA could be shown in various ways. Common ways to display data are in tables, charts, and graphs, where the latter might be the most preferred method. With the use of graphs, data can be visualized with different curves and plots to show the development in a population during the evaluation of the algorithm.
This could be done showing a curve of the different runs, mean fitness from every generation through the evolution or compare different EAs with each other. Displaying the data in this particular way is used to show the efficiency of the algorithm, while the variance in fitness can be plotted to show the robustness of the algorithm. Another way is to use scatter plots.
For example, show the candidate solutions and find the areas to determine the best-ever or worst-ever solutions. Additionally, 3D graphs can be used to show data that can gather hills which can visualize the best point of a certain measurement of an algorithm. As well is box plot a popular way of showing data. It is used for displaying numerical data points through their
quartiles (figure 2.6). A box plot contains of three quartiles: Q1, Q2 and Q3. Q1 is the middle number between the smallest value and Q2, where the latter is the median of the data. Q3is the middle number between Q2 and the highest value in the data. A box plot also contain whiskers which stretches fromQ1toQ1−1.5·IQRandQ1toQ3+1.5·IQR. The IQR is the interquartile range of data betweenQ1andQ3. Additionally, it can display outliers which are data that lies outside of the defined whiskers.
Q1 Q2 Q3
Outliers IQR
minimum maximum
Q1−1.5·IQR Q3+1.5·IQR
Figure 2.6: Explanation of a box plot displaying whiskers and outliers.
2.5 Evolutionary robotics
Evolutionary robotics (ER) is an experimental research field, and can be considered as a subfield of robotics [18], where the creation of robust, adaptive and autonomous robots is the main goal. Using EAs to optimize robot controllers or morphology are some examples of what differs ER from other robotic research fields, which nowadays use machine learning for optimization of different aspects of a robot. Where typical examples can be artificial neural networks [30] and reinforcement learning [58].
2.5.1 Offline and online evolution of robots
When designing robots one distinguish between offline andonline evolu- tion.
A common approach in evolutionary robotics is to use offline evolution.
It is used to evolve a controller before the operational period using an evolutionary algorithm, and is usually evolved in a simulator. When the user is satisfied with the results of the evolved controller, it is deployed on the real robot, where the operational stage can start. At this stage, the evolved controller does not change, and operates only with the static settings given before deployment. This approach can be a problem if the robot encounters environmental conditions that differ from the ones encountered in the simulation. A possible scenario is that the evolved controller might be incapable of solving the task since it is unable of self- adaptation.
An alternative to offline is online evolution. Instead of evolving a controller beforehand, the online method is used to evolve the controller on the spot during the operational stage using an evolutionary algorithm that executes while performing tasks. This enables the robot to be capable of self-adaptation and allows the robot to modify its behavior responding to changes in the environment in an autonomous manner. A typical workflow of both offline and online evolution is illustrated in figure 2.7.
Offline evolution Online evolution
Deploy on robot Simulate until
satisfied
Evolve on controller
Time flow
Design stage Operational stage
Figure 2.7: Workflow of robot design distinguishing offline and online evolution, where they are separated by the moment of deployment.
Most of the ER research has been experiments using simplified physics simulators [49]. In simulations, the only limitations lie within the computational power of the device, where the offline method is used to save time. However, a problem that follows is the leap in performance when applying the simulated results on a robot performing in the real world. The problem is referred to as the reality gap, and appears because of the physics simulation are not able to replicate the real world and uses ideal conditions without noise. The result might be that the physical robot performs differently than in simulation. One solution for closing the reality gap can be done by switching from simulation to evolve online on the hardware of the system. A system like this would yield more realistic results and display the same noise and unpredictability as in the physical environment [19].
2.5.2 Problems in ER
There are various problems that come with ER in the real world. In partic- ular, the fitness function can be a factor which can lead to problems, where the fitness functions being verynoisyorcostlyare common challenges. Ad- ditionally, the fitness landscape can have so-calledno-go areas.
Noisy fitness In the physical world, noise is inherent. It can come from small differences in the environment and can cause huge differences in measurements. Typical physical problems that can have an impact on the result are variation in heat and friction. Heat can cause differences on
hardware both in the computer and the controller, while friction on a robot can cause a difference in measurements or performance.
Costly fitness To measure real world robotics takes time. For example, a robot can spend several minutes navigating through a trail to find a target.
The algorithm might require the controller to record many measurements, which can be very time-consuming.
No-go areas In a simulated environment, evaluation of a poor candidate solution can be a waste of time. In contrast, using poor candidate solutions on a robot in the real world can damage the robot or harm the environment.
For this reason, using such candidate solutions in hardware must be avoided during the evolutionary search [20, p. 251].
Real world example A practical example of problems in ER is the paper in [23]. The paper presents a bipedal pneumatic robot performing in the real world using evolutionary algorithms to optimize the robot’s gait. In experiments, they find various practical side effects of running a robot in the real world such as time consumption and mechanical wear out.
In addition, they address unpredictable behavior in their system caused by noise due to heavy vibrations, varying pneumatic air pressure, and friction. These side effects could cause significant variation in fitness for the same chromosomes, and the robot to slip or stumble. However, their evolutionary algorithm was found to perform well despite the very noisy environment. Taking lessons from these observations can favor this thesis and help the investigation of the hypothesis.
Tools and engineering process
This chapter gives an introduction to the different tools and processes, focusing on the physical components and software that are needed in the thesis.
The following tables give a quick overview of the various hardware and software used in the thesis. Their functionality and usage are explained further in this chapter.
Hardware
Tool Name Model
Microcontroller Arduino Mega 2560
Servo Dynamixel MX-64
USB communication converter USB2Dynamixel TTL & RS232
Compressor HBM Machines HBM 120 liter
Valve SMC SY7120-6G
Cylinder Norgren RT/57216/M/40
Motor driver (dynamixel) SMPS2Dynamixel 12V
Motor driver (valve) Pololu md07a
Adapter 12V (dynamixel) DELL ADP-1500BB B
Adapter 12V (valve) Mascot Model 9827
Adapter 24V (encoder) DYS DYS650
-240210W-K
3D printer (FDM) Fortus 250mc
3D printer (FFF, CFF) Markforged Mark X7
Table 3.1: A list of the different hardware and equipment used in the thesis.
