Machine learning

As mentioned MATLAB includes several toolboxes for machine learning, one of them being the Neural Network Time Series toolbox. This toolbox allows for dynamic neural network training with several algorithms such as the Levenberg-Marquardt algorithm, Scaled Conjugate Gradient method and the Bayesian Regularization algorithm. A dynamic network allows for a delay in the response from previous inputs. This means that the

39

response at a current time step also depends on the response at previous time steps. A network with a maximum delay of 1 can remember the response from one time step back in time, while a delay of 3 means the network can look back at the response for the last three time steps. Here, a delay of 4 was used.

The goal is to make a model that can predict the strains in the beam over time and with varying temperature. Thus, the output when training the algorithm is the strain, while time and temperature are used as the inputs. As the data set contains data for 10 years with 100 steps each day this results in 365001 data points. The data is divided into one part used for creating the model and one part that is used to predict future strains with the model that has been created. 3 years of data were used to train the models and the rest of the data were used for checking how well the models could predict future strain responses. To create a model with good generalization and avoid overfitting, it is necessary to divide the data set into a training set, a validation set and a test set. Here, 60%

of the data points from the 3-year period was used for training, and 20% was used to validation and testing each.

When measuring strains with a sensor, there will always be some amounts of noise in the measured data due to the equipment itself or external disturbances. Signal-to-noise ratio SNR is a ratio used to describe how much noise a signal contains. The ratio takes the power of a signal divided by the power of the noise. If there is no noise in a signal, the SNR will go towards infinity. If the noise is of the same magnitude as the signal, then the SNR will be equal to 0. A high SNR value means that the signal contains less noise than a lower value.

Since there always will be some amounts of noise in data obtained from measurements of real structures, some noise was also added to the data used in this project. It is assumed that measurements at different points but with the same type of strain sensor will contain noise of the same magnitude. This means that for higher magnitudes of strains the SNR value will be higher while if the magnitude of strain is small the SNR value will also be small. It is assumed that the sensors used here were able to obtain a relatively clean signal for the highest strain values with an SNR = 25 dB. This power of noise resulted in an SNR value of 7 dB at points with lower strain signals, which is a very noisy signal. The signal-to-noise ratios for different points in the beam is shown in table 6.6.

Mean squared error was used as the optimization function. This means that under training the algorithm tries to find parameters for the model that will give an MSE value as close to zero as possible. Because the values of strains are very small, a number to the power of minus 4, the MSE value becomes very small as well, down to a number to the power of minus 10. Since the MSE value depends on the magnitude of the output and target values, it is hard to evaluate the performance of different models based solely on this value. To evaluate the generalization of the models, the coefficient of determination

40

R² was used, equation (2.5). R² is based on a normalization and gives a better foundation to evaluate the models against each other.

Table 6.6: Signal-to-noise ratios.

Node Coordinate (x , y) Signal type SNR [dB]

3 (12.5 , 1) Strain ~25

6 (12.5 , 0) Strain ~25

18 (0.1 , 0) Strain ~7

1143 (12.5 , 0.3) Strain ~13

All – Temperature ~25

Several different network structures were tried to find a structure with optimal performance for the problem. The number of inputs and outputs depends on the nature of the problem. In this case, there were two inputs, time and temperature, and one output, strain. The number of hidden layers and the number of neurons in each hidden layer is up to the user to decide. The default in MATLAB is set to one hidden layer with 10 neurons.

The optimal structure seems to be a network with two hidden layers with 4 neurons in the first hidden layer and 3 neurons in the second hidden layer, as shown in figure 6.13.

Figure 6.13: Neural network structure used in MATLAB.

Another training parameter that affects the performance of the model is the minimum gradient for the Levenberg-Marquardt algorithm. If the minimum gradient is reached under training the algorithm will stop and the parameters at that point will be used for the model. The default value in MATLAB is set as 1e-7, which proved to be too high of a value for this problem, probably because of the small strain values. To get a better generalization of the model, the minimum gradient was set to a smaller value, 1e-12.

41

7 Analysis