We have proposed delayed feedback systems as reservoir computer. By using only one nonlinear node and a delay line we succeed in creating an analogy with a vast network of nodes.
A first step required to successfully process in put by using delayed feedback systems is an input pre-processing stage. We have outlined a procedure that combines time-multiplexing of the input with imprinting a mask on every input value. This enables us to provide a proper scaling factor for each virtual node.
By defining the mask correctly we can ensure that the node output is always in a transient regime. Moreover, the chosen separation distance between two virtual nodes, defined by the mask, is an important tool to manipulate the interconnectivity between virtual nodes. We have deduced an analytical expression for the interconnectivity weights and validated this numerically.
Finally, we have decided on a way to read out reservoir states from the system.
These states can be used for training, using linear training algorithms.
In Chapters 3 and 4 we will model practical implementations of the de-layed feedback reservoir concept, evaluating their performance using several benchmark tasks. Next to elaborating on numerical results, the experimen-tally obtained performance of an electronic and an opto-electronic system are shown.
3
Modeling an electronic implementation
After explaining the general concept in Chapter 2, we now model a practical implementation of the single node delayed feedback reservoir. To demon-strate our concept, we have chosen the widely studied Mackey-Glass oscillator [91], which was originally introduced as a model of blood cell regulation. It has already been used extensively in the characterization of chaotic systems [92]. This choice originates from the fact that this nonlinearity can be im-plemented in an easy and noise robust way using a simple electronic circuit.
Moreover, the shape of the nonlinearity and the strength of the nonlinear contribution can be tuned, allowing for exploration of the optimal settings depending on the task at hand. The numerical simulations have been per-formed by the author. The experiment was carried out by dr. M.C. Soriano, prof. C.R. Mirasso and prof. I. Fischer at IFISC, Palma de Mallorca, Spain.
The author assisted in the input driving and the training of the experimental system. The work reported in this chapter is partly covered in Appeltant et al. [17]
3.1 Mackey-Glass delayed feedback oscillator
The equation of the system is given by
dx(t0) dt0 = 1
T
"
−x(t0) + C·x(t0−τ0) 1 +bp[x(t0−τ0)]p
#
, (3.1)
withCbeing the coupling factor,pthe nonlinearity exponent,ba nonlinearity coefficient,T the intrinsic timescale and τ0 the delay time. The intrinsic time
46 3 Modeling an electronic implementation
0 1 2 3 4 5
0 0.5
1 1.5
2 2.5
Input Voltage (V)
Output Voltage (V)
Fig. 3.1: Mackey-Glass nonlinearity shape and experimental fit. Experimental transfer function (black) compared to a fit us-ing the Mackey-Glass equation (red). Fit parameters correspond to C = 1.33, b = 0.4 and p = 6.88 in Eq.(3.4). Figure taken from sup-plementary material of Appeltant et al. [17].
scale is a measure for the response time of the system and will be used as a reference time scale in the stage where input is added to the equation. The delayed state of x appears in the nonlinearity term on the right hand side, implying that we have nonlinear feedback. The second term between the brackets is the nonlinear term and an example of the shape of this transfer function is given in Fig. 3.1.
When extending Eq.(3.1) to the case where also an external input is injected into the system this equation becomes
dx(t0) dt0 = 1
T
"
−x(t0) + C·[α·x(t0−τ0) +β·J(t0)]
1 +bp[α·x(t0−τ0) +β·J(t0)]p
#
. (3.2)
The factor α determines how much of the feedback signal is mixed with the input, while the factor β scales the magnitude of the input signal. The mixing of input and feedback signal happens just before the re-injection into the nonlinear node. We have rescaled the variables and parameters in the previous equation to obtain a minimum number of significant parameters, as follows: η =Cα, γ =bβ, X =bαx and t=t0/T. This transforms Eq.(3.2) to
3.1 Mackey-Glass delayed feedback oscillator 47
0 1 2 3 4
0 0.5
1 1.5
2 2.5
η
Extrema of X
Fig. 3.2: Orbit diagram for Mackey-Glass system given by Eq.(3.3) and γ = 0. The feedback strength η is varied, while the extrema of the X values are plotted for every value of η. Different dynamical regimes can be observed, for 0 ≤ η ≥ 1 we find a zero fixed-point that for larger values ofη evolves to a non-zero fixed-point, limit cycles and finally deterministic chaos. The delay time was kept constant atτ = 80.
dX(t)
dt =−X(t) + η[X(t−τ) +γJ(t)]
1 + [X(t−τ) +γJ(t)]p. (3.3) withXdenoting the rescaled dynamical variable, ta dimensionless time, and τ the delay in the feedback loop. The characteristic time scale of the oscilla-tor, determining the decay in the absence of the delayed feedback term, has now become equal to 1 because of the time normalization. The parameters η and γ represent feedback strength and input scaling, respectively. When numerically simulating the system without input (γ = 0) we can construct an orbit diagram as the one shown in Fig. 3.2. We investigate the dependence on the feedback strength η, while plotting the minima and maxima of the X variable. This representation allows to easily identify the different dynamical regimes that can be addressed when scanning one parameter of the system.
By adjusting the value of η we can guarantee that the system operates in
48 3 Modeling an electronic implementation a stable fixed-point in the absence of external input (γ = 0, p = 7). With input, however, the system might exhibit complex dynamics. The choice of this particular nonlinearity has two main advantages. Firstly, it can be easily implemented by an analogue electronic circuit [93], which allows for fine parameter tuning [94]. Secondly, the exponentpcan be used to tune the nonlinearity if needed. Whenpis chosen to be very small, the system becomes very weakly nonlinear. For p = 0 the nonlinear contribution disappears completely.