Santa Fe laser data

4.3.2.1 Numerically obtained performance

Next, the Santa Fe time series prediction task is used to evaluate the per-formance of this opto-electronic delayed feedback reservoir. The error is ex-pressed as the NMSE between the predicted point and the target. Numerical results are shown in Fig. 4.7.

We vary the input scaling versus the offset phase of the nonlinearity. The performance is clearly phase dependent, however, for small values of γ the phase dependence almost disappears. The region of the nonlinearity that is scanned decreases withγ, making the system less sensitive to the exact shape of the nonlinear transfer function. Given the fact that smaller input scalings will explore only a more linear part of the nonlinearity, one can conclude that to tackle the Santa Fe time series prediction task only a weakly nonlinear processing is required.

4.3.2.2 Experimentally obtained performance

This benchmark has also been solved experimentally. The corresponding results are depicted in Fig. 4.8. For an intermediate feedback strengthη= 0.2 (blue points), a strong dependence of the NMSE on φ is found. For φ = 0.1π we experimentally obtain the lowest prediction error with a NMSE = 0.124 ±4·10⁻⁴. The experimental value of γ is not explicitly known, but it is estimated to be around 0.02. We note that the numerically achieved results are up to one order of magnitude better, as can be seen in Fig. 4.7 or in ref. [15, 74]. When including quantization noise as is present in the experiment, the performance is of a similar level as the experimental one.

To provide evidence that the performance indeed stems from the interplay of high-dimensional mapping and nonlinearity, and not from the nonlinearity

4.3 Results 71

0.2 0.4 0.6 0.8 1

0.2 0.4 0.6 0.8 1

0.2 0.4 0.6 0.8 1

γ

φ/π

NMSE

Fig. 4.7: Numerical results for the Santa Fe laser prediction task. The used nonlinearity is of the Ikeda type. The dependence on the nonlinearity offset phase and the input scaling on the reservoir computing performance on the Santa-Fe laser data set is investigated.

Best performance forη = 0.2 is found for several values of φas long asγ is taken small enough. The minimum NMSE is found to be 0.04, comparable to values found in literature when using traditional reservoir computing approaches on this benchmark.

72 4 Modeling an opto-electronic implementation

η

Fig. 4.8: MZM phase dependence of the reservoir computing performance using the Santa-Fe data set. Best experimental per-formance forη= 0.2is found aroundφ₀= 0.1π, φ₀ = 0.5π, φ₀ = 0.7π and φ0 = 0.85π phase values in the vicinity of local extrema of the transfer function of the MZM (see Figs. 4.10(d), 4.10(a), and 4.10(b).

Figure taken from [69].

alone, we in addition plot the data obtained when disconnecting the feedback line (red points). The output value of the nonlinear node is sent into the delay line and is used for training. However, they are not fed back into the nonlinear node. The lower performance without feedback loop is clearly visible.

4.3.2.3 Comparison with state of the art

For quantitative performance results described in literature we refer to Table 4.2. Numerically we obtain a slightly worse performance for the Ikeda system than for the Mackey-Glass delayed feedback oscillator. The experimental implementation performs significantly worse. We attribute this mainly to the 10 bit resolution of the experimental setup. Effects of quantization noise are studied in Chapter 5, section 5.4.2.

4.3 Results 73

Table 4.2: Santa Fe performance literature review. The first performance column gives the results found by by Rodan et al. [79].

The second and the third column show the results found with a delayed feedback reservoir numerically and experimentally, respectively.

Res. size NMSE [79] NMSE Num. NMSE Exp.

200 0.00819 -

-400 - 0.04 0.124

4.3.3 Isolated spoken digit recognition

4.3.3.1 Numerically obtained performance

As we did for the case of the Mackey-Glass nonlinearity the TI46 speech corpus is used, see Chapter 2, section 2.2.2. The performance for this task is characterized by the word error rate (WER) as well as a margin to the closest competitor digit. In Fig. 4.9 two simulation results are shown. In Fig. 4.9(a) the input scaling is scanned versus the offset phase, while in Fig.

4.9(b) the feedback strength is scanned versus the offset phase. Contrary to what was observed for the NARMA10 benchmark, for the isolated spoken digit recognition the best operating point is not found at the lowest values of γ. For this test, memory is of less importance and nonlinear transformation plays a more crucial role. For spoken digit recognition the performance is clearly phase dependent and some relatively wide regions are found were the error corresponds to less then 1 mistake out of 500 on average.

In Fig. 4.9(a) we chose η = 0.3. The region of good performance is widest for γ = 0.4 or γ = 0.5. In Fig. 4.9(b) the input scaling is chosen asγ = 0.5.

The offset phase is scanned from 0 to π. In this figure, instead of the WER we plot the log(WER) to have a more clear distinction between regions with good performance and parameter regions that perform less well. We see that the best performance is obtained when the bias point is situated close to a maximum or a minimum of the nonlinearity shape. In this region the nonlinear shape is strongly nonlinear, which implies less memory, but more nonlinear mixing of the input signals. These results confirm that isolated spoken digit recognition relies more on identification of the general shape than on memory of previous inputs.

4.3.3.2 Experimentally obtained performance

The experimental performance of the system can be found in Fig. 4.10. Fig.

4.10(a) and Fig. 4.10(b) show the dependence of WER and margin in the

74 4 Modeling an opto-electronic implementation

Fig. 4.9: Numerical results on the isolated spoken digit recog-nition task. An Ikeda nonlinearity is used. The WER is shown in color code. (a)γ is scanned versusφfor a constant feedback strength η = 0.3, (b)ηis scanned versusφfor a constant input scalingγ = 0.5.

η-φ plane. The best operating point is found to be around η = 0.3 and φ= 0.89π, where after repeated measurements the WER<0.02% is reached.

Fig. 4.10(c) shows the WER and margin as a function of φ for η = 0.3. It can be seen that good performance is not limited to a single point with the WER remaining below 0.2% for the range 0.75≤φ≤0.95. Fig. 4.10(d) shows the MZM transmission as a function ofφ. A comparison between Figs.

4.10(c) and 4.10(d) allows an interpretation of the φ dependence. At values of φ not far from the points of strongest nonlinearity in the MZM response is where spoken digit recognition works best. In contrast the performance dramatically decreases in proximity to a linear response of the MZM.

The numerical results in Fig. 4.9(b) can be compared to the experimental results in Fig. 4.10(a). Both have the WER expressed in logarithmic scale to make the general trend more visible. In both situations valleys of good performance are found at the same positions, φ≈ π/2 and φ ≈9π/10. The small differences between the two performance plots can be explained by two points. Firstly, there is the fact that the exact value of γ cannot easily be matched between numerics and experiment. Secondly, in the experimental setup there is the presence of noise, both system noise and quantization noise. As was observed for the Mackey-Glass nonlinearity in Chapter 3, noise decreases the margin. In simulations, small irregularities in the shape of the valley of good performance can be observed because no noise was taken into account. In the experiment the margin is lower and we observe the general trends of good or bad performance.

4.3 Results 75

η ηη

WER

(a) (b)

(c) (d)

Fig. 4.10: Experimental results on Isolated Spoken Digit Recog-nition task. (a) and (b) show the W ER and margin, respectively, for spoken digit recognition in the (η−φ)-plane (feedback strength vs. MZM phase). The two figures of merit show a similar depen-dency on both parameters, with excellent performance atη = 0.3and φ= 0.89π. (c) Detailed dependence of the performance on the MZM phase at η = 0.3. (d) MZM transmission function as a function of phaseφ. Figure taken from [69].

76 4 Modeling an opto-electronic implementation

Table 4.3: Isolated Spoken Digit Recognition performance lit-erature review. The used nonlinearity is of the Ikeda type. The first performance column gives the results found by Paquotet al. [76]

and the second column the ones obtained with our delayed feedback reservoir. Both results are experimental.

Res. size WER (%) [76] WER (%) Our system

200 0.4

-400 - 0.02

4.3.3.3 Comparison with state of the art

For quantitative performance results described in literature we refer to Table 4.3. The best performance found here is similar to the result found for a Mackey-Glass type nonlinearity, indicating that this task is not so sensitive to the exact shape of the nonlinearity. In the similar experiment performed by Paquot et al. at the ULB [76], the obtained performance on the isolated spoken digit recognition task is WER = 0.4% or 200 nodes.