Optimal control of forward–backward mean-field stochastic delayed systems

Optimal control of forward-backward mean-field stochastic delayed systems ^∗

Nacira AGRAM

and Elin Engen Røse

9 September 2017

Abstract

We study methods for solving stochastic control problems of systems of forward- backward mean-field equations with delay, in finite and infinite time horizon. Necessary and sufficient maximum principles under partial information are given. The results are applied to solve a mean-field recursive utility optimal problem.

Keywords: Optimal control; Stochastic delay equation; Mean-field; Stochastic maximum prin- ciple; Hamiltonian; Advanced backward stochastic equation; Partial information.

2010 Mathematics Subject Classification:

Primary 93EXX; 93E20; 60J75; 34K50 Secondary 60H10; 60H20; 49J55

1 Introduction

Stochastic differential equations involving a large number of interacting particles can be approximated by mean-field stochastic differential equations (MFSDE). Solutions of MFSDE typically occurs as a limit in law of an increasing number of identically distributed interacting processes, where the coefficient depends on an average of the corresponding processes. See e.g. Carmona, Delarue and Lachapelle [4]. Even more general MFSDE with delay can be

∗The authors would like to dedicate this paper to Prof. Bernt Øksendal, in the occasion of his 70^th birthday, in gratitude for his support and help.

1Department of Mathematics, University of Oslo, P.O. Box 1053 Blindern, N–0316 Oslo, Norway and University of Biskra, Algeria. Email: naciraa@math.uio.no.

2This research was carried out with support of the Norwegian Research Council, within the research project Challenges in Stochastic Control, Information and Applications (STOCONINF), project number

used to model brain activity in the sense of interactions between cortical columns (i.e. large populations of neurons). As an example in Touboul [15], for a finite-dimensional compact set Γ, the state of each neuron in the network is described by a variableX ∈R, satisfies the network equations

dX(t, r) = f(t, x)dt+ Z

Γ

E[b(r, r⁰, x, X(t−τ(r, r⁰))(r⁰))]_x=X(t,r)λ(dr⁰)dt +σ(r)dB(t, r),

where f : Γ× R → R governs the intrinsic of each cell, B is a Brownian motion model the external noise, λ is a probability measure, σ : Γ → R is a bounded and measurable function of r ∈ Γ modeling the level of noise of each space location and b : Γ²×R² →R is the interaction function of a neuron located at r⁰ with voltage y on a neuron at location r with voltage x. The function τ(r, r⁰) : Γ² →[0,∞) is the interaction delay between neuron located atr and atr⁰.

Such equations are also used in systemic risk theory and other areas as it is mentioned in Hu, Øksendal and Sulem [7]. In Meng and Shen [9], they deal with stochastic optimal control of MFSDE with discrete lag in the sense that at time t, both the drift and the diffusion coefficients depend on the solution of the state X(t) and some previous values, X(t−δ) for a given a constant delay δ >0.

In our setting, we will consider a more general class of memory dynamics of mean-field forward-backward stochastic differential equations (MF-FBSDE) with delay. The delay in our states is of the form

Z 0

−δ

X(t+s)µ₁(ds), . . . , Z 0

−δ

X(t+s)µ_d(ds)

,

for a bounded Borel measures µ·.

Stochastic optimal control of SDE or FBSDE with delay in infinite horizon was studied by Agram et al [1] and Agram and Øksendal [2].

Different types of stochastic differential equations with memory has been discussed in the seminal work of S.E. A. Mohammed [10].

Moreover, when applying stochastic maximum principle of such an equation, we obtain an adjoint process which is a mean-field advanced backward stochastic differential equation (MF-ABSDE) with jumps. This type of ABSDE with jumps has been studied by Øksendal et al [11] in finite time horizon and by Agram et al [1] in infinite horizon. Existence and uniqueness of MF-ABSDE has been proved by Meng and Shen [9].

The paper is organized as follows: In the next section, we derive a sufficient version of the stochastic maximum principle (a verification theorem). The third section is devoted to study both sufficient and necessary stochastic maximum principles in infinite time horizon.

We apply our result to solve an optimal consumption with respect to mean-field recursive utility.

2 Finite horizon stochastic mean-field optimal control problem

Consider a L´evy process ζ defined on a probability space (Ω,F, P). The jump measure N([0, t],B) gives the number of jumps ofζ up to time t with jump size in the setB ⊂R⁰ :=

R− {0}. The L´evy measure ν(·) of ζ is defined by ν(B) = E[N([0, t],B)] and N(dt, de) is the differential notation of the random measure N([0, t],B). Intuitively, e can be regarded as a generic jump size. Let ˜N(·) denote the compensated jump measure of ζ defined by N˜(dt, de) := N(dt, de)−ν(de)dt, whereν(de)dt is the compensator of N.

Define F = {F_t}t≥0, the filtration generated by a standard Brownian motion B(·) and an independent compensated Poisson random measure ˜N. The information available to the controller may be less than the overall information.

Let δ >0 be a given delay constant, π is our control process. From now on, we will denote by the bold X(t),

X(t) := (X₁(t), X₂(t), ..., X_d(t)) :=

Z 0

−δ

X(t+s)µ1(ds), . . . , Z 0

−δ

X(t+s)µd(ds)

,

for bounded Borel measuresµi(ds), i= 1,2, .., dthat are either Dirac measures or absolutely constant. Let us include some examples of delay terms appearing in models with memory:

Example 2.1 Suppose d= 1 and µ:=µ₁ :

1. If µ is the Dirac measure concentrated at 0, then X(t) :=X(t).

2. If µ is the Dirac measure concentrated at −δ, then X(t) :=X(t−δ).

3. If µ(ds) = e^λsds, then X(t) := R0

−δe^λsX(s)ds.

Let φ denote the set of (equivalence classes) measurable functions r:R0 →R. Here b =b(ω, t,x, π) : Ω×[0, T]×R^d×U → R,

σ =σ(ω, t,x, π) : Ω×[0, T]×R^d×U → R, γ =γ(ω, t,x, π, e) : Ω×[0, T]×R^d×U ×R0 → R, g =g(ω, t,x, y, z, k(·), π) : Ω×[0, T]×R^d×R×φ×U → R,

K =K(ω, t, e) : Ω×[0, T]×R0 → R.

Let b, σ, g and γ are given F_t-measurable for all x∈R^d,y, z ∈ R, k∈ φ, π ∈U where U is a convex subset ofR and e∈R⁰.

We want to control a process X^π(t) = X(t) given by a following pair of MF-FBSDE with

dY(t) = −g(t,X(t), Y(t), Z(t), π(t), ω)dt+Z(t)dB(t) +R

R0K(t, e, ω) ˜N(de, dt), t ∈[0, T], (2.2) with initial conditionX(t) = X₀(t), t ∈[−δ,0] and terminal conditionY(T) = aX(T) which a is a given constant in R0.

The set U consists of the admissible control values. The information available to the con- troller is given by a sub-filtration G = {G_t}t≥0 such that G_t ⊆ F_t for all t ≥ 0. The set of admissible controls, i.e. the strategies available to the controller is given by a subset A_G of the c`adl`ag, U-valued and Gt-adapted processes in L²(Ω×[0, T]).

Assumption (I)

i) The functions b, σ, γ and g are assumed to be C¹ (Fr´echet) for each fixed t, ω and e.

ii) Lipschitz condition: The functions b, σ and g are Lipschitz continuous in the variables x, y, z, with the Lipschitz constant independent of the variables t, π, ω. Also, there exits a function L∈L²(ν) independent of t, π, ω, such that

|γ(t,x, u, e, ω)−γ(t,x⁰, u, e, ω)|

≤L(e)|x−x⁰|. (2.3)

iii) Linear growth: The functions b, σ, g and γ satisfy the linear growth condition in the variables with the linear growth constant independent of the variables t, u, ω. Also there exists a non-negative function L⁰ ∈L²(ν) independent of t, π, ω, such that

|γ(t,x, π, e, ω)| ≤L⁰(e) (1 +|x|).

The optimal control associated to this problem is to optimize the objective function of the form

J(u) =E Z T

0

f(t,X(t),E[Φ(X(t))], Y(t), Z(t), K(t,·), π(t), ω)dt (2.4) +h₁(Y(0)) +h₂(X(T),E[ψ(X(T))])],

over the admissible controls, for functions

f : Ω×[0, T]×R^d×3×φ×U →R, Φ : Ω×[0, T]×R→R,

h₁ : Ω×R→R, h2 : Ω×R×R→R, ψ : Ω×R→R.

That is, to find an optimal control π^∗ ∈ A_G, such that J(π^∗) = sup

π∈A

J(π). (2.5)

For now, the functions f,Φ, ψ , h_i, i = 1,2 are assumed to satisfy the following assump- tions.

Assumption (II)

i) The functions f(t,·, ω),Φ(t,·, ω), ψ(·, ω) , h_i(t,·, ω), i= 1,2 are C¹ for each t and ω.

ii) Integrability condition E

_T R

0

{|f(t,X(t),E[Φ(X(t))], Y(t), Z(t), K(t,·), π(t))|

+

∂f

∂x_i (t,X(t),E[Φ(X(t))], Y(t), Z(t), K(t,·), π(t))

2) dt

#

<∞.

(2.6)

2.1 The Hamiltonian and adjoint equations

Assume that

Z

R0

( sup

(x,π)∈D

|γ(t,x, π, e, ω)r(e)|

+ sup

(x,π)∈D

|∇γ(t,x, π, e, ω)r(e)|

)

ν(de)<∞

(2.7)

for eacht∈[0, T] and every boundedD⊂R×U, P−a.s. This integrability condition ensures that whenever r∈φ,

∇ Z

R0

γ(t,x, π, e, ω)r(e)ν(de) = Z

R0

∇γ(t,x, π, e, ω)r(e)ν(de), (2.8) and similarly for K(t,·).

Example 2.2 We notice that if the linear growth condition

|γ(t,x, π, e, ω)|+|∇γ(t,x, π, e, ω)| ≤D⁰(e){1 +|x|+|π|}

holds for some D⁰ ∈L²(ν) independent of t, ω, then L²(ν)⊂φ.

Now we define the Hamiltonian associated to this problem, H : Ω×[0, T]×R^d×R× R×R×φ×U ×R×R×φ×R→R,by

H(t,x,m,y, z, k(·), π, p, q, r(·), λ)

=f(t,x, m, y, z, k, π) +b(t,x, π)p+σ(t,x, π)q +g(t,x,y, z, π)λ+γ(t,x,π, e)r(e)ν(de),

(2.9)

admits Fr´echet derivative with respect to k ∈ φ, ∇_kH as a random measure is absoutey continuous with respect toν, with radom-Nikodym derivative ^d∇_dν^kH.

The adjoint equations for allt ∈[0, T] are defined as follows:

the MF-ABSDE in the unknown processes p(t), q(t), r(t, e) : dp(t) = E[Υ(t)|F_t]dt+q(t)dB(t) +

Z

R0

r(t, e) ˜N(dt, de), (2.10) where

Υ(t) = −

d

X

i=1

Z 0

−δ

∂H

∂x_i

t−s, π

µ_i(ds)−E h∂H

∂m

t−s, πi

Φ⁰(X(t)), (2.11) and with, terminal condition

p(T) = aλ(T) + ^∂h_∂x²(X(T),E[ψ(X(T))])

+^∂h_∂n²(X(T),E[ψ(X(T))])ψ⁰(X(T)), a∈R⁰, n =E[ψ(x)], and the SDE in the unknown processλ(t) :

dλ(t) = ∂H

∂y (t)dt+∂H

∂z (t)dB(t) + Z

R0

d∇_kH

dν (t, e) ˜N(dt, de), (2.12) initial condition λ(0) =h⁰₁(Y(0)) for all t∈[0, T].

Throughout this work, it would be useful to introduce the simplified notation h₂(T) = h₂(X(T),E[ψ(X(T))]).

Example 2.3 Suppose d= 1, µ :=µ₁ :

1. If µ is the Dirac measure concentrated at 0, then Υ(t) :=−∂H

∂x(t, π)−E h∂H

∂m(t, π)i .

2. If µ is the Dirac measure concentrated at δ, then Υ(t) :=−∂H

∂x(t+δ, π)−E h∂H

∂m(t+δ, π)i .

3. If µ(ds) = e^λsds, then Υ(t) : =−

Z 0

−δ

∂H

∂x(t−s, π)e^λs(ds)−E h∂H

∂m(t, π)i

=− Z t

t−δ

∂H

∂x(−s, π)e^λ(s−t)(ds)−E h∂H

∂m(t, π)i .

2.2 A sufficient maximum principle

When the Hamiltonian H and the functions (h_i)_i=1,2 are concave, under certain other limi- tations, it is also possible to derive a sufficient maximum principle.

Theorem 2.1 Let πˆ ∈ A_G with corresponding state processes X,ˆ Y ,ˆ Z,ˆ K(·)ˆ and adjoint processes p,ˆ q,ˆ ˆr(·) and λ. Suppose the following holds:ˆ

1. (Concavity) The functions

R×R3(x, n)7−→h_i(x, n), i= 1,2 (2.13) and

R^d×3×φ×U 3(x, m, y, z, k, π)7−→H(t,·,p(t),ˆ q(t),ˆ r(t,ˆ ·),λ(t))ˆ (2.14) are concave P−a.s. for each t∈[0, T].

2. (Maximum principle) E

h

H

t,X(t),ˆ Mˆ(t),Yˆ(t),Z(t),ˆ K(t,ˆ ·),π(t),ˆ p(t),ˆ q(t),ˆ r(t,ˆ ·),λ(t)ˆ G_ti

= sup

v∈UE h

H

t,X(t),ˆ Mˆ(t),Yˆ(t),Z(t),ˆ K(t,ˆ ·), v,p(t),ˆ q(t),ˆ ˆr(t,·),λ(t)ˆ G_ti

, (2.15) P−a.s. for each t ∈[0, T].

Then πˆ is an optimal control for the problem (2.5).

Proof By considering a suitable increasing family of stopping times converging to T, we may assume that all the local martingales appearing in the proof below are martingales. See the proof of Theorem 2.1 in [12] for details.

Letπ be an arbitrary admissible control. Consider the difference J(ˆπ)−J(π)

=E Z T

0

n

f(t,X(t),ˆ E[Φ( ˆX(t))],Yˆ(t),Z(t),ˆ K(t,ˆ ·),π(t))ˆ

−f(t,X(t),E[Φ(X(t))], Y(t), Z(t), K(t,·), π(t))}dt +h₁( ˆY(0))−h₁(Y(0)) + ˆh₂(T)−h₂(T)i

=E Z T

0

4fˆ(t)dt+4ˆh₁( ˆY(0)) +4ˆh₂( ˆX(T))

. (2.16)

We will use the simplified notation for 4H(t),ˆ 4X(t)..etc. Sinceˆ H is concave, we have 4H(t)ˆ ≥

2

X

i=0

Z 0

−δ

4X(t) ∂H

∂xi

t−s, π

µ_i(ds) + ∂Hˆ

∂m(t)E[4Φ( ˆˆ X(t))] + ∂Hˆ

∂y (t)4Yˆ(t) + ∂Hˆ

∂z (t)4Z(t) +ˆ Z

R0

∇_kH(t, e)4ˆ K(t, e)ν(de) +ˆ ∂Hˆ

∂π(t)4ˆπ(t)

≥

2

X

i=0

Z 0

−δ

4X(t) ∂H

∂x_i

t−s, π

µ_i(ds) + ∂Hˆ

∂m(t)E

hΦˆ⁰( ˆX(t))4X(t)ˆ i +∂Hˆ

∂y(t)4Yˆ(t) + ∂Hˆ

∂z (t)4Z(t) +ˆ Z

R0

∇_kH(t, e)4ˆ K(t, e)ν(de) +ˆ ∂Hˆ

∂π(t)4ˆπ(t). (2.17) By the concavity of h_i(.)_i=1,2, we find

4hˆ₁(0)≥ˆh⁰₁( ˆY(0))4Yˆ(0) = ˆλ(0)4Yˆ(0), (2.18) and

4ˆh₂(T)≥ 4X(Tˆ ) ∂ˆh₂

∂x (T) + ∂ˆh₂

∂n(T) ˆψ⁰( ˆX(T))

!

. (2.19)

Apply Itˆo’s formula to ˆλ(0)4Yˆ(0), we get E[ˆλ(0)4Yˆ(0)] =E

hλ(Tˆ )4Yˆ(T)

− Z T

0

(

−λ(t)4ˆˆ g(t) +4Yˆ(t)∂Hˆ

∂y (t) +4Zˆ(t)∂Hˆ

∂z (t) + Z

R0

∇_kH(t, e)4ˆ Kˆ(t, e)ν(de)





 dt





=E

"

4X(Tˆ ) p(Tˆ )− ∂ˆh2

∂x (T)− ∂ˆh2

∂n(T) ˆψ⁰( ˆX(T))

!

+ Z T

0

(

λ(t)4ˆˆ g(t)− ∂Hˆ

∂y (t)4Yˆ(t)

−∂Hˆ

∂z (t)4Zˆ(t)− Z

R0

∇_kH(t, e)4ˆ K(t, e)ν(de)ˆ





 dt





=E Z T

0

n

p(t)4ˆ ˆb(t) +4X(t)ˆ E[ ˆΥ(t)|F_t]]+4ˆσ(t)ˆq(t) +R

R0

4ˆγ(t, e)ˆr(t, e)ν(de) + ˆλ(t)4ˆg(t)− 4X(Tˆ ) ∂ˆh₂

∂x (T)− ∂hˆ₂

∂n(T) ˆψ⁰( ˆX(T))

!

−∂Hˆ

∂y(t)4Yˆ(t)− ∂Hˆ

∂z (t)4Z(t)ˆ − Z

R0

∇_kH(t, e)4ˆ K(t, e)ν(de)ˆ





 dt





=E

"

Z T 0

(

4H(t)ˆ − 4fˆ(t) +4X(t) ˆˆ Υ(t)− 4X(Tˆ ) ∂hˆ2

∂x(T)−∂ˆh2

∂n(T)) ˆψ⁰( ˆX(T))

!

−∂Hˆ

∂y(t)4Yˆ(t)− ∂Hˆ

∂z (t)4Z(t)ˆ − Z

R0

∇_kH(t, e)4ˆ K(t, e)ν(de)ˆ





 dt



.

(2.20) By the definition of Υ (2.11) and Fubini’s theorem, we can show that

Z T 0

Z 0

−δ

4X(t) ∂H

∂x_i

t−s, π

µ_i(ds)dt

= Z T

0

Z 0

−δ

∂H

∂x_i

t, π

4X(t+δ)µ_i(ds)dt (2.21)

Let us perform the change of variabler =t−s in the dt-integral to observe that E

hZ T 0

Z 0

−δ

X(t) ∂H

∂xi

(t−s, π)

µi(ds)dt i

=E hZ 0

−δ

Z T 0

X(t)∂H

∂xi

(t−s, π)

dt µ_i(ds)i

=E hZ 0

−δ

Z T s

X(t)∂H

∂x_i(t−s, π)

dt µ_i(ds)i

=E hZ ₀

−δ

Z _T

0

X(r+s)∂H

∂x_i(r, π)

dt µ_i(ds)i

=E hZ T

0

Z 0

−δ

X(t+s) ∂H

∂x_i(t, π)

µi(ds)dt i

. Putting (2.21) in (2.20), and combining (2.17) with (2.16), we obtain

J(ˆπ)−J(π)≥E

"

Z T 0

∂Hˆ

∂π (t)4ˆπ(t)dt

#

≥0, by the maximum condition of H (2.15).

3 Infinite horizon optimal control problem

In this section, we extend the results obtained in the previous section to infinite horizon. So it can be seen as a generalization to mean-field delayed problems of Theorems 3.1 and 4.1 in Anderson and Djehiche [3] and Peng [14] respectively. By following the same steps in the previous section but now with infinite time horizon, we consider that the state equations have the forms







dX(t) =b(t,X(t),E[X(t)], π(t), ω)dt+σ(t,X(t),E[X(t)], π(t), ω)dB(t) +R

R0γ(t,X(t⁻),E[X(t⁻)], π(t⁻), e, ω) ˜N(dt, de), t ∈[0,∞), X(t) =x₀(t), t ∈[−δ,0],

(3.1)

and

dY(t) =−g(t,X(t),E[X(t)], Y(t),E[Y(t)], Z(t), K(t,·), π(t))dt+Z(t)dB(t) +R

R0K(t, e, ω) ˜N(dt, de), t∈[0,∞), (3.2) which can be interpreted as in Pardoux [13] for all finite T,

Y(t) =Y(T) +RT

t g(s,X(s),E[X(s)], Y(s),E[Y(s)], Z(s), K(s,·), π(s))ds +R

R0K(t, e, ω) ˜N(dt, de),0≤t≤T, (3.3)

where

X(t) := (X₁(t), X₂,(t), . . . , X_d(t)) :=Z 0

−δ

X(t+s)µ₁(ds), . . . , Z 0

−δ

X(t+s)µ_d(ds) ,

for a bounded Borel measures µ₁(ds), . . . , µ_d(ds). We remark ifX is a c`adl`ag process, then X is also c`adl`ag. We always assume that the coefficient functional γ is evaluated for the predictable (i.e. left continuous) versions of the c`adl`ag processes X, Y and π, and we will omit the minus from the notation.

b= b(ω, t,x,m, π) :Ω×[0,∞)×R^d×R×U −→R, σ =σ(ω, t,x,m, π) :Ω×[0,∞)×R^d×R×U −→R, γ =γ(ω, t,x,m, π, e) :Ω×[0,∞)×R^d×R×U ×R0 −→R,

g :=g(ω, t,x,m, y, n, z, k(·), π)

: Ω×[0,∞)×R^d×R×R×R×R×φ×U −→R. We assume that the coefficient functional satisfy the following assumptions:

Assumptions (III)

1. The functions b, σ, γ, g are C¹ (Fr´echet) with respect to all variables except t and ω and e.

2. The functions b, σ, γ, g are jointly measurable.

Let U be a covex subset ofR. The setU will be the set of all admissible control values.

The information available to the controller is given by a sub-filtration G={G_t}t∈[0,T] with F₀ ⊂G_t⊂F_t.

The set of admissible controls, that is, the set of controls that are available to the controller, is denoted byA_G. It will be a given subset of the c`adl`ag,U-valued andGt-adapted processes inL²(Ω×[0,∞)), such that there exist a unique c`adl`ag adapted processesX =X^π, Y =Y^π, progressively measurableZ =Z^π, and predictableK =K^π satisfying (3.1) and (3.2), and if they also satisfy

E hZ ∞

0

|X(s)|²ds i

+E

"

sup

t≥0

e^κt(Y(t))²+

∞

R

0

e^κt((Z(t))²+R

R0

(K(t, e))²ν(de))dt

#

<∞, (3.4) for some constant κ >0.

3.1

We want to maximize the performance functional J(π) =E

Z ∞

0

f(t,X(t),E[X(t)], Y(t),E[Y(t)], Z(t), K(t,·), π(t))dt+h(Y(0))

over the set A_G, for some functions

f =f(ω, t,x,m, y, n, z, k(·), π) : Ω×[0,∞)×R^d×R×R×R×R×φ×U →R, and

h:R→R. That is, we want to find π^∗ ∈ A_G such that

sup

π∈A_G

J(π) = J(π^∗). (3.5)

We assume that the functions f and h satisfy the following assumptions:

Assumptions (IV)

1. The functions f, hare C¹ (Fr´echet) with respect to all variables except t and ω.

2. The function f is predictable, and h isF_t-measurable for fixed x,m, y, n, z, k,π.

3.2

H : Ω×[0,∞)×R^d×R×R×R×R×φ×U×R×R×φ×R→R, by

H(t,x,m,y, n, z, k(·), π, p, q, r(·), λ)

=b(t,x,m, π)p+σ(t,x,m, π)q+ Z

R0

γ(t,x,m,π, e)r(e)ν(de) +g(t,x,m,y, n, z, k(·), π)λ+f(t,x,m, y, n, z, k(·), π).

(3.6)

Now, to each admissible control π, we can define the adjoint processes p, q, r and λ by the following system of MF-FBSDE:

Backward equation

dp(t) = −E[Υ(t)|F_t]dt+q(t)dB(t) + Z

R⁰

r(t, e) ˜N(dt, de), t≥0, (3.7)

where

Υ(t) =

d

X

i=1

Z 0

−δ

∂H

∂xi

t−s,X(t−s),E[X(t−s)], Y(t−s),E[Y(t−s)], Z(t−s), K(t−s), π(t−s), p(t−s), q(t−s), r(t−s)

µi(ds) +Pd

i=1

Z 0

−δ

E h∂H

∂mi

t−s,X(t−s),E[X(t−s)], Y(t−s),E[Y(t−s)], Z(t−s), K(t−s), π(t−s), p(t−s), q(t−s), r(t−s)i

µ_i(ds).

(3.8)

Forward equation dλ(t) = ∂H

∂y

t,X(t),E[X(t)], Y(t),E[Y(t)], Z(t), K(t), π(t), p(t), q(t), r(t), λ(t)

+E h∂H

∂n

t,X(t),E[X(t)], Y(t),E[Y(t)], Z(t), K(t), π(t), p(t), q(t), r(t), λ(t) i

dt +∂H

∂z

t,X(t),E[X(t)], Y(t),E[Y(t)], Z(t), K(t), π(t), p(t), q(t), r(t), λ(t) dB(t) +

Z

R0

d∇_kH dν

t,X(t),E[X(t)], Y(t),E[Y(t)], Z(t), K(t), π(t), p(t), q(t), r(t), λ(t)

(e) ˜N(dt, de), λ(0) =h⁰(Y(0)).

(3.9) Notice also, Υ may not be adapted toF_t, as Υ(t) is defined using values of H at time t−s, where s <0.

Given an admissible control π, suppose there exist progressively measurable processes p = p^π, q =q^π and λ=λ^π and r=r^π, satisfying (3.7)-(3.9) and such that

E



sup

t≥0

e^κt(p(t))²+

∞

Z

0

(

|λ(t)|²+e^κt((q(t))²+R

R0

(r(t, e))²ν(de)) )

dt



<∞, (3.10) for some constantκ >0. Then, we say thatp, q, r and λ are adjoint equations to the mean- field forward backward system (3.1)-(3.2).

3.3

When Adjoint processes exist, we will frequently use the following short hand notation:

H(t, π) :=H t,X^π(t),E[X^π(t)], Y^π(t),E[Y^π(t)], Z^π(t), K^π(t), π(t), p^π(t), q^π(t), r^π(t)

. (3.11) Similar notation will be used for the coefficient functions b, σ, γ, and g, and the functions f, h from the performance functional, and for derivatives of the mentioned functions. We will write ∇H for the Fr´echet derivative ofH with respect to the variables x,n, y, n, z, k(·).

is given by

∇H(t, π)(¯x,n,¯ y,¯ n,¯ z,¯ ¯k(·),π) =¯ ∇_xH(t, π)¯x^T+∇_mH(t, π) ¯m^T + ∂H

∂y(t, π)¯y+∂H

∂y(t, π)¯n+∂H

∂z (t, π)¯z Z

R0

∇_kH(t, π)(e)¯k(e)ν(de) +∂H

∂π(t, π)¯π, where ∇x is the gradient (as a row vector) with respect to the variablex, etc.

Using this notation, the state equations and the adjoint equations can be written more compactly as

dX(t) = b(t, π)dt+σ(t, π)dB(t) + Z

R0

γ(t, π, e) ˜N(dt, de), t∈[0,∞), X(t) = x₀(t), t∈[−δ,0],

dY(t) = −g(t, π)dt+Z(t)dB(t) + Z

R0

K(t, e, ω) ˜N(dt, de), t∈[0,∞), and

dp(t) =−E[Υ(t)|F_t] +q(t)dB(t) + Z

R0

r(e, t) ˜N(dt, de), where

Υ(t) =

d

X

i=1

Z 0

−δ

∂H

∂x_i(t−s, π) +E h∂H

∂m(t−s, π) i

µi(ds) and

dλ(t) = n∂H

∂y(t, π) +E h∂H

∂n(t, π)io dt +∂H

∂z (t, π)dB(t) + Z

R0

d∇_kH

dν (t, π)(e) ˜N(dt, de), t∈[0,∞) λ(0) =h⁰(Y(0)).

Example 3.1 Suppose d= 1, µ:=µ₁:

1. If µ is the Dirac measure concentrated at 0, then Υ(t) := ∂H

∂x(t, π) +E h∂H

∂m(t, π)i .

2. If µ is the Dirac measure concentrated at −δ, then Υ(t) := ∂H

∂x(t+δ, π) +E h∂H

∂m(t+δ, π)i .

3. If µ(ds) = e^λsds, then Υ(t) : =

Z 0

−δ

∂H

∂x(t−s, π) +E h∂H

∂m(t−s, π)i

e^λs(ds)

= Z t

t−δ

∂H

∂x(−s, π) +E h∂H

∂m(−s, π)i

e^λ(s−t)(ds).

4

Suppose that a control π ∈ A_G is optimal and that η∈ A_G. If the functionα7−→J(π+αη) is well defined and differentiable on a neighbourhood of 0, then

d

dαJ(π+αη)|_α=0= 0. (4.1)

Under a set of suitable assumptions on the functions f, b, σ, g, h,γ and K, we will show that for every admissible π, and bounded admissibleη,

d

dαJ(π+αη)|_α=0=E Z ∞

0

∂

∂πH(t, π)η(t)dt

. (4.2)

Then, provided that the set of admissible controls A_G is sufficiently large, d

dαJ(π+αη)|_α=0= 0 (4.3)

is equivalent to E

∂

∂πH(t, π)|G_t

= 0 P −a.s. for each t∈[0,∞). (4.4) Consequently,

E ∂

∂πH(t, π)|G_t

= 0 P −a.s. for each t∈[0,∞), (4.5) is a necessary condition for optimality of π.

The first step of deriving a necessary maximum principle is to establish the following equal-

d

dαJ(π+αη)|_α=0

=E Z ∞

0

∇f(t,X^π(t), π)·(κ^π(t), η(t))^Tdt+h⁰(Y^π(0))· Y ^π,η(0)

=E hZ ∞

0

∂H

∂π(t, π)η(t)dt i

.

We will formalize this through Lemma 4.3 and Lemma 4.6, but first we need to impose a set of assumptions:

Assumptions (V)

i) Assumptions on the coefficient functions

• The functions∇b,∇σ and ∇g are bounded. The upper bound is denoted byD₀. Also, there exists a non-negative functionD∈L² such that

|∇γ(t,x, π, e)|+|K(t, e)| ≤D(e) (4.6)

• The functions ∇b,∇σ and ∇g are Lipschitz continuous in the variables x,m, π, uniformly int, ω,with the Lipschitz constantL₀ >0. Also, there exits a function L∈L²(ν) independent of t,ω, such that

|γ(t,x, π, e, ω)−γ(t,x⁰, π⁰, e)|

≤L(e)

|x−x⁰|+|π−π⁰|

. (4.7)

• The functionL⁰ from Assumption (I) is also in L²(ν).

ii) Assumptions on the performance functional

• The functions∇f,∇hand ∇g are bounded. The upper bound is still denoted by D₀.

• The functions∇f,∇hand∇gare Lipschitz continuous in the variables (x,y, z, k, π), uniformly int, w. The Lipschitz constant is still denoted byL₀.

iii) Assumptions on the set of admissible processes

• Wheneverπ∈ A_G and η∈ A_G is bounded, there exists >0 such that

π+αη ∈ A_G for each α∈(−, ). (4.8)

• For eacht₀ >0 and each boundedG_t0-measurable random variablesµ, the process η(t) =µ1_[t0,t0+h)(t) belongs to A_G.

4.1

Suppose that π, η ∈ A_G, with η bounded. Consider the equations dX(t) =∇b(t, π)·(X(t),E[X(t)], η(t))^Tdt

+∇σ(t, π)·(X(t),E[X(t)], η(t))^TdB(t) +

Z

R0

∇γ(t, π, e)·(X(t),E[X(t)], η(t))^TN˜(dt, de), t∈[0,∞), X(t) = 0, for t∈[−δ,0],

(4.9)

where

X(t) :=

Z 0

−δ

X(t+s)µ₁(ds), . . . , Z 0

−δ

X(t+s)µ_d(ds) , and

dY(t) =

− ∇g(t, π)

·

X(t),E[X(t)],Y(t),E[Y(t)],Z(t),K(t), η(t)T

dt +Z(t)dB(t) +

Z

R0

K(t, e) ˜N(dt, de), t∈[0,∞).

(4.10)

We say that a solutions X = X^π,η,Y = Y^π,η,Z = Z^π,η and K = K^π,η associated with the controls π, η exist if there are processesX,Y,Z and K satisfying (4.9)-(4.10), and

E

"

sup

t≥0

e^κt(Y(t))²+

∞

R

0

|X(t)|² +e^κt((Z(t))²+R

R0

(K(t, e))²ν(de))dt

#

<∞, (4.11) for some constant κ >0.

4.1.1 Differentiability of the forward state process

The proofs in this section are similar to e.g. the proofs of Lemmas 3.1, 4.1 and in Anderson and Djehiche [3] and Peng [14] respectively. However because of our jump term, we need to use Kunita’s inequality instead of Burkholder-Davis-Gundy’s inequality. We also do not require anyL⁴-boundedness and convergence of any of our processes as done e.g. in Anderson and Djehiche [3], to assure the convergence in our Lemma 4.3. For convenience to the reader, let us recall

Lemma 4.1 (Kunita’s inequality, corollary 2.12 in [8]) Suppose ρ≥2 and Z t Z t Z tZ

Then there exists a positive constant C_ρ,T, (depending only on ρ, T) such that the following inequality holds

E

sup

0≤s≤T

|X(s)|^ρ

≤C_ρ,T

|x|^ρ+ Z t

0

E[|b(r)|^ρ] +E[|σ(r)|^ρ] +E

Z

R0

|X(r, e)|^ρν(de)

+E

"

Z

R0

|X(r, e)|²ν(de) ^ρ₂#)

dr

# .

Now, define the random fields

F_α^η(t) := X^π+αη(t)−X^π(t), and

F^η_α(t) :=X^π+αη(t)−X^π(t) = Z 0

−δ

F_α^η(t+r)µ₁(dr), . . . , Z 0

−δ

F_α^η(t+r)µ_d(dr) .

Lemma 4.2 Let T ∈ (0,∞). There exists a constant C = C_T > 0, independent of π, η, such that

E[ sup

0≤v≤t

|F^π,η_α (v)|²]≤C kηk²_L2(Ω×[0,T]) α², (4.12) whenever t ≤T.

Moreover, there exists a measurable version of the map (t, α, ω)7→F_α(t, ω),

such that for a.e. ω, Fα(t)→0 as α→0 for every t ∈[0,∞).

Proof The proof follows that of Lemma 4.2 in Dahl et al [5] closely. Define β_α(t) := E[ sup

−δ≤v≤t

|F_α^η(v)|²].

Observe that using Jensen’s inequality, we find that E[ sup

0≤v≤t

|F^η_α(v)|²] =E h

sup

0≤v≤t d

X

i=1

Z 0

−δ

F_α^η(v+r)µ_i(dr)

2i

≤E h

sup

0≤v≤t d

X

i=1

|µ_i| Z 0

−δ

|F_α^η(v+r)|²µ_i(dr)i

≤E h

sup

0≤v≤t d

X

i=1

|µ|² sup

−δ≤r≤0

|F_α^η(v+r)|²i

≤ |µ|²β_α(t)

where |µ| := Pd

i=1|µ_i| := Pd

i=1µ_i[−δ,0]. Since ∇b,∇σ,∇γ are bounded, b, σ and γ are Lipschitz in the variable x,m, π. Now using the integral representation of X^π+αη and X^π, Kunita’s inequality, and finally the Lipschitz conditions on b, σ and γ, we find that

β_α(t)≤C_2,TE hZ t

0

|b(s, π+αη)−b(s, π)|²+|σ(s, π+αη)−σ(s, π)|² +

Z

R0

|γ(s, π+αη, e)−γ(s, π, e)|²ν(de)dsi

≤C_2,T(D₀²+kDk²_L2(ν))E hZ t

0

(|F_α(s)|²+α²|η(s)|²)dsi

≤C_2,T(D₀²+kDk²_L2(ν))Z t 0

β_α(s)ds+α² kηk²_L2(Ω×[0,T])

.

Now (4.12) holds by Gronwall’s inequality. The second part of the lemma follows by the same argument as in Dahl et al [5].

Now, fix π and define

A^η_α(t) := X^π+αη(t)−X^π(t)

α − X^π,η(t) A^η_α(t) := X^π(t)−X^π+αη(t)

α − X^π,η(t)

=Z 0

−δ

A^η_α(t+r)µ₁(dr), . . . , Z 0

−δ

A^η_α(t+r)µ_d(dr) ,

for each η.

Lemma 4.3 For each t ∈(0,∞), it holds that θ_α(t) :=E[ sup

0≤v≤t

|A^η_α(v)|²]→0, E[ sup

0≤v≤t

|A^η_α(v)|²]→0, as α →0.

Proof Similarly as in the previous proof, we find that E[ sup

0≤v≤t

|A^η_α(v)|²]≤N|µ|²θ_s(t).

The rest of the proof follows the exact same steps as the proof of Lemma 4.3 in Dahl et al [5] and is therefore omitted.

4.1.2 Differentiability of the backward state process

We will assume that the following convergence results hold for all t≥0 E

"

sup

0≤r≤t

Y^π+αη(r)−Y^π(r)

α − Y(r)

2#

→0, (4.13)

E

"

Z T 0

Z^π+αη(t)−Z^π(t)

α − Z(t)

2

dt

#

→0, (4.14)

and

E

"

Z T 0

Z

R0

K^π+αη(t, e)−K^π(t, e)

α − K(t, e)

2

ν(de)dt

#

→0, (4.15)

as α→0.In particular, Y, Z and Kare the solutions of (4.10).

4.2

Lemma 4.4 (Differentiability of J) Suppose π, η ∈ A_G with η bounded. Suppose there exists an interval I ⊂ R with 0 ∈ I, such that the perturbation π +αη is in A_G for each α∈I. Then the functionα 7→ d

dαJ(π+αη) has a (possibly one-sided) derivative at 0 with d

dαJ(π+αη)|_α=0

=E Z ∞

0

∇f(t,X^π(t), π)·(X^π(t), η(t))^Tdt+h⁰(Y^π(0))· Y ^π,η(0)

. Proof Recall that

J(π) =E Z ∞

0

f(t,X^π(t), π)dt+h(Y^π(0))

. Let

J_n(π) =E Z n

0

f(t,X^π(t), π)dt+h(Y^π(0))

. We show that

d

dsJ_n(π+sη)|_s=0

=E Z n

0

∇f(t,X^π(t), π)·(X^π(t), η(t))^Tdt+h⁰(Y^π(0))· Y ^π,η(0)

.

Since∇f andηare bounded andX is finite, we can use the dominated convergence theorem to get

d

dsJ(π+sη)|_s=0= lim

n→∞

d

dsJ_n(π+sη)|_s=0 . Now proving that

d dsE

h(Y^π+sη(0))

|_s=0=E[h⁰(Y^π(0))· Y ^π,η(0)].

Then E

1

sh(Y^π+sη(0))−h(Y^π(0))−h⁰(Y^π(0))· Y ^π,η(0)

= E

1 s

Z ₁

0

h⁰(Y^π(0) +λ(Y^π+sη(0))·(Y^π+sη(0)−Y^π(0))dλ

−h⁰(Y^π(0))· Y ^π,η(0)

= E

Z 1 0

h⁰(Y^π(0) +λ(Y^π+sη(0)−Y^π(0))·

Y^π+sη(0)−Y^π(0)

s − Y ^π,η(0)

+

h⁰(Y^π(0) +λ(Y^π+sη(0)−Y^π(0))−h⁰(Y^π(0))· Y ^π,η(0) dλ

≤ Z 1

0

DE

Y^π+sη(0)−Y^π(0)

s − Y ^π,η(0)

dλ +

Z 1 0

LE

λ(Y^π+sη(0)−Y^π(0))

· |Y ^π,η(0)|

dλ

→0,

ass→0, where the last estimation is obtained by assumption ofY (4.13) and by applying Cauchy-Schwartz inequality, we obtain

E

Y^π+sη(0)−Y^π(0)

· |Y ^π,η(0)|

≤E h

Y^π+sη(0)−Y^π(0)

2i¹₂ E

|Y ^π,η(0)|²¹₂

→0, ass→0.

Lemma 4.5 [Differentiability of J in terms of the Hamiltonian] Let π, η ∈ A_G withη bounded. LetX, Y, Z, K, p, q, r, λ be the state and corresponding toπ adjoint processes , and X,Y,Z,K, be derivative processes corresponding to π, η. Suppose there exists adjoint processes p, q, r corresponding to π and that

lim

T→∞E[ p(T)X(T)] = 0, (4.16)

lim

T→∞E[λ(T)Y(T)] = 0. (4.17)

Then

d

dαJ(π+αη)|_α=0=E Z ∞

0

∂H

∂π (t, π)η(t)dt

.

Proof For fixedT ≥0, define a sequence of stopping times, as follows τn(·) :=T ∧inf

t≥0 :

Z t 0

p(t)∇σ(t, π)·

X(t),E[X(t)], η(t) T

+X(t)q(t) 2

ds +

Z t 0

Z

R0

r(s, e)∇γ(s, π, e)·(X(s),E[X(s)], η(s))^T +p(s)∇γ(s, π, e)·(X(s),E[X(s)], η(s))^T+X(s)r(s, π, e)

2

ν(de)ds +

Z t 0

Y(s)∂H

∂z (s, π) +λ(s)Z(s)2

ds +

Z t 0

Z

R0

Y(s)∇_kH(s, e) +λ(s)K(s, e) +K(s, e)∇_kH(s, e) 2

ν(de)ds≥n

, n∈N, now it clearly holds that τn → T P-a.s. Observe that, with a slight abuse of notation (in particular, we write ∇b etc. both when we consider it as a Fr´echet derivative with respect the spacial variables from (3.1) and when considering it as the gradient with respect to all the spacial variables of the Hamiltonian, H), it holds that

∇H−λ∇g− ∇f =p∇b+q∇σ+ Z

R0

r∇γ dν. (4.18)

(This can be shown using the Chain rule for the Fr´echet derivative). By Itˆo’s formula, we can compute that

p(τ_n)X(τ_n) = Z τn

0

p(t)∇b(t, π) +q(t)∇σ(t, π) + Z

R0

r(t, e)∇γ(t, π, e)ν(de)

·

X(t),E[X(t)], η(t)T

− X(t)E[Υ(t)|F_t]

dt +

Z τn

0

p(t)∇σ(t, π)·

X(t),E[X(t)], η(t) T

+X(t)q(t)

dB(t) +

Z τn

0

Z

R⁰

n

r(t, e)∇γ(t, π, e)·(X(t),E[X(t)], η(t))^T +p(t)∇γ(t, π, e)·(X(t),E[X(t)], η(t))^T+X(t)r(t, π, e)o

N˜(dt, de).

(4.19)

The stochastic integral parts have zero mean by definition of the stopping time, and we recall thatX(0) = 0. Observe that since we have required that all solutions of the state and adjoint equations belongs to the spacesL²(Ω×[0,∞)) orL²(Ω×[0,∞)×R0), and that the gradient of the coefficient functionals are bounded, it holds that

E hZ ∞

0

p(t)∇b(t, π) +q(t)∇σ(t, π) + Z

R0

r(t, e)∇γ(t, π, e)ν(de)

·

X(t),E[X(t)], η(t)T

+|X(t)Υ(t)|dti

<∞.

(4.20)

Now, for all finite T, we have E[p(T)X(T)] = lim

n→∞E[p(τn)X(τn)]

=E hZ τn

0

p(t)∇b(t, π) +q(t)∇σ(t, π) + Z

R0

r(t, e)∇γ(t, π, e)ν(de)

·

X(t),E[X(t)], η(t) T

− X(t)E[Υ(t)|F_t]dt i

= lim

n→∞E hZ _τn

0

∇H(t, π)−λ(t)∇g(t, π)− ∇f(t, π)

·

X(t),E[X(t)],Y(t),E[Y(t)],Z(t),K(t), η(t)T

− X(t)Υ(t)dt i

=E hZ τn

0

∇H(t, π)−λ(t)∇g(t, π)− ∇f(t, π)

·

X(t),E[X(t)],Y(t),E[Y(t)],Z(t),K(t), η(t)T

− X(t)Υ(t)dti .

In the first and last equalities, we have used Lebesgue’s dominated convergence theorem and that the integrand is dominated by the integrable random variable in (4.20). In the second equality, we have used the integral representation (4.19) of p(τ_n)X(τ_n) and that the stochastic integrals have zero mean by definition of the stopping times τn, and in the third equality, we have used (4.18). From the assumption (4.17), and again using the fact that the integrands are dominated by the integrable random variable in (4.20), we find that

0 = lim

T→∞E[p(T)X(T)] =E hZ ∞

0

∇H(t, π)−λ(t)∇g(t, π)− ∇f(t, π)

·

X(t),E[X(t)],Y(t),E[Y(t)],Z(t),K(t), η(t)T

− X(t)Υ(t)dt i

.

(4.21)

Similarly, using Itˆo’s formula, we compute that λ(τn)Y(τn)−λ(0)Y(0) =

Z τn

0

n Y(t)

∂H

∂y(t, π) +E h∂H

∂n(t, π) i

+λ(t)

− ∇g(t, π)

·

X(t),E[X(t)],Y(t),E[Y(t)],Z(t),K(t), η(t)T

+Z(t)∂H

∂z (t, π) + Z

R0

K(t, e)∇_kH(t, e)ν(de) o

dt +

Z τn

0

Y(t)∂H

∂z (t, π) +λ(t)Z(t)

dB(t) +

Z τn

0

Z

R0

{Y(t)∇_kH(t, e) +λ(t)K(t, e) +K(t, e)∇_kH(t, e)}N˜(dt, de).

We recall that λ(0) =h⁰(Y(0)). Then proceeding as above, we find that 0 = lim

T→∞E h

λ(T)Y(T)i

=E h

h⁰(Y(0))Y(0)i

+EhZ ∞ 0

nY(t) ∂H

∂y(t, π) +E h∂H

∂n(t, π)i +λ(t)

− ∇g(t, π)

·

X(t),E[X(t)],Y(t),E[Y(t)],Z(t),K(t), η(t)T

+Z(t)∂H

∂z (t, π) + Z

R0

K(t, e)∇_kH(t, e)ν(de)o dti

.

(4.22)

Now, combining Lemma 4.4 with the equations (4.21) and (4.22) yields d

dαE h

J(π+αη)i

α=0 =E h

h⁰(Y(0))Y(0) +

Z ∞

0

∇f(t, π)·

X(t),E[X(t)],Y(t),E[Y(t)],Z(t),K(t,·), η(t)T

dti

=E hZ ∞

0

λ(t)

∇g(t, π)

·

X(t),E[X(t)],Y(t),E[Y(t)],Z(t),K(t), η(t)T

−n Y(t)

∂H

∂y(t, π) +E h∂H

∂n(t, π) i

+Z(t)∂H

∂z (t, π) + Z

R0

∇_kH(t, π)K(t, e)ν(de) o

dt i

+E hZ ∞

0

∇H(t, π)−λ(t)∇g(t, π)

·

X(t),E[X(t)],Y(t),E[Y(t)],Z(t),K(t), η(t) T

− X(t)Υ(t)dt i

=E hZ ∞

0

∇H(t, π)·

X(t),E[X(t)],Y(t),E[Y(t)],Z(t),K(t), η(t) T

−nX^d

i=1

Z 0

−δ

X(t)∂H

∂xi

(t−s, π) +E h∂H

∂mi

(t−s, π)i µ_i(ds) +Y(t)

∂H

∂y(t, π) +E h∂H

∂n(t, π)i

+Z(t)∂H

∂z (t, π) + Z

R0

K(t, e)∇_kH(t, e)ν(de)o dti

=E hX^d

i=1

n∂H

∂x_i(t, π)· Z 0

−δ

X(t+s)µ_i(ds) + ∂H

∂m_i(t, π)·E Z 0

−δ

X(t+s)µ_i(ds)o

(4.23) +∂H

∂y(t, π)Y(t) +∂H

∂nE[Y(t)] + ∂H

∂z (t, π)Z(t) + Z

R0

∇_kH(t, π)(e)K(t, e)ν(de) (4.24) +∂H

∂π(t, π)η(t)

−nX^d

i=1

Z 0

−δ

X(t)∂H

∂xi

(t−s, π) +E h∂H

∂mi

(t−s, π)i

µ_i(ds) (4.25)

+Y(t) ∂H

∂y(t, π) +E h∂H

∂n(t, π)i

+Z(t)∂H

∂z (t, π) + Z

R0

∇_kH(t, e)K(t, e)ν(de)o dti

(4.26)

=E hZ ∞

0

∂H

∂π(t, π)η(t)dti ,

which is what we wanted to prove. In order to see why the last equality holds, observe that one may use Fubini’s theorem to show that the sum of the lines (4.24) and (4.26) is 0. Also, we have that the sum of the lines (4.23) and (4.25) is 0. To see why the latter holds, recall that X(r) = 0, when r <0, and perform the change of variable r =t−s in the dt-integral

to observe that E

hZ ∞

0

Z 0

−δ

X(t) ∂H

∂xi

(t−s, π) +E h∂H

∂mi

(t−s, π) i

µi(ds)dt i

=E hZ ₀

−δ

Z ∞

0

X(t)∂H

∂x_i(t−s, π) +E h∂H

∂m_i(t−s, π)i

dt µ_i(ds)i

=E hZ 0

−δ

Z ∞

s

X(t)∂H

∂xi

(t−s, π) +E h∂H

∂mi

(t−s, π)i

dt µ_i(ds)i

=E hZ 0

−δ

Z ∞

0

X(r+s) ∂H

∂xi

(r, π) +E h∂H

∂mi

(r, π) i

dt µi(ds) i

=E hZ ∞

0

Z 0

−δ

X(t+s)∂H

∂x_i(t, π) +E h∂H

∂m_i(t, π)i

µi(ds)dti

=E hZ ∞

0

Z 0

−δ

X(t+s)∂H

∂xi

(t, π) +E h∂H

∂mi

(t, π)i

µ_i(ds)dti .

Theorem 4.6 (Necessary maximum principle) Under the assumptions of Lemma 4.5, we can prove the equivalence between:

(i) For each bounded η ∈ A_G, 0 = d

dαJ(π+αη)|_α=0=E Z ∞

0

∂H

∂π (t, π)η(t)dt

.

(ii) For each t ∈[0,∞), E

∂H

∂π (t, π) G_t

π=π^∗(t)

= 0, a.s.

Proof Using Lemma 4.4, the proof is similar to the proof of Theorem 4.1 in Agram and Øksendal [2].

4.3

Theorem 4.7 (Sufficient maximum principle) Let π ∈ A_G with corresponding solu- tions X(t), Y(t), Z(t), K(t,·), p(t), q(t), r(t,·), λ(t). Assume the following conditions hold:

(i)

E h

H(t, π) G_ti

= sup

v∈UE h

H(t, v) G_ti

,

for all t ∈[0,∞) a.s.

(ii) Transversality conditions

Tlim→∞E h

ˆ p(T)

X(Tˆ )−X(T)i

≤0,

Tlim→∞E

hλ(Tˆ )

Yˆ(T)−Y(T) i

≥0.

Then πˆ is an optimal control for the problem (3.5).

Proof The proof is similar to the proof of Theorem 2.5 but with infinite time horizon and Theorem 3.1 in Agram and Øksendal [2].

5 Optimal consumption with respect to mean-field re- cursive utility

Suppose now that the state equation is a cash flow of the form







dX(t) = [b₀(t,E[X(t)])−π(t)]dt+σ₀(t,X(t),E[X(t)], π(t))dB(t) +R

R0γ₀(t,X(t),E[X(t)], π(t), e) ˜N(dt, de), t∈[0,∞), X(t) =X₀(t), t ∈[−δ,0],

(5.1)

where the control π(t) ≥ 0 represents a consumption rate. The function b₀ is assumed to be deterministic, in addition to the assumptions from the previous sections. We want to consider an optimal recursive utility problem similar to the one in Agram and Øksendal [2]. See also Duffie and Epstein [6]. For notational convenience assume that µ₀ is the Dirac measure concentrated at 0.

Define the recursive utility processY(t) = Y^π(t),by the MF-BSDE in the unknown processes (Y, Z, K) = (Y^π, Z^π, K^π), by

dY(t) =−g₀(t,X(t),E[X(t)], Y(t),E[Y(t)], π(t), ω)dt+Z(t)dB(t) +R

R0K(t, e, ω) ˜N(dt, de), t ∈[0,∞). (5.2)

We assume that equations (5.1) and (5.2) satisfy (3.4) and for all finiteT, the equation (5.2) is equivalent to

Y(t) =E

Y(T) + Z T

t

g0(s,X(s),E[X(s)], Y(s),E[Y(s)], π(s))ds

Ft

;t≤T ≤ ∞.

Notice that the functionb₀ from the state equation (5.1) depends only on E[X(t)] and on the control π(t), and that the driver g₀is independent ofZ. We have put no further restrictions on the coefficient functionals so far. Let f = 0, h= 0 and h1(y) =y, in particular, we want

The admissible controls are assumed to be the c`adl`ag, Gt-adapted non-negative processes in L²(Ω×[0, T]).

The Hamiltonian for the mean-field forward-backward system is H(t, π) = b₀(t,E[x])−π

p+σ₀(t, π)q +

Z

R0

γ₀(t, π, e)r(t, e)ν(de) +g₀(t, π)λ. (5.3) The adjoint processes (p, q, r) = (p^π, q^π, r^π) and λ=λ^π corresponding to π are defined by

dp(t) =−E[Υ(t)|Ft]dt+q(t)dB(t) + Z

R0

r(t, e) ˜N(dt, de), (5.4) with

Υ(t) = _∂x^∂

0b0(t) +E h ∂

∂m0b0(t) i

+Pd i=1

R0

−δ

n∂H

∂xi

t−s, π +E

h∂H

∂mi

t−s, πio µ_i(ds) and

(

dλ(t) =λ(t)

∂

∂yg₀(t, π) +E h ∂

∂ng₀(t, π)i

dt, t∈[0,∞),

λ(0) = h⁰₁(Y(0)) = 1. (5.5)

We assume that the equations (5.4) and (5.5) satisfy the decay condition (3.10).We have

∂

∂πH t, π

=−p(t) + ∂

∂π

σ0(t, π(t))q(t) +

Z

R0

γ₀(t, π(t), e)r(t, e)ν(de) +g₀(t, π(t))λ(t)

. (5.6)

Now, applying the necessary maximum principle to the expression above yields the following:

Theorem 5.1 Suppose that π(t)ˆ is an optimal control. Then E[ˆp(t)|G_t] =E

h ∂

∂π

σ0(t,π(t))ˆˆ q(t) + Z

R0

γ0(t,π(t), e)ˆˆ r(t, e)ν(de) +g0(t,π(t))ˆˆ λ(t)

G_t i

. (5.7) We see that if we can put additional conditions on the mean-field forward-backward system, such that ˆq = 0,ˆr= 0,λˆ is deterministic with ˆλ >0 and that G_t=F_t, then (5.7) reduces to

ˆ p(t) λ(t)ˆ = ∂

∂πg₀(t,π).ˆ (5.8)

Example 5.1 Suppose that the following condition holds:

g₀is independent of xand m, for example let us take

g₀(t, π) :=−αY(t) +βE[Y(t)]−lnπ. (5.9) Then (ˆp,0,0) where ˆp solves the deterministic equation

ˆ

p(t) = ˆp(T)− Z T

t

∂

∂mb₀(s, m)p(s)ds,

solves (5.4), for all finite T. Combining (5.9) and (5.8),we deduce that ˆ

π(t) = −ˆλ(t) ˆ p(t) ,

with ∂

∂πg₀(t, π) = −1

π(t). (5.10)

Consequently

λ(t) =e^{−(α−β)t}, for all t ∈[0,∞).

Combining (5.10) and (5.8), if π is bounded away from 0 we have ˆ

p(T) = e^{−(α−β)T} ˆ

π(T) →

T→∞ 0, if β < α. (5.11)

Putπ = 0 in equation (5.1),integrating and taking expectation, we obtain h(t) :=E[X(t)] =x+

Z t 0

b₀(s,E[X(s)])ds.

First we assume that b₀ has at most linear growth, in the sense that there exists a constant csuch that b₀(t, x)≤cx. Then we get

h(t)≤x+c Z t

0

h(s)ds, and hence by the Gronwall inequality it follows that

h(t)≤xe^ct, for all t. (5.12)

For a given consumption rateπ, letX^π(t) be the corresponding solution of (5.1).Then since π(t) ≥ 0 for all t, we always have X^π(t) ≤ X⁰(t). Therefore, to prove the transversality condition it suffices to prove that E[ˆp(T)X⁰(T)] goes to 0 as T goes to infinity.

Let us compare (5.12) with the decay of ˆp(T) in (5.11) we get E

ˆ

p(T)X⁰(T)

=E[ˆp(T)X(T)]

= ˆp(T)E[X(T)]