Optimal stopping and stochastic control differential games for jump diffusions

OPTIMAL STOPPING AND STOCHASTIC CONTROL DIFFERENTIAL GAMES FOR JUMP DIFFUSIONS

5 December 2011

Fouzia Baghery¹, Sven Haadem², Bernt Øksendal^3,4, Isabelle Turpin¹ Abstract

We study stochastic differential games of jump diffusions driven by Brownian motions and compensated Poisson random measures, where one of the players can choose the stochastic con- trol and the other player can decide when to stop the system. We prove a verification theorem for such games in terms of a Hamilton-Jacobi-Bellman variational inequality (HJBVI). We also prove that the value function of the game is a viscosity solution of this associated HJBVI.

The results are applied to study some specific examples, including optimal resource extraction in a worst case scenario, and risk minimizing optimal portfolio and stopping.

1 Introduction

LetX(t) =X(t, ω)∈[0,∞)×Ωbe a stochastic process on a filtered probability space(Ω,F,(F_t)t≥0,P) representing the wealth of an investment at timet. The owner of the investment wants to find the op- timal time for selling the investment. If we interpret “optimal” in the sense of “risk minimal”, then the problem is to find a stopping time τ = τ(ω) which minimizesρ(X(τ)), whereρ denotes arisk measure. If the risk measureρis chosen to be a convex risk measure in the sense of [10] and (or) [9], then it can be given the representation

ρ(X) = sup

Q∈N

{EQ[−X]−ζ(Q)}, (1)

1Laboratoire LAMAV, Universit de Valenciennes, 59313 Valenciennes, France.

Emails: fouzia.baghery@univ-valenciennes.fr Isabelle.Turpin@univ-valenciennes.fr

2Center of Mathematics for Applications (CMA), University of Oslo, Box 1053 Blindern, N-0316 Oslo, Norway.

Email: svenhaa@math.uio.no

3Center of Mathematics for Applications (CMA), University of Oslo, Box 1053 Blindern, N-0316 Oslo, Nor- way. The research leading to these results has received funding from the European Research Council under the Euro- pean Community’s Seventh Framework Programme (FP7/2007-2013) / ERC grant agreement no [228087] Email: ok- sendal@math.uio.no

4Norwegian School of Economics and Business Administration (NHH), Helleveien 30, N-5045 Bergen, Norway.

for some setN of probability measuresQPand some convex “penalty” functionζ :N →R. Using this representation the optimal stopping problem above gets the form

τ∈Tinf

sup

Q∈N

{EQ[−X(τ)]−ζ(Q)}

(2) where T is a given family of admissible F_t- stopping times. This may be regarded as an optimal stopping-stochastic control differential game.

In this paper we study this problem in a jump diffusion context. In Section 2 we formulate a general optimal stopping-stochastic control differential game problem in this context and we prove a general verification theorem for such games in terms of variational inequality-Hamilton-Jacobi- Bellman (VIHJB) equations. Then in Section 3 we apply the general results obtained in Section 2 to study the problem (2). By parametrizing the measures Q ∈ N by a stochastic processθ(t, z) = (θ₀(t), θ₁(t, z)) we may regard (2) as a special case of the general stochastic differential game in Section 2. We use this to solve the problem in some special cases.

2 General formulation

In this section we put the problem in the introduction into a general framework of optimal stopping and stochastic control differential game for jump diffusions and we prove a verification theorem for the value function of such a game. We refer to [16] for information about optimal stopping and stochastic control for jump diffusions. The following presentation follows [15] closely.

Suppose the stateY(t) =Y^u(t) = Y_t^y,uat timetis given as the solution of a stochastic differential equation of the form











dY(t) = b(Y(t), u₀(t))dt+σ(Y(t), u₀(t))dB(t) +R

R^k0γ(Y(t⁻), u₁(t, z), z) ˜N(dt, dz);

Y(0) =y∈R^k.

(3)

Hereb :R^k×K → R^k, σ :R^k×K → R^k×k andγ : R^k×K ×R^k → R^k×k are given functions, B(t) is a k-dimensional Brownian motion and N˜(., .) =

N˜₁(., .), ...,N˜_k(., .)

are k independent compensated Poisson random measures independent of B(.), while K is a given subset of R^p. For eachj = 1, ..., kwe haveN˜_j(dt, dz) =N_j(dt, dz)−ν_j(dz)dt, whereν_jis the Lévy measure (intensity measure) of the Poisson random measureN_j(., .).

We may regardu(t, z) = (u₀(t), u₁(t, z))as our control process, assumed to bec`adl`ag,F_t-adapted and with values inK ×Kfor a.a. t, z, ω.

ThusY(t) = Y^(u)(t)is acontrolled jump diffusion.

Let f : R^k × K → R and g : R^k → R be given functions. Let A be a given set of controls contained in the set ofu= (u₀, u₁)such that (3) has a unique strong solution and such that

E^y Z τS

0

|f(Y(t), u(t))|dt

<∞ (4)

(whereE^y denotes expectation whenY(0) =y) where

τ_S = inf{t >0;Y(t)∈ S}/ (the bankruptcy time) (5) is the first exit time of a given opensolvency setS⊂R^k. We letT denote the set of all stopping times τ ≤τ_S. We assume that

g⁻(X(τ)) _τ∈T is uniformly integrable. (6) Note thatY^(u)(t)is quasi-left continuous, in the sense that for each givenτ ∈ T we have

lim

t→τ⁻Y^(u)(t) = Y^(u)(τ), see [12], Proposition I. 2.26 and Proposition I. 3.27.

Forτ ∈ T andu∈ Awe define theperformance functionalJ^τ,u(y)by J^τ,u(y) =E^y

Z τ 0

f(Y(t), u(t))dt+g(Y(τ))

(7) (we interpretg(Y(τ))as0ifτ =∞).

We regard τ as the “control” of player number 1 and u as the control of player number 2, and consider thestochastic differential game to find the value functionΦand an optimal pair (τ^∗, u^∗) ∈ T × Asuch that

Φ(y) = inf

u∈A

sup

τ∈T

J^τ,u(y)

=J^τ∗,u∗(y). (8)

We restrict ourselves to Markov controls u = (u₀, u₁), i.e. we assume that u₀(t) = ¯u₀(Y(t))and u₁(t) = ¯u₁(Y(t), z)for some functionsu¯₀ :R^k→K,u¯₁ :R^k×R^k →K. For simplicity of notation we will in the following not distinguish betweenu₀andu¯₀,u₁andu¯₁.

When the controlu is Markovian the corresponding processY^(u)(t) becomes a Markov process, with generatorA^u given by

A^uϕ(y) =

k

X

i=1

b_i(y, u₀(y))∂ϕ

∂yi

(y) (9)

+ 1 2

k

X

i,j=1

(σσ^t)_ij(y, u₀(y)) ∂²ϕ

∂yi∂yj

(y)

+

k

X

j=1

Z

R

ϕ(y+γ^(j)(y, u1(y, z), z)−ϕ(y)

−∇ϕ(y)·γ^(j)(y, u₁(y, z), z)}ν_j(dz) ; ϕ∈ C²(R^k).

Here∇ϕ= (_∂y^∂ϕ

1, ...,_∂y^∂ϕ

k)is the gradient ofϕandγ^(j)is column numberj of thek×kmatrixγ.

We can now formulate the main result of this section:

Theorem 2.1 (Verification theorem for stopping-control games) Suppose there exists a functionϕ: ¯S →Rsuch that

(i)ϕ∈ C¹(S)T C( ¯S) (ii)ϕ≥g onS Define

D={y ∈ S; ϕ(y)> g(y)} (the continuation region). (10) Suppose, withY(t) =Y^(u)(t),

(iii)E^yRτS

0 χ_∂D(Y(t))dt

= 0for allu∈ A (iv)∂Dis a Lipschitz surface

(v)ϕ∈ C²(S \∂D), with locally bounded derivatives near∂D (vi) there existsuˆ∈ Asuch that

A^uˆϕ(y) +f(y,u(y)) = infˆ

u∈A{A^uϕ(y) +f(y, u(y))}







= 0, fory ∈D,

≤0, fory∈ S \D.

(vii)E^y

|ϕ(Y(τ))|+Rτ

0 |A^uϕ(Y(t))|dt

<∞, for allτ ∈ T and allu∈ A.

Foru∈ Adefine

τ_D =τ_D^(u) = inf{t >0;Y^(u)(t)∈/ D} (11) and, in particular,

ˆ

τ =τ_D^(ˆu) = inf{t >0;Y^(ˆu)(t)∈/ D}.

(viii) Suppose that the family{ϕ(Y(τ)); τ ∈ T, τ ≤τ_D}is uniformly integrable, for eachu∈ A, y ∈ S.

Thenϕ(y) = Φ(y)and(ˆτ ,u)ˆ ∈ T × Ais an optimal pair, in the sense that Φ(y) = inf

u

sup

τ

J^τ,u(y)

= sup

τ

J^τ,ˆu(y) =J^{ˆτ ,ˆu}(y) = ϕ(y) = inf

u J^τD,u(y) = sup

τ

infu J^τ,u(y) . (12) Proof. Chooseτ ∈ T and letuˆ ∈ Abe as in (vi). By an approximation argument (see Theorem 3.1 in [16]) we may assume thatϕ∈ C²(S). Then by the Dynkin formula (see Theorem 1.24 in [16]) and (vi) we have, withYˆ =Y^(ˆu)

E^y h

ϕ

Yˆ(τ_mi

= ϕ(y) +E^y Z τm

0

A^uˆϕ Yˆ(t)

dt

≤ ϕ(y)−E^y Z τm

0

f

Yˆ(t),u(t)ˆ dt

, whereτ_m =τ ∧m;m= 1,2, ....

Lettingm→ ∞this gives, by (4), (6), (vii), (i) and the Fatou Lemma, ϕ(y) ≥ lim inf

m→∞ E^y Z τm

0

f

Yˆ(t),u(t)ˆ

dt+ϕ( ˆY(τ_m))

≥ E^y Z τ

0

f

Yˆ(t),u(t)ˆ

dt+g( ˆY(τ)χ{τ <∞}

=J^τ,ˆu(y). (13)

Since this holds for allτ we have ϕ(y)≥sup

τ

J^τ,ˆu(y)≥inf

u

sup

τ

J^τ,u(y)

, for all u∈ A. (14)

Next, for givenu∈ Adefine, withY(t) = Y^(u)(t),

τ_D =τ_D^u = inf{t >0;Y(t)∈/ D}.

Choose a sequence{D_m}^∞_m=1of open sets such thatD¯_mis compact,D¯_m ⊂D_m+1andD =

∞

[

m=1

D_m and define

τ_D(m) = m∧inf{t >0; Y(t)∈/ D_m}.

By the Dynkin formula we have, by(vi), form= 1,2, ..., ϕ(y) = E^y

"

−

Z τ_D(m) 0

A^uϕ(Y(t))dt+ϕ(Y(τ_D(m)))

#

(15)

≤ E^y

"

Z τD(m) 0

f(Y(t), u(t))dt+ϕ(Y(τ_D(m))

# .

By the quasi-left continuity ofY(.)(see [12], Proposition I. 2. 26 and Proposition I. 3. 27), we get Y(τ_D(m))→Y(τ_D) a.s. as m→ ∞.

Therefore, if we letm → ∞in (15) we get ϕ(y)≤E^y

Z τD

0

f(Y(t), u(t))dt+g(Y(τ_D))

=J^τD,u(y).

Since this holds for allu∈ Awe get ϕ(y)≤inf

u J^τD,u(y)≤sup

τ

infu J^τ,u(y)

. (16)

In particular, applying this tou= ˆuwe get equality, i.e.

ϕ(y) = J^{τ ,ˆˆu}(y). (17)

Combining (14), (16) and (17) we obtain infu

sup

τ

J^τ,u(y)

≤ sup

τ

J^τ,ˆu(y)≤ϕ(y) = J^{τ ,ˆˆu}(y) =ϕ(y)

≤inf

u J^τD,u(y) ≤ sup

τ

infu J^τ,u(y)

≤inf

u

sup

τ

J^τ,u(y)

. (18)

Since we always have

sup

τ

infu J^τ,u(y)

≤inf

u

sup

τ

J^τ,u(y)

(19) we conclude that we have equality everywhere in (18) and the proof is complete.

Remark 2.1 It is natural to ask if the value function in the above theorem is the unique viscosity solution of the corresponding HJB variational inequalities. This will be proved to be the case by some of us in a subsequent paper (work in progress).

3 Viscosity solutions

Let the state, Y(t) = Y^u(t), be given by equation (3), the performance functional by equation (7) and the value function by equation (8). In the following we will assume that the functionsb, σ, γ, f, g are continuous with respect to(y, u). Further, the following standard assumptions are adopted; there existsC > 0,α:R^k→R^kwithR

α²(z)ν(dz)<∞such that for allx, y ∈R^k,z ∈R^kandu∈K, A1. |b(x, u)−b(y, u)|+|σ(x, u)−σ(y, u)|+|f(x, u)−f(y, u)|+|g(x)−g(y)| ≤C|x−y|, A2. |b(y, u)|+|σ(y, u)| ≤C(1 +|y|),

A3. |f(y, u)|+|g(y)| ≤C(1 +|y|)^m,

A4. |γ(x, u₁, z)−γ(y, u₁, z)| ≤α(z)|x−y|,

A5. |γ(x, u₁, z)| ≤α(z)(1 +|x|)and|γ(x, u₁, z)|1_|z|<1 ≤C_x,C_x ∈R. Let us define a HJB variational inequality by

max

u∈Kinf [A^uϕ(y) +f(y, u(y))], g(y)−ϕ(y)

= 0, (20)

and

ϕ=gon∂S. (21)

whereA^yϕ(y)is defined by equation (9).

Definition 3.1 (Viscosity solutions) A locally bounded functionϕ ∈ U SC( ¯S) is called a viscosity subsolution of (20)-(21)inSif (21)holds and for eachψ ∈C₀²(S)and eachy₀ ∈ Ssuch thatψ ≥ϕ onSandψ(y₀) = ϕ(y₀), we have

max

u∈Kinf [A^uψ(y₀) +f(y₀, u(y₀))], g(y₀)−ψ(y₀)

≥0 (22)

A functionϕ∈ LSC( ¯S)is called a viscosity supersolution of the(20)-(21)inS if (21)holds and for eachψ ∈C₀²(S)and eachy₀ ∈ S such thatψ ≤ϕonSandψ(y₀) = ϕ(y₀), we have

max

u∈Kinf [A^uψ(y0) +f(y0, u(y0))], g(y0)−ψ(y0)

≤0 (23)

Further, ifϕ ∈ C([0, T]×Rⁿ) is both a viscosity subsolution and a viscosity supersolution it is called a viscosity solution.

Proposition 3.2 (Dynamic programming principle) LetΦbe as in(8). Then we have (i) ∀h >0,∀y∈R^k

Φ(y) = sup

τ∈T

u∈Ainf E^y[ Z τ∧h

0

f(Y(s), u(s))ds+g(Y_τ)1_{τ <h}+ Φ(Y_h)1h≤τ].

(ii) Letε >0,y∈R^k,u∈ Aand define the stopping time

τ_y,u^ε = inf{0≤t≤τ_s; Φ(Y_t^y,u)≤g(Y_t^y,u) +ε}.

Then, ifτ_u ≤τ_y,u^ε , for allu∈ A, we have that:

Φ(y) = inf

u∈AE^y[ Z τu

0

f(Y(s))ds+ Φ(Y_τ^y_u)].

Remark 3.1 Prop.3.2 (i) is a consequence of Prop. 3.2 (ii) as observed in Krylov [14] p135.

Proof.The demonstration being long is postponed for more brightness in Part 5.

Theorem 3.3 Under assumptions A1-A4, the value functionΦis a viscosity solution of (20)-(21).

Proof. Φ is continuous according to the estimates of the moments of the jump diffusion state process (see Lemma 3.1 p.9 in [18]) and from Lipschitz condition A2 onf andg we get that

Φ(y) =g(y)on∂S.

We now prove thatΦis a subsolution of (20)-(21). Letψ ∈C₀²(S)andy₀ ∈ S such that 0 = (ψ−Φ)(y₀) = min

y (ψ −Φ). (24)

Define

D={y∈ S|Φ(y)> g(y)}.

If y₀ ∈/ D then g(y₀) = Φ(y₀) and hence (22) holds. Next suppose y₀ ∈ D. Then we have by Proposition 3.2 forτˆ=τ_D andh >0small enough:

Φ(y₀) = inf

u∈AE^y0[ Z h

0

f(Y^y0(t), u(t))dt+ Φ(Y^y0(h))].

From (24) we get

0≤ inf

u∈AE^y0[ Z h

0

f(Y^y0(t), u(t))dt+ψ(Y^y0(h))−ψ(y₀)].

By Itô ’s formula we obtain that 0≤ inf

u∈A

1 hE^y0

Z h 0

[A^uψ(Y_t^y0) +f(Y^y0(t), u(t))]dt

.

Using assumptions A1-A4 with estimates on the moments of a jump diffusion and by lettingh→0⁺, we have

u∈Kinf [A^uψ(y₀) +f(y₀, u(y₀))]≥0, and hence

max

u∈Kinf[A^uψ(y₀) +f(y₀, u(y₀))], g(y₀)−ψ(y₀)

≥0.

This shows thatΦis a viscosity subsolution. The proof for supersolution is similar.

The problem of showing uniqueness of viscosity solution is not addressed in this paper but will be considered in a future article.

4 Examples

Let us look at some control problems where we include stopping times as one of the controls. We then apply the result of the previous section to find a solution. We will look at both a jump and a non-jump market.

Exemple 4.1 (Optimal Resource Extraction in a Worst Case Scenario) Let

dP(t) = P(t)[αdt+βdB(t) + Z

R0

γ(z) ˜N(dt, dz)];P(0) =y₁ >0, where α, β are constants andγ(z)is a given function such that R

R0γ²(z)ν(dz) < ∞. LetQ(t)be the amount of remaining resources at time t, and let the dynamics be described by

dQ(t) = −u(t)Q(t)dt;Q(0) =y2 ≥0.

whereu(t)controls the consumption rate of the resourceQ(t), and m is the maximum extraction rate.

We let

dY(t) =



















dY₀(t) =dt

dY₁(t) =dP(t);P(0) =y₁ >0, dY₂(t) =dQ(t);Q(0) =y₂ ≥0, dY3(t) =−Y3(t)

h

θ0(t)dB(t) +R

R0θ1(t, z) ˜N(dt, dz) i

;Y3(0) =y3 >0.

(25)

Let the running cost be given byK₀+K₁u_t(K₀, K₁ ≥0, constants). Then we let our performance functional be given by, withθ= (θ₀, θ₁),

J^τ,u,θ(s, y₁, y₂, y₃) (26)

=E^y Z τ

0

e^−δ(s+t)(u(t)(P(t)Q(t)−K1)−K0)Y3(t)dt+e^−δ(s+τ)(M P(τ)Q(τ)−a)Y3(τ)

, whereδ >0is the discounting rate andM > 0, a > 0are constants (a can be seen as a transaction cost). Our problem is to find(ˆτ ,u,ˆ θ)ˆ inT ×U ×Θsuch that

Φ(y) = Φ(s, y₁, y₂, y₃) = sup

u

inf

θ

sup

τ

J^τ,u,θ(y)

=J^{τ ,ˆˆu,θˆ}(y). (27) Then the generator ofY^u,θ is given by;

A^u,θϕ(y) =A^u,θϕ(s, y₁, y₂, y₃) = ∂ϕ

∂s +y₁α∂ϕ

∂y₁ −uy₂∂ϕ

∂y₂ +1

2y₁²β²∂²ϕ

∂²y₁ + 1

2y₃²θ²₀∂²ϕ

∂²y₃

−y₁y₃βθ₀ ∂²ϕ

∂y1∂y3

+ Z

R0

ϕ(s, y₁ +y₁γ(z), y₂, y₃−y₃θ₁(z))−ϕ(s, y₁, y₂, y₃)−y₁γ(z)∂ϕ

∂y₁ +y₃θ₁(z)∂ϕ

∂y₃

ν(dz).

We need to find a subsetDofS =R⁴+ = [0,∞)⁴andϕ(s, y1, y2, y3)such that ϕ(s, y₁, y₂, y₃) = g(s, y₁, y₂, y₃) := e^−δs(M y₁y₂−a)y₃, ∀(s, y₁, y₂, y₃)∈/ D, ϕ(s, y₁, y₂, y₃)≥e^−δs(M y₁y₂ −a)y₃, ∀(s, y₁, y₂, y₃)∈ S,

A^u,θϕ(s, y₁, y₂, y₃) +f(s, y₁, y₂, y₃, u) :=A^u,θϕ(s, y₁, y₂, y₃) +e^−δs(u(y₁y₂−K₁)−K₀)y₃

≤0, ∀(s, y₁, y₂, y₃)∈ S\D, ∀u∈[0, m], sup

u

h inf

θ {A^u,θϕ(s, y₁, y₂, y₃) +e^−δs(u(y₁y₂−K₁)−K₀)y₃}i

= 0, ∀(s, y₁, y₂, y₃)∈D.

Then

θˆ₀ = y₁ y₃βϕ₁₃

ϕ₃₃, (28)

is a minimizer ofθ₀ 7→A^u,θϕ(s, y₁, y₂, y₃)where we are using the notation ϕ_ij = ∂²ϕ

∂y_j∂y_i.

Letθˆ₁(z)be the minimizer ofθ₁(z) 7→ A^u,θϕ(y)and letuˆbe the the maximizer ofu 7→ A^u,θϕ(y) + f(y, u)i.e.

u7→A^u,θϕ(s, y₁, y₂, y₃) +e^−δsuy₃(y₁y₂−K₁)−uy₂ϕ₂ −y₃K₀. (29) Let us try a function on the form

ϕ(s, y₁, y₂, y₃) = e^−δsF(w), wherew=y₁y₂y₃. (30) Then

ˆ u=







m, ifwF⁰(w)< w−y₃K₁ 0, otherwise,

(31)

and

θˆ=β

1 + F⁰(w) F⁰⁰(w)w

. (32)

Further, the first order condition forθˆ₁(z)is Z

R0

n

(1 +γ(z))F⁰(w(1 +γ(z))(1−θˆ₁(z)))−F⁰(w)o

ν(dz) = 0. (33)

ForwF⁰(w)< w−y3K1 we have

A^ˆu,θˆe^−δsF(y₁, y₂, y₃) =−δe^−δsF(w) +we^−δsαF⁰(w)−mwe^−δsF⁰(w) (34) + 1

2w²β²F⁰⁰(w)e^−δs+ 1

2w²β²F⁰⁰(w)e^−δs

1 + ( F⁰(w)

F⁰⁰(w)w)²+ 2F⁰(w) F⁰⁰(w)w

−wβ²e^−δs

F⁰(w) + (F⁰(w))²

F⁰⁰(w)w +wF⁰⁰(w) +F⁰(w)

+e^−δs Z

R0

n

F(w(1 +γ(z))(1−θˆ₁(z)))−F(w)−wγ(z)F⁰(w) + ˆθ₁(z)wF⁰(w)o ν(dz)

=−δe^−δsF(w) +we^−δsαF⁰(w)−mwe^−δsF⁰(w) +β²e^−δs

−(F⁰(w))²

2F⁰⁰(w) −wF⁰(w)

+e^−δs Z

R0

n

F(w(1 +γ(z))(1−θˆ₁(z)))−F(w)−wγ(z)F⁰(w) + ˆθ₁(z)wF⁰(w)o ν(dz).

We then need that ifwF⁰(w)< w−y3K1, then

A^u,ˆθˆF(w) + (m(y₁y₂−K₁)−K₀)y₃ =−δF(w) +wαF⁰(w)−mwF⁰(w) (35)

−β²

(F⁰(w))²

2F⁰⁰(w) +wF⁰(w)

+ Z

R0

n

F(w(1 +γ(z))(1−θˆ₁(z)))−F(w)−wγ(z)F⁰(w) + ˆθ₁(z)wF⁰(w)o ν(dz) + (m(y₁y₂−K₁)−K₀)y₃ = 0.

Similarly, ifwF⁰(w)≥w−y₃K₁, thenuˆ= 0and hence we must have

A^u,ˆθˆF(w)−K₀y₃ =−δF(w) +wαF⁰(w) (36)

−β²

(F⁰(w))²

2F⁰⁰(w) +wF⁰(w)

+ Z

R0

n

F(w(1 +γ(z))(1−θˆ1(z)))−F(w)−wγ(z)F⁰(w) + ˆθ1(z)wF⁰(w) o

ν(dz)

−K0y3 = 0.

The continuation regionDgets the form

D={(s, y₁, y₂, y₃) :F(w)>(M y₁y₂ −a)y₃} Therefore we get the requirement

F(w) = (M y₁y₂−a)y₃, ∀(s, y₁, y₂, y₃)∈/ D. (37) In light of this requirement and in order forϕto be on the form (30) we see that we needK₀, K₁ anda to be zero. Hence we letK₀ = K₁ = a = 0from now on. Then we need thatF satisfies the variational inequality

max{A^u,₀^ˆθˆF(w) + ˜mw, M w−F(w)}= 0, w >0, (38)

where

A^u,₀^ˆθˆF(w) =−δF(w) +wαF⁰(w)−mwF˜ ⁰(w)−β²

(F⁰(w))²

2F⁰⁰(w) +wF⁰(w)

(39) +

Z

R0

n

F(w(1 +γ(z))(1−θˆ₁(z)))−F(w)−wγ(z)F⁰(w) + ˆθ₁(z)wF⁰(w)o ν(dz), with

˜

m:=mχ(−∞,1)(F⁰(w)). (40)

The variational inequality (38) - (40) is hard to solve analytically, but it may be accessible by numer- ical methods.

Exemple 4.2 (Worst case scenario optimal control and stopping in a Lévy -market) Let our dy- namics be given by

dY0(t) =dt; Y0(0) =s∈R.

dY₁(t) = (Y₁(t)α(t)−u(t))dt+Y₁(t)β)dB(t) +Y₁(t⁻)

Z

R

γ(s, z) ˜N(ds, dz); Y₁(0) =y₁ >0.

dY₂(t) =−Y₂(t)θ₀(t)dB(t)−Y₂(t) Z

R

θ₁(s, z) ˜N(ds, dz); Y₂(0) =y₂ >0.

Solve

Φ(s, x) = sup

u

sup

τ

θinf0,θ1

J^θ,u,τ(s, x)

where

J^θ,u,τ(s, x) =E^x Z τ

0

e^−δ(s+t)u^λ

λ Y₂(t)dt

The interpretation of this problem is the following:

Y₁(T)represents the size of the population (e.g. fish) when a harvesting strategyu(t)is applied to it.

The processY₂(t)represents the Radon-Nikodym derivative of a measureQwith respect toP, i.e.

Y₂(t) = d(Q|F_t)

d(P|F_t) =E[dQ

dP|F_t]; 0≤t ≤T.

This means that we can write

J^θ,y,τ(s, x) =E^x[ Z ∞

0

e^−δ(s+t)χ_[0,τ](t)u^λ(t) λ E[dQ

dP|F_t]dt]

=E_Q^x[ Z ∞

0

e^−δ(s+t)u^λ(t) λ dt].

HenceJ^θ,y,τ represents the expected utility up to the stopping timeτ, measured in terms of a scenario (probability measureQ) chosen by the market. Therefore our problem may be regarded as a worst case scenario optimal harvesting/stopping problem. Alternatively, the problem may be interpreted as a risk

minimizing optimal stopping and control problem. To see this, we use the following representation of a given convex risk measureρ:

ρ(F) = sup

Q∈P

{E_Q[−F]−ς(Q)};F ∈L^∞(P),

whereP is the set of all measuresQabove andς : P → Ris a given convex “penalty” function. If ς = 0as above, the risk measureρis called coherent. See [1], [9] and [10] .

In this case our generator becomes A^u,θϕ(s, y₁, y₂) = ∂ϕ

∂s + (y₁α−u)∂ϕ

∂y₁ + 1

2y²₁β²∂²ϕ

∂²y₁ +1

2y²₂θ²₀∂²ϕ

∂²y₂ −y₁y₂βθ₀ ∂²ϕ

∂y₁∂y₂ +

Z

R

ϕ(s, y₁ +y₁γ(s, z), y₂−y₂θ₁(s, z))−ϕ(s, y₁, y₂)−y₁γ(s, z)∂ϕ

∂y₁ +y₂θ₁(z)∂ϕ

∂y₂

ν(dz).

and hence

A^u,θϕ(s, y₁, y₂) +f(s, y₁, y₂) = ∂ϕ

∂s + (y₁α−u)∂ϕ

∂y₁ +1

2y₁²β²∂²ϕ

∂²y₁ + 1

2y₂²θ²₀∂²ϕ

∂²y₂ −y₁y₂βθ₀ ∂²ϕ

∂y₁∂y₂ +

Z

R

ϕ(s, y₁+y₁γ(s, z), y₂−y₂θ₁(s, z))−ϕ(s, y₁, y₂)−y₁γ(s, z)∂ϕ

∂y₁ +y₂θ₁(z)∂ϕ

∂y₂

ν(dz) +e^−δsu^λ

λ y₂.

Imposing the first-order condition we get the following equations for the optimal control processes θˆ0,θˆ1andu:ˆ

θˆ₀ = y₁ y2

βϕ₁₂ ϕ22

,

Z

R

{ϕ₂(s, y₁+y₁γ(s, z), y₂ −y₂θˆ₁(s, z))−ϕ₂(s, y₁, y₂)}ν(dz) = 0, and

ˆ

u= (e^δsϕ₁ y₂ )^λ−11 , whereϕ_i = _∂y^∂ϕ

i;i= 1,2. This gives A^u,ˆθˆϕ(s, y₁, y₂) +f(s, y₁, y₂,u) =ˆ ∂ϕ

∂s + (y₁α−(e^δsϕ₁

y₂)^λ−11 )ϕ₁+ 1

2y₁²β²ϕ₁₁−1

2y²₁β²ϕ²₁₂

ϕ₂₂ (41) +

Z

R

ϕ(s, y₁+y₁γ(s, z), y₂−y₂θˆ₁(s, z))−ϕ(s, y₁, y₂)−y₁γ(s, z)∂ϕ

∂y1

+y₂θˆ₁(z)∂ϕ

∂y2

ν(dz)

+e^−δs(^φ1_y^eδs

2 )^λ−1λ λ y₂.

Let us try a value function of the form

ϕ(s, y₁, y₂) = e^−δsy₁^λF(y₂), (42) for some functionF (to be determined). Then

θˆ₀ =βλβF⁰(y₂)

y₂F⁰⁰(y₂), (43)

Z

R

{(1 +γ(s, z))^γF⁰(y2−y2θˆ1(s, z))−F⁰(y2)}ν(dz) = 0, (44) and

ˆ

u= (F(y₂)λ y2

)^λ−11 y₁. (45)

Withθˆ₁ as in(44)put

A^θ,ˆ₀^ˆuF(y₂) =−δF(y₂) + (α−(λF(y₂)

y₂ )^λ−11 )λF(y₂) + 1

2β²λ(λ−1)F(y₂)− 1

2β²F⁰²(y₂)

F⁰⁰(y₂) (46) +y₂

λ(λF(y₂)

y₂ )^λ−1λ + Z

R

h

(1 +γ(z))^λF(y2−y2θˆ1(z))

−F(y₂)−γ(z)λF(y₂) +y₂θˆ₁(z)F⁰(y₂)i ν(dz).

Thus we see that the problem reduces to the problem of solving a non-linear variational-integro inequality as follows:

Suppose there exits a processθˆ₁(s, z)satisfying(44)and aC¹-functionF :R+→R+such that if we put

D={y₂ >0;F(y₂)>0}

thenF ∈C²(D)and

A^θ,ˆ₀^ˆuF(y₂) = 0fory₂ ∈D.

Then the function ϕgiven by (42)is the value function of the problem. The optimal control process are as in(43)-(45)and an optimal stopping time is

τ^∗ = inf{t >0;Y2(t)∈/ D}.

Exemple 4.3 (Risk minimizing optimal portfolio and stopping)

dY₀(t) = dt; Y₀(0) =s∈R. (47)

dY₁(t) = Y₁(t)[(r+ (α−r)π(t))dt+βπ(t)dB(t)]; Y₁(0) =y₁ >0. (48) dY₂(t) = −Y₂(t)θ(t)dB(t); Y₂(0) =y₂ >0, (49)

wherer, αandβ >0are constants. Solve Φ(s, x) = sup

π

sup

τ

infθ J^π,θ,τ

(50) where

J^π,θ,τ(s, x) =E^x

e^−δτλY₁(τ)Y₂(τ)

, (51)

where0< λ≤1and(1−λ)is a percentage transaction cost. The generator is A^θ,πϕ(s, y₁, y₂) +f(s, y₁, y₂) = ∂ϕ

∂s +y₁(r+ (α−r)π)∂ϕ

∂y₁ +1

2y₁²β²π²∂²ϕ

∂²y₁ + 1

2y₂²θ²∂²ϕ

∂²y₂ −y₁y₂βθπ ∂²ϕ

∂y₁∂y₂. From the first order conditions we get that

ˆ

π = (α−r)ϕ₁ϕ₂₂ y1β²(ϕ²₁₂−ϕ11ϕ22), and

θˆ= (α−r)ϕ₁ϕ₁₂ βy₂(ϕ²₁₂−ϕ₁₁ϕ₂₂). Let us try to put

ϕ(s, y₁, y₂) =e^−δsλy₁y₂. (52) Then we get

A^θ,ˆˆπϕ(s, y1, y2) =y1y2(r−δ), θˆ= α−r

β . (53)

and

ˆ

π= 0. (54)

So if

r−δ≤0,

thenA^θ,ˆˆπϕ≤0and the best is to stop immediately andϕ= Φ. If r−δ >0,

then

D= [0, T]×R^k×R^k, soτˆ=T.

Remark 4.1 Note that the optimal value given in(53)forθˆcorresponds to choosing the measureQ defined by

dQ(ω) = Y2(T)dP(ω)

to be an equivalent martingale measure for the underlying financial market(S₀(t), S₁(t))defined by dS₀(t) =rdt;S₀(0) = 0,

dS₁(t) =S₁(t)[αdt+βdB(t)];S₁(0) >0.

This illustrates that equivalent martingale measures often appear as solutions of stochastic differen- tial games between the agent and the market. This was first proved in [17] and subsequent in a partial information context in [2] and [3].

5 Proof of the Dynamical Programming Principle (Prop. 3.2)

The Proposition3.2 (ii) is proved by Krylov [14] for diffusion processes and mixed strategies whereas Pham [18] has mentioned how to generalize it to this context of jump diffusions. Let us explain how it may be adapted to our case of stochastic differential games with optimal stopping and stochastic control for jump diffusions. First we need a Bellman’s Principle for stochastic differential games. We here refer to Fleming and Sougadinis [8] Theorem 1.6 and Biswas [5] Theorem 2.2, whose proofs rest on the continuity of the value functions and the introduction of a restrictive class of admissible strategies. Next, to generalize the Dynamic Programming Principle to our optimal stopping and stochastic control differential games problem, we use the technique of randomized stopping developed by Krylov [14] p. 36. For greater generality, we establish a version of the Dynamical Programming Principle whose Prop. 3.2. is a particular case.

5.1 A general context

5.1.1 Dynamics

For a fixed positive constantT ands∈[0, T), the stateY(t) =Y_t^s,y,uwhere0≤t≤T −sis driven by

dY(t) = b(s+t, Y(t), u(t))dt+σ(s+t, Y(t), u(t))dB(t) +

Z

R^k₀

γ s+t, Y(t⁻), u(t), zN(dt, dz)˜ (55) with the initial condition

Y(s) =y∈S.

b: [0, T]×R^k×K →R^k ,σ: [0, T]×R^k×K →R^k×kandγ : [0, T]×R^k×K×R^k →R^k×k are given functions and verify the assumptions of regularity of the part 3 uniformly with respect tot and are lipschitz continuous in the variabletfor all(y, u). B(t)andN˜(., .)are defined as in part 2.

u(t)is the control process assumed to be predictable and with values inKfor a.a. t, ω. The set of controls is denoted byM(s).

τS = inf{0< t≤T −s;Y(t)∈ S}/ (56) Ts,T denotes the set of all stopping timesτ ≤τS

Forτ ∈ Ts,T andu∈M(s)the performance functional is J^τ,u(s, y) =E^us,y

Z τ 0

f(s+t, Y(t), u(t))e^−ϕtdt+g(τ, Y(τ))e^−ϕτ

(57) whereϕ^s,y,u_t = Rt

0c^ur(s+r, Y_r^s,y,u)dr, c: [0, T]×R^k×K → R⁺, f : [0, T]×R^k ×K → Rand g : [0, T]×R^k →Rare given functions.

The value function is defined as

Φ(s, y) = sup

τ∈Ts,T

inf

u∈M(s)

J^τ,u(s, y). (58)

5.1.2 The canonical sample space

We work in a canonical Wiener-Poisson space, following [5], [11] and [6]. For a constant T and 0 ≤ s < t ≤ T, letΩ¹_s,t be the standard Wiener space i.e. the set of all functions from[s, t]to R^k starting from 0and topologized by the sup-norm. We denote the corresponding Borelσ-algebra by B₁⁰and letP_s,t¹ be the Wiener measure on(Ω¹_s,t,B₁⁰).

In addition, upon denotingQ^∗_s,t = [s, t]×(R^k\0), letΩ²_s,tbe the set of allNS{∞}-valued measures on(Q^∗_s,t,B(Q^∗_s,t))whereB(Q^∗_s,t)is the usual Borelσ-algebra ofQ^∗_s,t. We denoteB₂⁰to be the smallest σ-algebra over Ω²_s,t so that the mappings q ∈ Ω²_s,t 7→ q(A) ∈ NS{∞} are measurable for all A ∈ B(Q^∗_s,t). Let the co-ordinate random measure N_s,t be defined as N_s,t(q, A) = q(A) for all q ∈Ω²_s,t,A∈ B(Q^∗_s,t)and denoteP_s,t² to be the probability measure on(Ω²_s,t,B₂⁰)under whichN_s,tis a Poisson random measure with Lévy measureνsatisfying

Z

R\{0}

min(|z|²,1)ν(dz)<∞.

Next, for very 0 ≤ s < t ≤ T, we define Ω_s,t = Ω¹_s,t ×Ω²_s,t, P_s,t ≡ P_s,t¹ N

P_s,t² and B_s,t ≡ B₁⁰ × NB⁰₂ i.e. the completion of B⁰₁ × NB₂⁰ with respect to the probability measure P_s,t. We will follow the convention that Ω_t,T ≡ Ω_t andB_t,T ≡ F_t . A generic element of Ω_t is denoted by ω = (ω₁, ω₂), whereω_i ∈Ωⁱ_t,T fori∈ {1,2}, and we define the coordinate functions

W_s^t(ω) =ω1(s) and N^t(ω,A) =ω2(A)

for all 0 ≤ t ≤ s ≤ T, ω ∈ Ω, A ∈ B(Q^∗_t,T). The processW^t is a Brownian motion starting at t andN^tis a Poisson random measure on the probability space(Ω_t,F_t, P_t), and they are independent.

Also, fort∈[0, T], the filtrationF_t,.= (F_t,s)s∈[t,T]is defined as follows:

.

We make Fˆt,. to be right-continuous and denote it by F_t,.⁺. Finally, we augment F_t,.⁺ byPt-null sets and call it Ft,.. As and when it necessitates, we extend the filtrationFt,. fors < tby choosing Ft,s

as the trivialσ algebra augmented byPt-null sets. We follow the convention thatFt,T = Ft. When the terminal time T is replaced by another time point, sayτ, the filtration we have just described is denoted byF_t,.^τ.

Finally, note that the space Ωs,t is defined as the product of canonical Wiener space and Poisson space. Therefore, for anyτ ∈(t, T), we can identify the probability space(Ωt,Ft,., Pt)with(Ωt,τ × Ωτ,F_t,.^τ N

Fτ,., Pt,τN

Pτ)by the following bijection π : Ωt → Ωt,τ ×Ωτ . For a generic element ω = (ω1, ω2)∈Ωt= Ω¹_t,T ×Ω²_t,T, we define

ω^t,τ = (ω₁|_[t,τ], ω₂|_[t,τ])∈Ω_t,τ ω^τ,T = ((ω₁−ω₁(τ))|_[t,τ], ω₂|_[t,τ])∈Ω_τ,T

π(ω) = (ω^t,τ, ω^τ,T)

The description of the inverse mapπ⁻¹is also apparent from above.

5.1.3 The general DPP

Theorem 5.1 Lets∈[0, T], y ∈R^kand letτ =τ^u ∈ T_s,T be defined for eachu∈ M(s). Then Φ(s, y) = sup

γ∈T_s,T

u∈Minf(s)E^us,y

Z τ∧γ 0

f^ut(s+t, Yt)e^−ϕtdt +g(s+γ, Y_γ)e^−ϕγχγ≤τ + Φ(s+τ, Y_τ)e^−ϕτχτ≤γ

, (59) wheref^ut(s+t, Y_t) = f(s+t, Y_t, u_t).

This theorem is in fact a consequence of the more general following result. Letε >0, we define:

τ_s,y,u^ε = inf{t≥0 : Φ(s+t, Y_t^s,y,u)≤g(s+t, Y_t^s,y,u) +ε}.

Theorem 5.2 Lets∈[0, T], y ∈R^k, u∈ M(s)andτ^u ∈ Ts,T . We are given a nonnegative process r_t^u, progressively mesurable and bounded. Then

Φ(s, y) ≥ inf

u∈M(s)E^us,y

Z τ

0

(f^ut +r_tΦ)(s+t, Y_t)e^{−ϕt−R0trpdp}dt +Φ(s+τ, Y_τ)e^{−ϕτ−R0τrpdp}i

. (60)

Ifτ^u≤τ_y,u^ε for someε >0and allu∈ M(s), we have equality in (60).

Proof.(Theorem (5.2 to Theorem 5.1)

We write the right side of (59) asW₁(s, y)then W₁(s, y) ≥ inf

u∈M(s)E^us,y

Z τ∧τ^ε 0

f^ut(s+t, Y_t)e^−ϕtdt+g(s+τ^ε, Y_τ^ε)e^{−ϕτ ε}χ_τ^ε_<τ +Φ(s+τ, Y_τ)e^−ϕτχ_{τ <τ}^ε

. (61)

and from the inequality

g(s+τ^ε, Yτ^ε)≥Φ(s+τ^ε, Yτ^ε)−ε, it follows that

W1(s, y) ≥ inf

u∈M(s)E^us,y

Z τ∧τ^ε 0

f^ut(s+t, Yt)e^−ϕtdt+ Φ(s+τ ∧τ^ε, Yτ∧τ^ε)e^{−ϕτ∧τ ε} −ε

.(62) Sinceτ∧τ^ε≤τ^ε, by Theorem 5.2 forr^u_t = 0, we have that the last lowerbound is equal toΦ(s, y)−ε.

Now letεtend to zero thereforeW1(s, y)≥Φ(s, y).

On the other hand,g(s, y)≤Φ(s, y)so that W₁(s, y) ≤ sup

γ

u∈Minf(s)E^us,y

Z τ∧γ 0

f^ut(s+t, Y_t)e^−ϕtdt+ Φ(s+τ∧γ, Yτ∧γ)e^−ϕτ∧γ

. (63) Let assume in Theorem 5.2 that r_t^u = 0, we note that the last upper bound does not exceedΦ(s, y).

HenceW₁(s, y)≤Φ(s, y).

In order to investigate the proof of the Theorem 5.2 we introduce the case of stochastic differential games (see [5]).

5.2 The stochastic games

5.2.1 Context

We introduce a two-player zero-sum stochastic differential game where the state is governed by con- trolled jump-diffusions. Fort∈[0, T −s]

dY(t) = b(s+t, Y(t), u(t), v(t))dt+σ(s+t, Y(t), u(t), v(t))dB(t) +

Z

R^k₀

γ s+t, Y(t⁻), u(t), v(t), zN(dt, dz);˜ (64) Y(s) = y∈R^k.

Remark 5.3 For us and on the rest of the paperb(t, y, u, v) = b(t, y, u),σ(t, y, u, v) = σ(t, y, u)and γ(t, y, u, v, z) = γ(t, y, u, z)so thatY_t^s,y,u,v =Y_t^s,y,u=Y(t).

For convenience, we shall use the superscriptsu, vand the subscriptss, yon the expectation sign to indicate expectation of quantites which depend ons, yand strategiesu, v. We introducef(t, y, u, v) = f^u(t, y) +vg(t, y);c^u,v(t, y) =c^u(t, y) +v andϕ^s,y,u,v_t =Rt

0 c^ur,vr(s+r, Y_r)dr. We use the notation ψ = (u, v)and define

J_n^ψ(s, y) =E^ψs,y

Z T−s 0

f(s+t, Y_t, u_t, v_t)e^−ϕtdt+g(T −s, Y_T−s)e^−ϕT−s

, (65)

for strategiev = (v_t)with values in[0, n],n∈N^∗.

5.2.2 Admissible controls and strategies

Definition 5.4 An admissible control processu(.)(resp.v(.)) for playerI(resp. playerII) on[s, T] is a K (resp.[0, n])-valued process which is Fs,.-predictable. The set of all admissible controls for playerI(resp.II) is denoted byM(s)(resp.N(s)).

We say the controls u,u˜ ∈ M(s)are the same on [s,t] and writeu ≈ u˜ on[s, t]if P^t(u(r) = ˜u(r) for a.e. r∈[s, t]) = 1. A similar convention is followed for members ofN(s).

Definition 5.5 An admissible strategyα (resp. β) for playerI (resp. II) is a mappingα :N(s) → M(s) (resp. β : M(s) → N(s))such that if v(.) ≈ v(resp.u˜ ≈ u))˜ on[s, t], thenα[v] ≈ α[˜v]on [s, t]for everyt ∈ [s, T](resp.β[u] ≈ β[˜u]). The set of admissible strategies for playerI (resp. II) on[s, T]is denoted byΓ_n(s)(resp.∆_n(s)).

Remark 5.6 We will denoteΓ(s) := S

n

Γ_n(s)and∆(s) :=S

n

∆_n(s).

We set Bn = K ×[0, n] and Bn the set of strategies. Let Rn be a set of nonnegative processes

¯

rt which are progressively measurable with respect to (Ft) and such that r¯t(ω) ≤ n for all (t, ω), B = S

n

Bn and R = S

n

Rn. Each strategy ψ ∈ Bn is obviously a pair of processes (u,v¯) with u= (u_t)∈ M(s),v¯= (β[u]_t)∈∆_n(s).

Definition 5.7 i) The lower value of the SDG (64- 65) with initial data(s, y)is given by Φ_n(s, y) := inf

α∈Γn(s) sup

v∈N(s)

J_n(s, y, α[v], v)

!

(66) ii) The upper value of the SDG (64 - 65) is

Φ_n(s, y) := sup

β∈∆n(s)

u∈M(s)inf J_n(s, y, u, β[u])

. (67)

5.2.3 DPP for stochastic games

Proposition 5.8 The upper and lower value functions are Lipschitz continuous in y, Hölder contin- uous intand verify|Φn(s, y)|+|Φn(s, y)| ≤C(1 +|y|)^m.

Proposition 5.9 Lets∈[0, T],τ ∈ T_s,T, for everyy∈IR^k, Φ_n(s, y) = sup

β∈∆n(s)

u∈M(s)inf E_s,y^u,β[ Z τ

0

(f^ut +β[u]_tg)(s+t, Y_t)e^−ϕtdt

+Φ_n(s+τ, Y_τ)e^−ϕτ] (68)

Φ_n(s, y) = inf

α∈Γ_n(s) sup

v∈N(s)

E_s,y^α,v[ Z τ

0

(f^α[v]t+v_tg)(s+t, Y_t)e^−ϕtdt

+Φ_n(s+τ, Y_τ)e^−ϕτ]. (69)