Dept. of Math. University of Oslo Pure Mathematics No. 23 ISSN 0806–2439 June 2004
Optimal stopping with delayed information
Bernt Øksendal
1,2Revised in May 2005
1 Center of Mathematics for Applications (CMA) Department of Mathematics, University of Oslo P.O. Box 1053 Blindern , N–0316, Oslo, Norway and
2 Norwegian School of Economics and Business Administration, Helleveien 30, N–5045, Bergen, Norway
Abstract
We study a general optimal stopping problem for a strong Markov process in the case when there is a time lagδ >0 from the timeτ when the decision to stop is taken (a stopping time) to the time τ +δ when the system actually stops. Equivalently, we impose the constraint that the admissible times for stopping are stopping (Markov) times with respect to thedelayed flow of information. It is shown that such a problem can be reduced to a classical optimal stopping problem by a simple transformation.
The results are applied
(i) to find the optimal time to sell an asset
(ii) to solve an optimal resource extraction problem, in both cases under delayed information flow.
MSC (200): 93C41, 93E20, 60J25, 91B28
Key words: Optimal stopping, delayed information flow, strong Markov processes, optimal time to sell, optimal time to stop resource extraction.
1 Introduction
Let Y(t) be a strong Markov process in Rk on a filtered probability space (Ω,F,{Ft}t≥0, {Py}y∈Rn), where Py is the probability measure giving the law of {Y(t)}t≥0 when Y(0) = y∈Rk. Letδ ≥0 be a fixed constant. In this paper we consider optimal stopping problems of the form
(1.1) Φδ(y) := sup
α∈Tδ
EyhZ α 0
f(Y(t))dt+g(Y(α))i
where Ey denotes expectation with respect to Py and f : Rk → R, g : Rk → R are given continuous functions such that
(1.2) EyhZ α
0
|f(Y(t))|dt+|g(Y(α))|i
<∞ for all α∈ Tδ
where we interpret g(Y(α)) as 0 if α = ∞. Here Tδ is the set of δ-delayed stopping times, defined as follows
Definition 1.1 A function α: Ω→[δ,∞] is called a δ-delayed stopping time if (1.3) {ω;α(ω)≤t} ∈ Ft−δ for all t ≥δ
or, equivalently,
(1.4) {ω;α(ω)≤s+δ} ∈ Fs for all s≥0 The set of all δ-delayed stopping times is denoted by Tδ.
In other words, if we interpretα(ω) as the time to stop, thenα∈ Tδif the decision whether or not to stop at or before timetis based on the information represented byFt−δ. In particular, if δ= 0 thenTδ =T0 is the family of classical stopping times and (1.1) becomes the classical optimal stopping problem, discussed in many texts (see e.g. [Ø, Ch. 10]).
In the delayed case problem (1.1) models the situation when there is a delay δ > 0 in the flow of information available to the agent searching for the optimal time to stop. An alternative way of stating this, is that there is a delay δ > 0 from the decided stopping time τ ∈ T0 (based on the complete current information available from the system) to the time α =τ+δ ∈ Tδ when the system actually stops. This new formulation is based on the following simple observation:
Lemma 1.2 (i) τ ∈ T0 ⇐⇒α:=τ+δ∈ Tδ (ii) α∈ Tδ ⇐⇒τ :=α−δ∈ T0
Proof. It suffices to prove (i).
First, assume τ ∈ T0. Then, for t≥δ,
{ω;τ(ω) +δ≤t}={ω;τ(ω)≤t−δ} ∈ Ft−δ, and hence α:=τ+δ∈ Tδ.
Conversely, if α:=τ +δ∈ Tδ then
{ω;τ(ω)≤t}={ω;τ(ω) +δ ≤t+δ}={ω;α(ω)≤t+δ} ∈ F(t+δ)−δ =Ft,
and hence τ ∈ T0.
Remark 1.3 In view of this result we see that it is possible to give another interpretation of problem (1.1), namely
(1.5) Φδ(y) = sup
τ∈T0
Ey
hZ τ+δ 0
f(Y(t))dt+g(Y(τ+δ)) i
In this formulation the problem appears as an optimal stopping problem over classical stop- ping times τ ∈ T0, but with delayed effect of the stopping: If the stopping time τ ∈ T0 is chosen, then the system itself is stopped at time τ +δ, i.e. after a delay δ >0.
Note that Tδ⊂ T0 for δ >0 and hence
Φδ(y)≤Φ0(y)
and we can interpret Φ0(y)−Φδ(y) as the loss of value due to the delay of information.
In this paper we show that the delayed optimal stopping problem (1.1) can be reduced to a classical optimal stopping problem by a simple transformation (Theorem 2.1).
We call α∗ ∈ Tδ an optimal stopping time for the problem (1.1) if (1.6) Φδ(y) = EyhZ α∗
0
f(Y(t))dt+g(Y(α∗))i .
This paper may be regarded as a partial extension of [AK2], where the geometric Brow- nian motion case is studied and solved (see Example 3.1), with a more general (Markovian) delay δ(X) ≤ 0. See also [AK1]. For a related type of problem involving impulse control with delivery lags see [BS].
2 Optimal stopping with δ-delayed information
We are now ready to state and prove the main result of this paper:
Theorem 2.1 a) Consider the two optimal stopping problems:
Φδ(y) := sup
α∈Tδ
EyhZ α 0
f(Y(t))dt+g(Y(α))i (2.1)
Φ(y) := sup˜
τ∈T0
EyhZ τ 0
f(Y(t))dt+ ˜gδ(Y(τ))i (2.2)
where
(2.3) g˜δ(y) = EyhZ δ 0
f(Y(t))dt+g(Y(δ))i . Then we have
Φδ(y) = ˜Φ(y) for all y∈Rk, δ≥0.
b) Moreover, α∗ ∈ Tδ is an optimal stopping time for the delayed problem (2.1) if and only if
(2.4) α∗ :=τ∗+δ
where τ∗ ∈ T0 is an optimal stopping time for the non-delayed problem (2.2).
Proof. a) Define
(2.5) J(α)(y) =EyhZ α 0
f(Y(t))dt+g(Y(α))i
; α∈ Tδ, and
(2.6) J˜(τ)(y) = EyhZ τ 0
f(Y(t))dt+ ˜gδ(Y(τ))i
; τ ∈ T0. Choose α∈ Tδ and put
τ =α−δ∈ T0
Then α=τ+δ and hence J(α)(y) =EyhZ α
0
f(Y(t))dt+g(Y(α))i
=EyhZ τ+δ 0
f(Y(t))dt+g(Y(τ +δ))i
=EyhZ τ 0
f(Y(t))dt+ Z τ+δ
τ
f(Y(t))dt+g(Y(τ +δ))i
=EyhZ τ 0
f(Y(t))dt+Eyh
θτnZ δ 0
f(Y(t))dt+g(Y(δ))oii , (2.7)
where θτ is theshift operator, defined by
θτ{h(Y(s))}=h(Y(τ +s)) for s≥0, for all measurable h:Rk −→R, and we have used that
θτ
Z δ 0
f(Y(t))dt
= Z τ+δ
τ
f(Y(t))dt.
We refer to [BG] for more information about Markov processes. By the strong Markov property we now get from (2.7) that
J(α)(y) =EyhZ τ 0
f(Y(t))dt+Eyh
θτnZ τ 0
f(Y(t))dt+g(Y(δ))o Fτii
=EyhZ τ 0
f(Y(t))dt+EY(τ)hZ δ 0
f(Y(t))dt+g(Y(δ))ii
=EyhZ τ 0
f(Y(t))dt+ ˜gδ(Y(τ))i
= ˜J(τ)(y).
(2.8)
Hence, by Lemma 1.2 (ii),
Φδ(y) = sup
α∈Tδ
J(α)(y) = sup
α−δ∈T0
J(α)(y)
= sup
α−δ∈T0
J˜(α−δ)(y) = sup
τ∈T0
J˜(τ)(y) = ˜Φ(y), as claimed.
b) Supposeτ∗ ∈ T0 is optimal for (2.2). Define α∗ :=τ∗+δ.
Then α∗ ∈ Tδ by Lemma 1.2 and by (2.8) combined with a) we have J(α∗)(y) = ˜J(τ∗)(y) = ˜Φ(y) = Φδ(y).
Hence α∗ is optimal for (2.1).
Conversely, if α∗ ∈ Tδ is optimal for (2.1) a similar argument gives that τ∗ :=α∗−δ is
optimal for (2.2).
3 Application 1: The optimal time to sell an asset
In this section we illustrate Theorem 2.1 by solving the following problem:
Example 3.1 (The optimal time to sell an asset)
This case (without the jump part) was first solved by [AK1], with a more general (Markovian) delay δ(X)≥0.
Suppose the value X(t) of an asset at time t is modelled by a geometric L´evy process of the form
(3.1) dX(t) = X(t−)h
µ dt+σ dB(t) + Z
R
zN˜(dt, dz)i
, X(0) =x >0, whereµ, σandxare constants. HereB(t) andη(t) := Rt
0
R
RzN˜(ds, dz) is a Brownian motion and an independent pure jump L´evy process, respectively, where
N˜(dt, dz) =N(dt, dz)−ν(dz)dt
is the compensated Poisson random measure ofη(·),N(dt, dz) is the Poisson random measure of η(·) and ν(dz) is the L´evy measure of η(·). We assume that
(3.2) 0≥z ≥ −1 a.s. ν.
This guarantees that X(t) never jumps down to a negative value. For convenience, we also assume that
(3.3) E[η2(t)]<∞ for all t≥0.
Then by the Itˆo formula for L´evy processes (see e.g. [ØS]) the solution of equation (3.1) is X(t) =xexph
(µ− 12σ2)t+σB(t) +
Z t 0
Z
R
{ln(1 +z)−z}ν(dz)ds+ Z t
0
Z
R
ln(1 +z) ˜N(ds, dz)i
; t ≥0.
(3.4)
We now study the following problem
(3.5) Φδ(s, x) = sup
α∈Tδ
Es,x[e−ρ(s+α)(X(α)−q)],
where Es,x denotes expectation with respect to the probability law Ps,x of the time-space process
dY(t) = dt
dX(t)
; Y(0) = s
x
and ρ >0,q >0 are constants. We assume that
(3.6) ρ > µ.
One possible interpretation of this problem is that Φδ(s, x) represents the maximal expected discounted net payment obtained by selling the asset at a δ-delayed stopping time (ρ is the discounting exponent and q is the transaction cost).
It is well-known that in the no delay case (δ= 0) the solution of the problem (3.5) is the following (under some additional assumptions on the L´evy measure ν):
(3.7) Φ0(s, x) = e−ρsΨ0(x)
where
(3.8) Ψ0(x) =
(x−q ; x≥x∗0 C0xλ ; 0< x < x∗0 Hereλ >1 is uniquely determined by the equation
(3.9) −ρ+µλ+12σ2λ(λ−1) + Z
R
{(1 +z)λ−1−λ z}ν(dz) = 0, and x∗0 and C0 are given by
x∗0 = λ q λ−1 (3.10)
C0 = 1
λ(x∗0)1−λ. (3.11)
The corresponding optimal stopping time τ∗ ∈ T0 is
(3.12) τ∗ = inf{t >0;X(t)≥x∗0}
Thus it is optimal to sell at the first time the priceX(t) equals or exceeds the value x∗0. We refer to [ØS, Example 2.5] for details.
To find the solution in the delay case (δ >0) we note that we have f = 0 and g(y) =g(s, x) =e−ρs(x−q)
Hence, by (2.2),
˜
gδ(y) =Ey[g(Y(δ))] =Es,x[e−ρ(s+δ)(X(δ)−q)]
=e−ρ(s+δ)(Ex[X(δ)]−q) = e−ρ(s+δ)(x eµδ−q)
=e−ρs+δ(µ−ρ)(x−q e−µδ) =K e−ρs(x−q),˜ (3.13)
where
(3.14) K =eδ(µ−ρ) and q˜=q e−µδ.
Thus ˜gδ has the same form as g, so we can apply the results (3.7)–(3.12) to find ˜Φ(y) and the corresponding optimalτ∗:
(3.15) Φ(y) = ˜˜ Φ(s, x) =e−ρsΨ(x)˜ where
(3.16) Ψ(x) =˜
(K(x−q)˜ ; x≥x˜∗ C x˜ λ ; 0< x <x˜∗, with λ as in (3.9). Here ˜x∗ and ˜C are given by
˜
x∗ = λq˜ λ−1 (3.17)
C˜ = 1
λ(˜x∗)1−λ. (3.18)
The corresponding optimal stopping time for problem (2.2) and (2.1), respectively, is
˜
τ∗ = inf{t >0;X(t)≥x˜∗} (3.19)
α∗ = ˜τ∗+δ.
(3.20)
Using Theorem 2.1 we conclude the following:
Theorem 3.2 The value function Φδ(y) for the delayed optimal stopping problem (3.5) is given by
Φδ(y) = ˜Φ(y),
where Φ˜ is as in (3.15)–(3.18). The corresponding optimal stopping time α∗ ∈ Tδ is α∗ = inf{t >0;X(t)≥x˜∗}+δ.
Remark 3.3 Assume for example that
µ >0.
Then comparing (3.17) with the non-delayed case (3.10) we see that ˜q > q and hence
˜ x∗ < x∗0
Thus, in terms of thedelayed effect of the stopping time formulation (see (1.5)), it is optimal to stop at the first timet= ˜τ∗ whenX(t)≥x˜∗. This is sooner than in the non-delayed case, because of the anticipation that during the delay time interval [τ∗, τ∗+δ] X(t) is likely to increase (since µ >0). See Figure 1.
t τ0∗
z }| {
δ
α∗= ˜τ∗+δ
˜ τ∗
˜
x∗=λqeλ−1−µδ (delay case)
˜ x∗
x∗=λ−1λq x∗0 (δ= 0 case)
X(t)
Figure 1. The optimal stopping times for Example 3.1 (µ >0)
4 Application 2: An optimal resource extraction prob- lem
In the no delay case the following example was discussed in [Ø] (continuous case) and [ØS]
(jump diffusion case). Our example models the situation when there is a time lag δ > 0 between the decided stopping time τ ∈ T0 and the time α =τ +δ ∈ Tδ when the result of the stopping decision comes into effect.
Example 4.1 (Optimal time to stop resource extraction) Suppose the price P(t) at time t per unit of a resource (oil, gas, . . . ) is given by
(4.1) dP(t) = P(t−)h
µdt+σdB(t) + Z
R
zN(dt, dz)˜ i
; P(0) =p >0
where, as in Example 3.1, µand σ are given constants and we assume that z ≥0 a.s. with respect to ν.
Let Q(t) denote the amount of remaining resources at time t. As long as the extraction field is open, we assume that the extraction rate is proportional to the remaining amount, i.e.
(4.2) dQ(t) = −λQ(t)dt; Q(0) =q >0
where λ >0 is a known constant.
If we decide to stop the extraction and close the field at a (delayed) stopping timeα ∈ Tδ, then the expected total discounted net profit Jα(s, p, q) is assumed to have the form
(4.3) Jα(s, p, q) =E(s,p,q)h
α
Z
0
e−ρ(s+t)(λP(t)Q(t)−K)dt+θe−ρ(s+α)P(α)Q(α)i
where K >0 is the (constant) running cost rate and ρ >0, θ >0 are other constants. The expectationE(s,p,q) is taken with respect to the probability law P(s,p,q) of the strong Markov process
(4.4) Y(t) :=
s+t P(t) Q(t)
, which starts at y=
s p q
at time t= 0.
The explanation of the quantity Jα(s, p, q) in (4.3) is the following:
As long as the field is open (i.e. as long as t < α) the gross income rate from the production is price times production rate, i.e. P(t)λQ(t). Subtracting the running cost rate K we get the net profit rate
λP(t)Q(t)−K for 0≤t < α.
If the field is closed at time α the net value of the remaining resources is estimated to be θP(α)Q(α). Discounting and integrating/adding these quantities and taking expectation we get (4.3).
We want to find the value function Φδ(s, p, q) and the corresponding optimal delayed stopping time α∗ ∈ Tδ such that
(4.5) Φδ(y) = Φδ(s, p, q) = sup
α∈Tδ
Jα(s, p, q) =Jα∗(s, p, q)
In the case of no delay (δ = 0) it is shown in [ØS, p. 158–162] that if the following relations between the parameters hold:
(4.6) 0< θ(λ+ρ−µ)< λ
then the optimal stopping time τ0∗ ∈ T0 is
(4.7) τ0∗ = inf{t >0;P(t)Q(t)≤w∗0}, where
(4.8) w∗0 = (−r2)K(λ+ρ−µ)
(1−r2)ρ(λ−θ(λ+ρ−µ)), r2 <0 being the negative solution of the equation
(4.9) h(r) := −ρ+ (µ−λ)r+ 12σ2r(r−1) + Z
R
{(1 +z)r−1−rz}ν(dz) = 0
In this case we have
f(y) = f(s, p, q) = e−ρs(λpq−K) and
g(y) =g(s, p, q) =θe−ρspq Thus
˜
gδ(y) =Eyh
δ
Z
0
e−ρ(s+t)(λP(t)Q(t)−K)dti
+Ey[θe−ρ(s+δ)P(δ)Q(δ)]
=
δ
Z
0
e−ρ(s+t)(λE[P(t)Q(t)]−K)dt+θe−ρ(s+δ)Ey[P(δ)Q(δ)]
=
δ
Z
0
e−ρ(s+t)(λpqe(µ−λ)t−K)dt+θe−ρ(s+δ)pqe(µ−λ)δ
=e−ρsh
{(λ+ρ−µ)−1λ(1−e−(λ+ρ−µ)δ) +θe−(λ+ρ−µ)δ}pq− K
ρ (1−e−ρδ)i
=e−ρs[F1pq−F2], where (4.10)
F1 = (λ+ρ−µ)−1λ(1−e−(λ+ρ−µ)δ) +θe−(λ+ρ−µ)δ (4.11)
and F2 = K
ρ (1−e−ρδ) (4.12)
Therefore, according to Theorem 2.1 we have (4.13) Φδ(y) = sup
τ∈T0
Eyh
τ
Z
0
e−ρ(s+t)(λP(t)Q(t)−K)dti
+Ey[e−ρ(s+τ)(F1P(τ)Q(τ) +F2)]
The method used in [ØS] to provide the solution (4.7)–4.9) in the no delay case can easily be modified to find the optimal stopping time τ∗ for the problem (4.13). The result is (4.14) w∗δ = (−r2)K(λ+ρ−µ)e(λ−µ)δ
(1−r)ρ[λ−θ(λ+ρ−µ)] =w0∗e(λ−µ)δ. We have proved:
Theorem 4.2 The optimal stopping time α∗ ∈ Tδ for the delayed optimal stopping problem is
(4.15) α∗ =τδ∗+δ,
where
(4.16) τδ∗ = inf{t >0;P(t)Q(t)≤w∗δ}, with w∗δ given by (4.14).
Remark 4.3 Note that the thresholdwδ∗ for the decision to close down in the case of a time lag in the action only differs from the corresponding threshold w0∗ in the no delay case by the factore(λ−µ)δ.
Assume, for example, thatλ > µ. Then we should decide to stopsoonerin the delay case than in the no delay case, because of the anticipation that P(t)Q(t) will probably decrease during the extra time δ it takes before the closing down actually takes place.
References
[AK1] L. H. R. Alvarez and J. Keppo: The impact of delivery lags on irreversible investment demand under uncertainty. Manuscript March 1998.
[AK2] L. H. R. Alvarez and J. Keppo: The impact of delivery lags on irreversible investment under uncertainty. European J. Operational Research 136 (2002), 173–180.
[BG] R. M. Blumenthal and R. K. Getoor: Markov Processes and Potential Theory. Aca- demic Press 1968.
[BS] A. Bar-Ilan and A. Sulem: Explicit solution of inventory problems with delivery lags.
Math. Operations Research 20 (1995), 709–720.
[Ø] B. Øksendal: Stochastic Differential Equations. 6th edition. Springer-Verlag 2003.
[ØS] B. Øksendal and A. Sulem: Applied Stochastic Control of Jump Diffusions. Springer- Verlag 2004.
[S] A: Shiryaev: Optimal Stopping Rules. Springer-Verlag 1978.
