The Study Group discussed the significance of changes in outputs resulting from changes in the configuration of an assessment, based on Working Document 23 (Darby; Assessment model structural uncertainty in the estimation of Precautionary Reference Points.) contained in Annex 7. Darby highlighted the effect of ‘assessment model structure uncertainty’ on the reference point estimates estimated for the Northern hake stock (Divisions IIIa, Subareas IV, VI, VII and VIIIa,b,d).
The framework of the Precautionary Approach outlined in Annex II of the UN Agreement on Straddling Fish Stocks and Highly Migratory Fish Stocks states that:
“Precautionary reference points should be stock-specific to account, inter alia, for the reproductive capacity, the resilience of each stock and the characteristics of fisheries exploiting the stock, as well as other sources of mortality and major sources of uncertainty.”
As outlined in the 2001 Study Group, ICES has acknowledged that it must:
“... explicitly consider and incorporate uncertainty about the state of stocks into management scenarios; explain clearly and usefully the implications of uncertainty to fisheries management agencies.”
In general, ICES has interpreted uncertainty as the errors associated with estimates obtained from a single stock assessment model structure and reference point estimation method. In instances where multiple scenarios have been presented, based on alternative models, there is no formal procedure for quantifying the additional uncertainty and the
“best available” has been taken to provide advice. Recent studies (Patterson et al. 2001, also described in Gavaris et. al.
2000) have shown that the choice of estimation method can have an appreciable impact on the perception of uncertainty and the risks associated with the consequences of fisheries management decisions.
It was shown that the XSA assessment model specified by the Southern Shelf Demersal Species Working Group is not a unique interpretation of the available assessment information but is one solution from a range of feasible solutions. A review of the model sensitivities and the underlying causes was presented.
The sensitivity of the trends in exploitation rate and biomass arises directly from the reduction in the age range of the assessment from a 10+ age group to 8+, based on the uncertainty of age determination in older hake. This has resulted in 30% of the mature catch in numbers being aggregated into the plus group and the oldest age and ~50% in the oldest two ages and the plus group. Due to poor VPA convergence at the oldest ages, VPA based assessment models fitted to data sets with significant numbers in the oldest age and plus group, are extremely sensitive to the method by which fishing mortality at the oldest age is estimated.
In recent years the WGSSDS has made substantial changes to the XSA model used to assess the Northern hake stock.
As a result the assessment model structure may have become unstable due to the aggregation into fewer age groups.
The sensitivity of the estimated biomass and average fishing mortality trends to changes in the model assumptions was examined. It was shown that the hake assessment model has a range of what were considered to be equally valid
Each of the solutions generated a differing perception of the trends in the stock metrics with the majority being more pessimistic of the current state of the stock than the current Working Group analysis. Figure 3.17 shows the wide difference in stock trend resulting from differences in ‘shrinkage’, the weighting given to the assumption that the selection pattern is flat topped at the oldest age. The 2001 assessment used high shrinkage producing low SSB with a shallow trend, and a high F. Low shrinkage produced a lower F and a higher SSB, with a marked peak in 1985 followed by a much steeper decline. Comparable differences are generated by changing the time period and weighting applied to commercial catch per effort data used in tuning (Figure 3.18) or by selecting different national fleet data for tuning (Figure 3.19). The sensitivity in the XSA estimates was shown to be carried forward into uncertainty in the Precautionary Approach reference points for the stock (Figures 3.20 and 3.21).
In the case of the Northern hake, due to the current catch-at-age data structure, changes to the model structure have resulted in changes in the perception of risk that may have nothing to do with any real change in the state of a stock.
Unless the structural uncertainty in the model can be resolved by the inclusion of additional information and new analysis, the interpretation of risk must be clearly linked to the XSA model assumptions and the alternative, more pessimistic alternatives considered.
These conclusions are consistent with the findings of Patterson et al (2001) who stated that:
“
Many uncertainty estimates are predicated on a single structural population model which is accepted as the 'best' representation of reality. However, in some circumstances alternative representations of reality may be almost equally plausible (whether this is expressed as an expert opinion or as a likelihood function value) and the admission of such alternative representations as possibilities may greatly affect the perceived uncertainty. Conditioning of uncertainty estimates on a single structural model may result in such underestimation of uncertainty that for practical purposes the estimates of uncertainty in forecasts so generated bear little relation to the real likelihood of alternative eventual outcomes.”“The relative performance of different management options, and some parameters also will be more robust to structural uncertainty (for example, a parameter which is expressed in relative terms spawning biomass relative to virgin biomass is more robust than absolute measures of stock size). The importance of structural uncertainty will therefore depend on the parameters which are being used for management purposes.”
The results for Northern Hake suggest that the changes in the inputs and outputs of the 2001 hake assessment may not be unique to hake, but are part of the wider problem of assessment model structure uncertainty. The Study Group concluded that the ICES Working Group on the Assessment of Southern Shelf Stocks of Hake Monk and Megrim [WGHMM] should examine in detail the sensitivity of the current management reference points to structural assumptions in the current assessment model. The review should include any additional information that can be provided on the dynamics of historic fishing effort directed towards the oldest ages and the application of alternative approaches.
1975 1980 1985 1990 1995 2000 2005
WG 2001 cv 1.0
1975 1980 1985 1990 1995 2000 2005
WG 2001 cv 1.0 Shrinkage 0.01 Shrinkage 0.1 Shrinkage 0.5
Figure 3.17 a, b. The time-series of spawning stock biomass and average fishing mortality as estimated within the XSA assessment fitted with increasing weight given to the assumption of a flat topped selection pattern at the oldest ages
SSB
1975 1980 1985 1990 1995 2000 2005 WG 2001 cv
1975 1980 1985 1990 1995 2000 2005 WG 2001 cv 1.0 WG10yrs cpue data
Figure 3.18 a & b. The time-series of spawning stock biomass and average fishing mortality as estimated within the XSA assessment fitted with a 20 year tri-cubic time-series weighting and no time-series weighting with CPUE calibration data for only the final 10 years.
SSB
1975 1980 1985 1990 1995 2000 2005
WG 2001 cv 1.0
1975 1980 1985 1990 1995 2000 2005
WG 2001 cv 1.0 Area VII Area VIII
Figure 3.19 a & b. The time-series of Spawning stock biomass and average fishing mortality as estimated within the XSA assessment fitted independently to Subarea VII and Subarea VIII CPUE data series.
5th,25th,50th,75th and 95th percentiles
Legend: WG2001 – WGSSDS 2001. WG1998 – WGSSDS1998, the assessment used to estimate the current reference points. WG cvx.x – The SSDS 2001 XSA model structure with increasing weight given to the average selection pattern at age, lower CV’s indicate more weight to the flat-topped selection pattern. Area VIII – an XSA assessment fitted to commercial data and survey information from ICES Division VIII. Area VII – an XSA assessment fitted to the commercial data from ICES Division VII.
5th,25th,50th,75th and 95th percentiles
Figures 3.21 Estimates of SSB corresponding to the intersection of the 90%ile of observed survival rate (R/SSB) and the 90%ile of the recruitment observations, derived from alternative XSA assessment model structures. Legend:
WG2001 – WGSSDS 2001. WG1998 – WGSSDS1998, the assessment used to estimate the current reference points.
WG cvx.x – The SSDS 2001 XSA model structure with increasing weight given to the average selection pattern at age, lower CV’s indicate more weight to the flat-topped selection pattern Area VIII – an XSA assessment fitted to commercial data and survey information from ICES Division VIII. Area VII – an XSA assessment fitted to the commercial data from ICES Division VII.