6.8                                             ARE BRED VECTORS THE SAME AS LYAPUNOV VECTORS?


Eugenia Kalnay1,*, Matteo Corazza1,2, and Ming Cai1

1University of Maryland, College Park, MD 20742

2INFM- DIFI, Universita di Genova, 16146 Genova, Italy




1.       Introduction


The use of ensemble forecasting and data assimilation has made apparent the importance of local predictability properties of the atmosphere in space and in time (e.g., Toth and Kalnay, 1993, Molteni and Palmer, 1993). The “spaghetti” plots and other methods used to display operational ensemble products frequently show simultaneous high predictability in some areas and low predictability in others. The regional loss of predictability is an indication of the instability of the underlying flow, where small errors in the initial conditions (or imperfections in the model) grow to large amplitudes in finite times. The stability properties of evolving flows have been studied using Lyapunov vectors (e.g., Alligood et al, 1996, Ott, 1993, Kalnay, 2001), singular vectors (e.g., Lorenz, 1965, Farrell, 1988, Molteni and Palmer, 1993), and, more recently, with bred vectors (e.g., Szunyogh et al, 1997, Cai et al, 2001).


Bred vectors (BVs) are, by construction, closely related to Lyapunov vectors (LVs). In fact, after an infinitely long breeding time, and with the use of infinitesimal amplitudes, bred vectors are identical to leading Lyapunov vectors. In practical applications, however, bred vectors are different from Lyapunov vectors in two important ways: a) bred vectors are never globally orthogonalized and are intrinsically local in space and time, and b) they are finite amplitude, finite time vectors. These two differences are very significant in a very large dynamical system. For example, the atmosphere is large enough to have “room” for several synoptic scale instabilities (e.g., storms) to develop independently in different regions (say, North America and Australia), and it is complex enough to have different possible types of instabilities (such as barotropic, baroclinic, convective, and even Brownian motion). Errico and Langland (1999) pointed out that bred vectors are indeed different from LVs, and suggested that these differences would be detrimental to the applications of bred vectors to ensemble forecasting, a conclusion disputed in the response by Toth et al (1999). In this paper we suggest that the local and finite amplitude properties of BVs make them preferable to the use of LVs for both ensemble forecasting and data assimilation. In section 2 we describe the construction of LVs and BVs. In section 3 we review the properties of BVs compared to LVs. In section 4 we indicate that the number of bred vectors needed to describe the unstable growth of perturbations is much smaller than the number of LVs with positive exponents and provide a heuristic argument to explain this observation. Section 5 is a brief summary.


2. Computation of Lyapunov Vectors and Bred Vectors


Assume that we have an evolving basic solution that satisfies the equations of a nonlinear model with a given discretization in space and time integration scheme . If the initial condition is perturbed, the linear evolution of the perturbation is given by


where the matrix  is the tangent linear model (TLM) or propagator.


2.1 Computation of Lyapunov vectors


The leading Lyapunov vector is computed as follows:

1) Start with an arbitrary perturbation of arbitrary size

2) Evolve it from  to using the TLM

3) Repeat 2) for the succeeding time intervals

After a sufficiently long time , the perturbation converges to the leading Lyapunov vector. The direction of this vector is independent of the initial perturbation, the length of the time interval or the choice of norm of the perturbation, properties not shared by singular vectors. If during the repeated application of the TLM the LV becomes too large it may be scaled down to avoid computational blow up. Additional LVs can be obtained by the same procedure, except that after each time step the perturbation has to be orthogonalized with respect to the subspace of the previous LVs, since otherwise all the LVs would converge to the leading LV.


2.2 Computation of Bred vectors


Bred Vectors (BVs) are computed as follows:


1) Start with an arbitrary initial perturbation of size  defined with an arbitrary norm. This initialization step is executed only once. The size of  is essentially the only tunable parameter of breeding.

2) Add the perturbation to the basic solution, integrate the perturbed initial condition with the nonlinear model, and subtract the original unperturbed solution from the perturbed nonlinear integration


3) Measure the size  of the evolved perturbation , and divide the perturbation by the measured amplification factor so that its size remains equal to A:



Steps 2) and 3) are repeated for the next time interval and so on. It has been found that after a short transient time of the order of the time scale of the dominant instabilities (3-5 days for baroclinic instabilities, Toth and Kalnay, 1993, Corazza et al., 2001a, a few months for a coupled ENSO model, Cai et al, 2001), the procedure converges in a statistical sense. Additional bred vectors can be obtained by choosing different arbitrary initial perturbations and following the same procedure. Therefore all bred vectors are related to the leading Lyapunov vector, since the additional BVs are never orthogonalized. As discussed further in the next section, it has been found that for global atmospheric models based on primitive equations or on quasi-geostrophic equations, as well as for other strongly nonlinear models, the bred vectors remain distinct, rather than converging to a single leading bred vector, presumably because the nonlinear terms and physical parameterizations introduce sufficient stochastic forcing to avoid such convergence.

An alternative method is “self-breeding” (Toth and Kalnay, 1997). This approach, cost-free when performing ensemble forecasting, uses pairs of ensemble forecasts to generate the perturbation at the new time:


This difference is scaled down as before, and added and subtracted to the analysis valid at . The two-sided self-breeding has the advantage that it maintains the linearity of the perturbation to second order compared to the one-sided generation of the bred vector which is linear to first order, but otherwise the procedures produce similar results.


3. Properties of the bred vectors


Bred vectors share some of their properties with leading LVs (Corazza et al, 2001a, 2001b, Toth and Kalnay, 1993, Cai et al, 2001):


·    Bred vectors are independent of the norm used to define the size of the perturbation. Corazza et al. (2001) showed that bred vectors obtained using a potential enstrophy norm were indistinguishable from bred vectors obtained using a streamfunction squared norm, in contrast with singular vectors.

·    Bred vectors are independent of the length of the rescaling period as long as the perturbations remain approximately linear (for example, for atmospheric models the interval for rescaling could be varied between a single time step and 1 day without affecting qualitatively the characteristics of the bred vectors.

·    In regions that undergo strong instabilities, the bred vectors tend to be locally dominated by simple, low-dimensional structures. Patil et al., (2001) defined local bred vectors around a point in the 3-dimensional grid of the model by taking the 24 closest horizontal neighbors. If there are k bred vectors available, and N model variables for each grid point, the k local bred vectors form the columns of a 25Nxk matrix B. The kxk covariance matrix is C=BTB. Its eigenvalues are positive, and its eigenvectors are the singular vectors of the local bred vector subspace. The local effective dimension can be measured by the Bred Vector dimension (BV-dim)


where  are the square roots of the eigenvalues of the covariance matrix. They showed that the BV-dim gives a good estimate of the number of dominant directions (shapes) of the local k bred vectors. For example, if half of them are aligned in one direction, and half in a different direction, the BV-dim is about two. If the majority of the bred vectors are aligned predominantly in one direction and only a few are aligned in a second direction, then the BV-dim is between 1 and 2. The BV-dim attains its maximum value k if all the bred vectors are pointing in different directions. This happens either in quiescent regions, where the amplitudes are small and there is little error growth, or after an integration long enough to allow perturbations to become nonlinear and very different from each other. Patil et al., (2001) showed that the regions with low dimensionality cover about 20% of the atmosphere. They also found that these low-dimensionality regions have a very well defined vertical structure, and a typical lifetime of 3-7 days.

·    Using a Quasi-Geostrophic simulation system of data assimilation developed by Morss (1999), Corazza et al (2001a, b) found that bred vectors have structures that closely resemble the background (short forecasts used as first guess) errors, which in turn dominate the local analysis errors (Fig. 2a). This structural similarity had been conjectured by Kalnay and Toth, 1994 based on the qualitative similarity between the analysis cycle and the breeding cycle. Corazza et al., (2001a) showed that the background error has a very substantial projection into the subspace of 10 bred vectors, with an angle less than 10o (a local pattern correlation of 0.985 or more) in most areas. The angle became as large as 45o only in some areas were the background error was very small.

·    The number of bred vectors needed to represent the unstable subspace in the QG system is small (about 6-10). This was shown by computing the local BV-dim as a function of the number of independent bred vectors. Convergence in the local dimension starts to occur at about 6 BVs, and is essentially complete when the number of vectors is about 10-15 (Corazza et al, 2001a). This should be contrasted with the results of Snyder and Joly (1998) and Palmer et al (1998) who show that hundreds of Lyapunov vectors with positive Lyapunov exponents are needed to represent the attractor of the system in quasi-geostrophic models.

·    The fact that only a few bred vectors are needed, and that background errors project strongly in the subspace of bred vectors, allowed Corazza et al (2001b) to develop cost-efficient methods to improve the 3D-Var data assimilation by adding to the background error covariance terms proportional to the outer product of the bred vectors, thus representing the “errors of the day”. This approach led to a reduction of analysis error variance of about 40% at very low cost.


4.  Comparison of bred vectors and LVs


We have seen that BVs and LVs share some properties but also have significant differences, namely that BV are finite amplitude, finite time, and that they have local properties in space. These properties result in significant advantages of BVs for data assimilation and ensemble forecasting.


4.1 Finite amplitude, finite time


The fact that BVs have finite amplitude provides a natural way to filter out instabilities present in the system that have fast growth, but saturate nonlinearly at such small amplitudes that they are irrelevant for ensemble perturbations. For example, Toth and Kalnay (1993) found that with amplitudes characteristic of the analysis errors for the geopotential height (between 1m to 10m), bred vectors grew by about 1.5 per day, and were similar to the NCEP operational bred vectors. With amplitudes of the order of 1 cm, the bred vectors (dominated by convective instabilities) grew 10 times faster and had maximum amplitude in the tropics. With even smaller amplitudes, James Geiger (personal communication, 2001) has found growth rates of the order of 50/day in a tropical primitive equations model. These convective instabilities saturate at very small amplitudes. If the model had enough resolution to include molecular motion, Brownian motion would provide even faster (but clearly irrelevant) instabilities. As shown by Lorenz (1996) Lyapunov vectors (and singular vectors) of models including these physical phenomena would be dominated by the fast but small amplitude instabilities, unless they are explicitly excluded from the linearized models. Bred vectors, on the other hand, through the choice of an appropriate size A of the perturbation, provide a natural filter based on nonlinear saturation of fast but irrelevant instabilities.


4.2 Local versus global representation


The local representation of BVs results in a reduction of the number of modes needed to represent a field of growing errors in the initial conditions, compared to LVs obtained by global orthogonalization. We now present a heuristic argument with a simple example of a system containing three independent regions of instability (denoted A, B and C in Figure 1). Assume that in each of them there are two unstable modes or shapes that have a similar growth rate and are therefore equally probable, one with total wave number 1 (a single high or low) and one with total wavenumber 2 (a high and a low). From a local point of view, only two bred vectors are needed to represent all possible local combinations (since each local area has two possible modes). For example, the first two bred vectors in the figure are enough to represent all possible local structures. On the other hand, the number of independent global shapes is

n = (2´2)3 – 1 = 63,

given that there are two modes in each of the three areas, and each of them can have either a positive or a negative sign. Therefore, with global orthogonalization we would need 63 global Lyapunov vectors to capture all possible directions. If we increase the number of local independent areas and/or the number of dominant modes, the number of global degrees of freedom increases very rapidly, whereas the probability of capturing all local independent shapes with a small number of bred vectors n decreases very slowly.


4.3 Comparison of BVs and LVs


As mentioned before, a comparison of the bred vectors with the forecast errors that dominate the analysis errors in the QG simulation system shows that they have very similar characteristics. Fig. 2a shows this similarity between the background error (contours) and an arbitrary bred vector (shaded) chosen as the leading LV  (Fig. 2a). Every bred vector is qualitatively similar to the leading LV. Figs. 2b and 2c show the same background error (contours) with the 10th and 100th LVs superimposed (shades). The successive LVs are obtained by orthogonalizing after each time step with respect to the previous LVs subspace. The orthogonalization requires the introduction of a norm and we have chosen the square of the potential vorticity norm. As a result, the successive LVs have larger and larger horizontal scales (a choice of a streamfunction norm would have led to successively smaller scales in the LVs). Beyond the first few LVs, there is little qualitative similarity between the background errors and the LVs.



In a system like the atmosphere with enough physical space for several independent local instabilities, BVs and LVs share some properties but they also have significant differences. BV are finite amplitude, finite time, and because they are never globally orthogonalized, they have local properties in space. Bred vectors are akin to the leading LV, but bred vectors derived from different arbitrary initial perturbations remain distinct from each other, instead of collapsing into a single leading vector, presumably because the nonlinear terms and physical parameterizations introduce sufficient stochastic forcing to avoid such convergence. As a result, there is no need for global orthogonalization, and the number of bred vectors required to describe the natural instabilities in an atmospheric system is much smaller than the number of Lyapunov vectors with positive Lyapunov exponents. The BVs are independent of the norm, whereas the LVs beyond the first one do depend on the choice of norm.

These properties of BVs result in significant advantages for data assimilation and ensemble forecasting for the atmosphere. Errors in the analysis have structures very similar to bred vectors, and it is found that they project very strongly on the subspace of a few bred vectors. This is not true for either Lyapunov vectors beyond the leading LV, or for singular vectors unless they are constructed with a norm based on the analysis error covariance matrix (or a bred vector covariance). The similarity between bred vectors and analysis errors leads to the ability to include “errors of the day” in the background error covariance and a significant improvement of the analysis beyond 3D-Var at a very low cost (Corazza, 2001b).




Alligood K. T., T. D. Sauer and J. A. Yorke, 1996: Chaos: an introduction to dynamical systems. Springer-Verlag, New York.

Buizza R., J. Tribbia, F. Molteni and T. Palmer, 1993: Computation of optimal unstable structures for numerical weather prediction models. Tellus, 45A, 388-407.

Cai, M., E. Kalnay and Z. Toth, 2001: Potential impact of bred vectors on ensemble forecasting and data assimilation in the Zebiak-Cane model. Submitted to J of Climate.

Corazza, M., E. Kalnay, D. J. Patil, R. Morss, M. Cai, I. Szunyogh, B. R. Hunt, E. Ott and J. Yorke, 2001: Use of the breeding technique to determine the structure of the “errors of the day”. Submitted to Nonlinear Processes in Geophysics.

Corazza, M., E. Kalnay, DJ Patil, E. Ott, J. Yorke, I Szunyogh and M. Cai, 2001: Use of the breeding technique in the estimation of the background error covariance matrix for a quasigeostrophic model. This volume.

Errico, R. and R. Langland, 1999: Notes on the appropriateness of bred modes for generating perturbations used in ensemble forecasting. Tellus, 51A, 442-449.

Farrell, B., 1988: Small error dynamics and the predictability opf atmospheric flow, J. Atmos. Sciences, 45, 163-172.

Kalnay, E 2001: Atmospheric modeling, data assimilation and predictability. Chapter 6. Cambridge University Press, UK. In press.

Kalnay E and Z Toth 1994: Removing growing errors in the analysis. Preprints, Tenth Conference on Numerical Weather Prediction, pp 212-215. Amer. Meteor. Soc., July 18-22, 1994.

Lorenz, E.N., 1965: A study of the predictability of a 28-variable atmospheric model. Tellus, 21, 289-307.

Lorenz, E.N., 1996: Predictability- A problem partly solved. Proceedings of the ECMWF Seminar on Predictability, Reading, England, Vol. 1 1-18.  

Molteni F. and TN Palmer, 1993: Predictability and finite-time instability of the northern winter circulation. Q. J. Roy. Meteorol. Soc. 119, 269-298.

Morss, R.E.: 1999: Adaptive observations: Idealized sampling strategies for improving numerical weather prediction. Ph.D. Thesis, Massachussetts Institute of Technology, 225pp.

Ott, E., 1993: Chaos in Dynamical Systems. Cambridge University Press. New York.

Palmer, TN, R. Gelaro, J. Barkmeijer and R. Buizza, 1998: Singular vectors, metrics and adaptive observations. J. Atmos Sciences, 55, 633-653.

Patil, DJ, BR Hunt, E Kalnay, J. Yorke, and E. Ott, 2001: Local low dimensionality of atmospheric dynamics. Phys. Rev. Lett., 86, 5878.

Patil, DJ, I. Szunyogh, BR Hunt, E Kalnay, E Ott, and J. Yorke, 2001: Using large member ensembles to isolate local low dimensionality of atmospheric dynamics. This volume.

Snyder, C. and A. Joly, 1998: Development of perturbations within growing baroclinic waves. Q. J. Roy. Meteor. Soc., 124, pp 1961.

Szunyogh, I, E. Kalnay and Z. Toth, 1997: A comparison of Lyapunov and Singular vectors in a low resolution GCM. Tellus, 49A, 200-227.

Toth, Z and E Kalnay 1993: Ensemble forecasting at NMC – the generation of perturbations. Bull. Amer. Meteorolo. Soc., 74, 2317-2330.

Toth, Z and E Kalnay 1997: Ensemble forecasting at NCEP and the breeding method. Mon Wea Rev, 125, 3297-3319.

Toth, Z., I Szunyogh and E Kalnay, 1999: Response to Notes on the appropriateness of bred modes for generating perturbations used in ensemble forecasting. Tellus, 51A, 442-449.

Figure 1: Schematic of a system with three independent low dimensional regional areas A, B, C, each of which can have two equally probable instabilities, either a single center (high or low), or a double center (a high and a low). Shown are three of the 63 possible realizations of bred vectors, with full thin lines indicating a positive center and dashed lines a negative center. Note that from a local point of view, the first two bred vectors are sufficient to cover all possible shapes, and that the second and third bred vectors are equivalent, since they describe the same local shapes. From a global point of view, on the other hand, the second and third bred vectors are independent, and we need a total of 63 independent combinations to describe all the possible global degrees of freedom.















Figure 2: Three examples of Lyapunov vectors (shaded) superimposed with the background error (contours) for the QG simulation system at an arbitrary time. a) First (leading) LV, i.e., a BV; b) 10th LV; c) 100th LV. The LV were computed using a potential vorticity norm, which leads to successively larger scales. Note that only the first LV has a structure qualitatively similar to the background error.


  * Corresponding author address: Eugenia Kalnay, Meteorology Department, University of Maryland, College Park, MD 20742-2425, USA;

email: ekalnay@atmos.umd.edu