Specifics of sources for Russian macroeconomic forecasts

Analysis of systemic biases and predictability of indicators

  • There is a fairly wide range of regularly updated public macroeconomic forecasts in the Russian market. Most of them contain inflation data and at least one economic activity indicator; other than that, the set of indicators is different. The forecast horizon does not usually exceed five years — most of the time it is two to four years. The majority of forecasts are updated semiannually or more often and contain annual data. Stress scenarios and other scenarios that differ to the baseline ones are rarely published.
  • Between a third and half of consensus survey participants in forecasts for the year ahead are relative “optimists” or “pessimists”. Their expectations have a systematic bias relative to the consensus, which in some cases correlates with the industry affiliation of the author of the forecast. Therefore, economic growth forecasts for the year ahead produced by investment companies, on average, almost always turn out to be more optimistic than the consensus forecast. These deviations are less typical for public macroeconomic forecasts, which are usually equally likely to be above or below the market consensus for most indicators. For the user this means that the use of a public forecast may protect against inadvertently accepting extreme assumptions.
  • Systemic bias relative to the fact and absolute “optimism” or “pessimism” of authors of forecasts are less common. It is possible that these deviations arise because forecasted indicators are also targets for their authors. Typical examples include real disposable income in the Russian Ministry of Economic Development’s (MED) forecast and the budget balance in the planning documents of the Russian Ministry of Finance. Users of these figures should view them not as a forecast, but rather as scenarios designed to protect against certain undesirable consequences in unfavorable conditions (budget sequestration), or as rationale for current or proposed economic policy (an experiment that assumes high efficiency of measures).
  • The average deviation of the forecast from the fact depends to a greater extent on a specific indicator than on the selected forecast source, which is due to different levels of volatility of indicators. Adjusting the baseline scenario for the maximum historical deviation of the forecast from the fact is one of the simple ways to build macroeconomic stress scenarios.

Public sources for Russian macroeconomic forecasts

Macroeconomic preconditions are definitely taken into account when forecasting the performance of a particular industry, as well as during medium and long-term planning of companies’ activities. Quantitative estimates of the main trends are used in financial and operating models calculated by companies and characterize the conditions in which they will operate in the coming years, while stress scenarios allow to conclusions to be drawn on the necessary level of reserve funds and measures to be taken to improve resilience.

Far from every company is capable of independently building various economic scenarios because the preparation of well-grounded macro forecasts is a rather time-consuming process. Nevertheless, it is still possible to satisfy the need for determining macroeconomic preconditions since there is a wide range of regularly updated forecasts in the Russian market, which are either freely distributed or offered on a subscription basis (Table 1).

Table 1. Regular sources of Russian macroeconomic forecasts1

 

Official

International organizations and development institutions

Consensus surveys

Credit rating agencies

MED

Bank of Russia

IMF

World Bank

EEU

EDB

VEB

FocusEconomics

Consensus Economics

Reuters

Bloomberg

Center of Development

ACRA

Moody's

S&P

Fitch

Indicators

Economic activity

 •

Price indices

 

• 

Exchange rates

 

 

 

 

• 

Balance of payments

 

 

• 

• 

Interest rates

 

 

 

 

 

 

 

 

 

Labor market

 

 

 

 

• 

• 

Budgets

 

 

 

 

• 

• 

Money supply and banking sector

 

 

 

 

 

 

• 

*

• 

Update frequency (times per year)

2

4

2

4

4

2

2

12

12

12

12

4

2

1

 1

Frequency of indicators (annual/quarterly)

A

A

A

A

A

A

A

A/Q

A/Q

A/Q

A/Q

A/Q

A

A

A

A

Forecast horizon (years)

3–5

3

5

2

2–3

2

3

5

 3

2

1–6

4

2

 2

 2

Alternative scenarios

+

+

 

 

 

 

+

 

 

 

 

 

+/-

 

 

 

Free distribution of data

+

+

+

+

+

+

+

paid subscription or participation

paid subscription

+

+

+

+

+

* Separate banking sector forecast is published.
Source: ACRA


1Sources are specified that have published forecasts in table format on their websites at least once a year over the past three years. Consequently, Table 1 does not include the Accounts Chamber of the Russian Federation, Expert RA or the Center for Macroeconomic Analysis and Short-Term Forecasting, which either started publishing forecasts recently or publish them at intervals of more than a year. A group of indicators is shown is present in a forecast if at least one of the indicators of the group has been disclosed over the past three years.

ACRA talks in detail about the specifics of constructing and applying macroeconomic forecasts at the following workshops: Forecasting in credit analysis. Course 1: macroeconomic and industry forecasting fundamentals and Forecasting in credit analysis. Course 2: practical aspects of economic modeling.

Most of the publicly available macroeconomic forecasts for Russia include at least one indicator that characterizes the level of economic activity (GDP growth rate, industrial production, etc.) and inflation. The other sets of indicators are different. In almost all cases, forecasts provide annual data, and the forecast horizon is usually less than five years (most often, it is two to four years). The majority of forecasts are updated twice a year or more often. Scenarios other than the baseline are very rarely provided, and pessimistic scenarios are designed to take into account the uncertainty in preconditions rather than the likelihood of a full-blown crisis (i.e. they usually do not provide an opportunity to conduct stress testing).

The only way to determine the quality of a macro forecast’s source and the degree of confidence in it is by looking at the history of previous forecasts. Besides historical accuracy, frequency of updates and the selection of indicators, sources of forecasts also differ in terms of other characteristics (including according to internal ideas about the purposes of forecasting; more on this below), and the indicators themselves are predictable to varying degrees.

Table 2. Examples of using macroeconomic indicators when forecasting the performance of sectors or companies

Macroeconomic indicator

Examples of related indicators

Real GDP growth of the country

- Growth of domestic market sales

- Growth of the population’s real wages

- Growth of the budget’s real revenues

Real GDP growth of countries

- Growth of sales of exported goods and services

- Demand from export-oriented companies

Retail trade

- Demand for consumer market goods

- Demand for intermediate goods from consumer goods companies

Inflation

- Background “indexation” of product prices in the consumer market and most ruble costs

Oil prices

- Prices for other commodities

Exchange rates

- Indexation of foreign currency costs

- Indexation of ruble costs for goods and services with a large share of import in costs

Growth of average wages

- Dynamics of labor costs

Interest rates

- Debt service expenses

Source: ACRA

Systemic bias in forecasters’ expectations

The preconditions and methods which the analyst choses when forecasting are influenced not only by market incentives or the specifics of the organization in which the analyst works, but also by his or her psychological characteristics. In a number of cases, this leads to the fact that for some indicators, the forecast steadily shifts either in the positive or negative direction. During this study, we will conditionally identify stable “optimists and “pessimists” among forecasters and explain why this factor must be taken into account when using their forecasts for practical purposes.

MED – Ministry of Economic Development of Russia

Bias of expectations relative to the fact (absolute bias). The MED’s forecasts of the growth of real GDP and household income can be seen as typical examples of this sort of bias as they are historically overestimated over a forecast horizon of more than two years (Fig. 1). This may be largely related to the fact that these indicators are targets for the MED, and therefore, assumptions about the effectiveness of stimulus or structural measures cannot be too conservative and should imply a certain improvement in dynamics. Other analysts’ forecasts for similar horizons are usually not biased or biased to a lesser extent. This observation does not at all mean that all forecasts of the MED are biased relative to the fact (for example, in terms of the nominal value of retail turnover at all forecast horizons, on average, deviations from the fact are historically close to zero), nor does it mean that all conditionally target indicators of all forecasters will definitely be subject to bias.

Figure 1. Bias of the forecasts of the Russian Ministry of Economic Development for economic growth rates and real disposable income of the population, 2005–2020*

* In the right graph, the dotted lines represent the average deviations from the fact, excluding forecasts for crisis periods (2009, 2015, and 2020).
Sources: Russian Ministry of Economic Development, ACRA

Inflation targeted by the Bank of Russia is also an important target indicator. Over the period from 2015 (when the inflation-targeting regime formally began to operate) to 2020, deviations of the medium-term inflation forecast at different time intervals from the actual indicator were both positive and negative. However, in general the Bank of Russia’s forecasts are more conservative than they are optimistic: the middle of the range of forecasts of the Bank of Russia for the next year is on average 0.36 pps higher than actual inflation (calculated December to December). At the same time, unlike the example with the growth rates of real GDP and real disposable income of the population, in this case, the bias can hardly be called systemic, since the width of the interval given in the forecast was up to 0.5 pps, and the historical spread of deviations from the fact considerably exceeded 0.36 pps (Fig. 2, left graph).

The balance of the federal budget is a kind of cross between a plan and a forecast for the Ministry of Finance of Russia as it is determined by target expenditures and projected revenues. From 2005 to 2020, in the December version of the budget for the following year, revenues and the federal budget balance were more frequently underestimated than overestimated. Consequently, the established practice of short-term planning can be called systemically conservative. This is not surprising as an unexpected sequestration is associated with significantly higher costs for the ministry than additional revenues at the end of the year (Fig. 2, right graph).

Figure 2. Deviations of forecasts for inflation and parameters of the federal budget from the actual value of the following year

* In the right graph, the deviation from the actual value was calculated in nominal monetary terms. Conversion to GDP shares is done using actual nominal GDP (actual estimate).
Sources: Bank of Russia, Russian Ministry of Finance, ACRA

Over these years, analysts from about 60 Russian and foreign organizations have participated in the survey, with more than 30 of them being permanent responders. These organizations include, in approximately equal proportions, commercial banks, investment companies, and think tanks. The survey results are not public and available only to the survey company's members and clients.

Forecast biases are not only related to the targets of their authors. Some analysts may encounter the impact of the systematic revision of statistics by Rosstat, which may lead to conservative short-term forecasts of economic activity. More details about this effect can be found in ACRA’s analytical commentary “A significant positive revision of industry data is the norm for Russia and many other countries” dated October 20, 2020.

With the exception of the examples listed above, the bias of the predictions relative to the fact is, as a rule, not absolute, but relative.

Bias relative to the consensus (relative bias). Most public and non-public sources of forecasts for a particular indicator do not demonstrate consistently positive or negative deviations from the fact; however, they are often persistently “pessimistic” or “optimistic” in comparison with other forecasters.

The monthly consensus surveys of FocusEconomics for the period from 2017 to 2021 reveal the following patterns in the forecasts concerning Russian macroeconomic indicators (Fig. 3):

- In more than 80% of surveys, analysts from investment companies are, on average, more optimistic than others in their expectations for economic growth and the federal budget balance in the next year.

- In more than 85% of surveys, analysts from commercial banks are, on average, more pessimistic than others in their expectations for economic growth in the next year.

- In more than 80% of surveys, sectoral consensus forecasts prepared by think tanks contain a lower key interest rate and weaker budget balance than the consensus forecast of all survey participants.

Figure 3. Share of surveys that showed more “optimistic” consensus forecast that the consensus forecast of all survey participants (2017–2021, forecast for the next year)*

* Explanations for the chart: the left column should be interpreted so that in 43 out of 50 surveys (86% of cases), the sectoral consensus forecast of investment companies for expected GDP growth next year was more optimistic than the consensus forecast of all survey participants; the second column means that in eight out of 50 surveys (16% of cases), the sectoral consensus forecast of commercial banks for the same indicator was more optimistic than the consensus forecast of all survey participants (and more pessimistic in 84% of cases, respectively).
Sources: FocusEconomics, ACRA

The sign of deviation of forecasts from the consensus that corresponds to the “optimistic” approach is as follows: GDP growth - plus, ruble exchange rate - upward, inflation - minus, key rate - plus, budget balance - plus.

Surveys conducted in other countries often have similar results, but the question of whether the forecaster’s industry influences the bias of its expectations against the consensus forecast remains uncertain in Russia. For a more complete understanding of the situation, the analyzed time interval should include multiple cycles of growth/decline in indicators and increase/decrease in uncertainty, and a deeper analysis of the characteristics of survey participants is also required.

A practically useful conclusion is that, regardless of the sector of forecasters, there seem to be persistent “pessimists” and “optimists” among them, and there are quite a few of them. For example, in the aforementioned consensus survey, the forecasts of 25–40% of analysts for each of the indicators are consistently above or consistently below the consensus most of the time (>80% of periods). Moreover, the degree of an analyst’s “optimism” in one indicator correlates with that in all other indicators, which theoretically makes it possible to present the degree of “optimism” as a specific feature of a forecaster that is independent of the selected indicator.

The good news is that, with rare exceptions, public forecasters’ forecasts (outside of consensus surveys) for the next year do not have a systemic bias relative to the consensus; and they almost never represent extreme viewpoints (Appendix 2).

Those analysts who are users of external forecasts should be aware of potential systematic biases in outlooks and adjust macroeconomic prerequisites expertly or choose a forecast source that allows them to obtain the required degree of conservatism in estimates.

Predictability of selected macroeconomic indicators

A longer history of forecasts for less numerous sources is shown in Table 4.

Even if forecasts contain no systematic bias relative to the fact, that is, the average deviation of a forecast for any indicator over several years is close to zero, each individual period is predicted with varying accuracy. Macroeconomic indicators are characterized by differing volatilities, which, as a rule, determines the level of their predictability: the distribution of forecast biases for each indicator depends more on the indicator itself than on the forecast source. This is confirmed, for example, by the closeness of the mean and maximum deviation moduli for different public forecasters (Table 3).

Tables 3 and 4 should not be taken as a comparison of the forecast accuracy of certain forecasters, since (1) such a comparison is highly dependent on the chosen accuracy metrics and time period, and, as shown on pages 4–5, (2) there are additional goals set by forecasters from different organizations and internal quality criteria. Rather, these tables reflect the degree of uncertainty faced by all macroeconomic forecasters in Russia, and, as a consequence, users of their forecasts.

Table 3. Deviation distribution characteristics of forecasts for certain indicators, 2017–2020, forecast for next calendar year

Indicator

Calculation type*

Mean deviation modulus

Maximum deviation modulus

Average deviation (with sign)

   

Official sources**

EADB

IMF

FocusEconomics

ACRA

Official
 sources

EADB

IMF

FocusEconomics

ACRA

Official
 sources

EADB

IMF

FocusEconomics

ACRA

Real GDP growth rate, %

A

2.2

2.1

2.2

2.0

2.0

5.0

5.0

4.9

4.7

3.8

1.0

1.1

1.0

1.1

0.5

Real disposable income growth rate, %

R

2.3

-

-

-

1.7

5.0

-

-

-

3.8

2.2

-

-

-

1.7

Real industrial production growth rate, %

A

2.4

-

-

2.6

2.7

5.2

-

-

5.1

3.5

1.0

-

-

0.7

-0.4

Retail trade turnover,
RUB bln

R

1.7

-

-

-

1.8

5.3

-

-

-

4.9

1.4

-

-

-

-0.5

CPI, % Dec/Dec

A

1.0

0.8

1.0

0.8

0.6

1.9-2.2

1.9

2.7

1.2

1.6

-0.2-0.1

0

0.2

-0.1

0.4

Urals oil price,
USD/bbl

R

26-27

28

-

-

18

43-44

55

-

-

27

-3.1-0

9.1

-

-

0.1

USD/RUB exchange rate (annual average)

R

6.3

4.6

4.1

6.5

3.8

11.3

8.7

9.1

10.2

9.0

-0.7

-4.3

-3.2

-5.5

-1.4

Unemployment rate, %

A

0.5

-

0.9

-

0.8

1.3

-

1.1

-

1.1

-0.3

-

0.1

-

0.2

Key rate, % end-of-year

A

-

-

-

1.1

1.2

-

-

-

2.7

3.0

-

-

-

0.7

1.0

Exports, USD bln

R

18-21

-

-

19

14

33

-

-

32

27

-3.7-0.9

-

-

2.9

-7.8

Imports, USD bln

R

5.8-9.5

-

-

8.5

3.4

12-16

-

-

12

6.0

0.1-2.4

-

-

3.9

-2.7

Federal budget balance,
% of GDP

A

2.9

-

-

3.7

3.1

4.7

-

-

5.1

4.4

0.2

-

-

-0.3

-0.5

* Deviations of indicators measured as percentage are calculated as a simple difference between the factual and forecasted values (absolute difference, A). For other indicators, the relative difference is calculated as a percentage of the factual value (R).
** The “Official sources” column shows the range of deviations for the entire set of Russian ministries and agencies. For certain indicators, it includes the Ministry of Economic Development, the Ministry of Finance, and the Bank of Russia.
Sources: organizations’ publications, Rosstat, ACRA

The mean deviation moduli given in the tables allow the forecast user to understand the level of uncertainty of the macroeconomic assumptions used by them and, for example, to build confidence intervals for indicators depending on these assumptions. In addition, these numbers can serve as a reference point for analysts who build predictive models for such indicators. To be considered applicable to real problems, the model must show at least comparable characteristics of deviations from the fact in out-of-sample forecasts for one year ahead. Table 3 shows, in a sense, a typical period (four-year period with one crisis year), while Table 4 characterizes the period with as many as two crises. The set of organizations in Table 4 is significantly smaller, which is primarily due to the availability of a long history of forecasts.

Table 4. Deviation distribution characteristics of forecasts for certain indicators, 2012–2020, forecast for next calendar year

Indicator

Calculation type*

Mean deviation modulus

Maximum deviation modulus

Average deviation (with sign)

   

MED**

IMF

MED

IMF

MED

IMF

Real GDP growth rate, %

A

2.1

2.2

5.0

4.9

1.4

1.1

Real disposable income growth rate, %

R

2.7

-

5.9

-

2.6

-

Real industrial production growth rate, %

A

2.0

-

5.2

-

0.7

-

Retail trade turnover, RUB bln

R

2.3

-

6.8

-

1.7

-

CPI, % Dec/Dec

A

2.8

2.7

7.9

7.6

-1.5

-0.8

Urals oil price, USD/bbl

R

30.5

-

95.3

-

11.7

-

USD/RUB exchange rate (annual average)

R

12.1

11.5

39.4

39.4

-7.0

-6.8

Unemployment rate, %

A

0.5

0.8

1.3

1.1

0.2

0.5

Key rate, % end-of-year

A

19.3

-

46.7

-

7.1

-

Exports, USD bln

R

21.0

-

75.1

-

12.9

 

Imports, USD bln

R

1.6

-

4.7

-

0.2

-

* Deviations of indicators measured as percentages are calculated as the simple difference between the factual and forecasted values (absolute difference, A). For other indicators, the relative difference is calculated as a percentage of the factual value (R).** In Table 4, the federal budget balance is shown in the MED column, although the Ministry of Finance compiles the actual forecast for the indicator, taking into account the forecast of the Ministry of Economic Development for the rest of the indicators. This approach is applied solely for the sake of a more compact data presentation.
Sources: organizations’ publications, Rosstat, ACRA

The maximum modulus of deviation from the fact is an important indicator that may be helpful in building stress scenarios, which, as seen from Table 1, are almost always missing from public macroeconomic forecasts. By adjusting the baseline forecast for the value of the maximum deviation modulus for each of the indicators, we actually reproduce the previous crises.

Appendix 1. Specifics of deviation calculations

1. General. We considered the forecast publication date to be the date when the forecast was posted on the organization’s website. To calculate deviations from the fact, we took data that was actual as of April 1, 2021. Organizations in Table 3 were selected based on the following principles: official sources plus one or two representatives from the other three groups of forecasters (generally, those with the easiest way to obtain the forecast history).

The deviation distribution parameters in Table 3 and Table 4 were selected for the most illustrative reflection of the thoughts described in this paper. Distribution parameters not listed in the tables are available upon request.

The boundaries of systematic (significant) “optimism” or “pessimism” indicated by dotted lines in Fig. 3 were chosen heuristically, without making assumptions about forecast distributions in the groups of forecasters. More in-depth research is needed to clarify the boundaries.

For calculations in Table 3 and in Fig. 1 and 2, averaging by year was carried out so that each forecast year was taken with the same weight, regardless of the number of forecasts made by an organization. This was necessary so that if the organization for some reason made fewer forecasts, for example, for the crisis year of 2020, this would not lead to a perceived improvement in the average quality of forecasts.

2. Russian Ministry of Economic Development. For calculations, a scenario was taken that was included in the calculations for the federal budget, regardless of its name. At the beginning of 2020, the ministry did not publicly update the forecast. The federal budget balance in Tables 3 and 4 is included in the MED column, although in fact the relevant forecast is made by the Russian Ministry of Finance taking into account the Ministry of Economic Development’s forecast for other indicators. This was done solely for the sake of more compact data presentation.

3. Bank of Russia. Pre-2018 forecasts were taken from the Monetary Policy Reports, post-2018 forecasts were taken from quarterly Medium-Term Forecasts. For indicators with interval forecasts, the middle of the interval was taken.

4. IMF. The forecast for the ruble exchange rate was calculated by ACRA as a quotient of the forecast for the ruble-denominated nominal GDP and the forecast for the US dollar-denominated nominal GDP.

5. FocusEconomics. To calculate the deviations in Table 3, the inflation consensus forecast (December/December) was used. In Fig. 3, for each of the forecasters and each of the sectoral consensus forecasts, the proportion of “optimism” periods was calculated based on Dec/Dec inflation through January 2019. After January 2019, if the forecaster reported its expectations for both Dec/Dec inflation and the annual average inflation, the latter was used for calculations. If expectations were known for only one of the indicators, we used this indicator. In each case, the forecaster's expectations were compared with the corresponding consensus. The classification of forecasters by sector was made expertly on the basis of a qualitative understanding of the company’s core business.

Appendix 2. Relative “optimism” of certain public forecast sources

In the graphs below, the X-axis represents the proportion of forecasts that are more “optimistic” than the consensus forecast based on FocusEconomics’ survey. To calculate the share of public forecasters, we took the forecasts made in 2017–2021 for the next calendar year (seven to 20 forecasts for each). Each of them was compared with the figures from the closest publication of the consensus forecast (generally, no more than two weeks away). The shares are marked with vertical lines. The dots show similar shares for the consensus survey participants; in contrast to the shares for public forecasts, they were calculated on the basis of 15–50 periods of participation in the consensus survey in 2017–2021. As before, the following was considered the “optimistic” deviation of forecasts from the consensus: GDP growth — plus, the ruble exchange rate — stronger, inflation — minus.

For example, the left-most gray dot in the top graph suggests that there was a forecaster among the participants in the consensus survey who expected higher GDP growth rates than the consensus rates in 0% of cases, i.e. never. In 100% of cases, its forecasts were more pessimistic than the average market expectations for this indicator. A vertical line labeled “IMF” in the same graph means that the IMF’s public forecasts for GDP growth were more optimistic than the market consensus in 32% of cases.

As in Table 3, official sources include the Russian Ministry of Economic Development, the Bank of Russia or the Russian Ministry of Finance. Their relative “optimism” is shown as a range (blue box). For example, the left border of the rectangle in the inflation graph means that there was at least one official source that gave a higher inflation forecast than the market consensus in 40% of cases. And, accordingly, the right border indicates that there was at least one source that gave a lower forecast in more than 80% of cases.

Log in

Forgot password

Sign up

Reset password

Reset password

Termsofuse

Полное использование материалов сайта разрешается только с письменного согласия правообладателя, АКРА (АО). Частичное использование материалов сайта (не более 30% текста статьи) разрешается только при условии указания гиперссылки на непосредственный адрес материала на сайте www.acra-ratings.ru . Гиперссылка должна быть размещена в подзаголовке или в первом абзаце материала. Размер шрифта гиперссылки не должен быть меньше шрифта текста используемого материала.