Forecasting electricity prices in the Swedish regulation market using random forest

(1)

Master’s Programme in Computational Engineering and Technical Physics Technomathematics Major

Master’s Thesis

Oduwole Eunice Mojolaoluwa

FORECASTING ELECTRICITY PRICES IN THE SWEDISH REGULATION MARKET USING RANDOM FOREST

Examiners: Professor Tuomo Kauranne, Dr. Matylda Jabło´nska-Sabuka Supervisors: Dr. Matylda Jabło´nska-Sabuka

Professor Tuomo Kauranne

(2)

ABSTRACT

Lappeenranta University of Technology School of Engineering Science

Master’s Programme in Computational Engineering and Technical Physics Technomathematics Major

FORECASTING ELECTRICITY PRICES IN THE SWEDISH REGULATION MAR- KET USING RANDOM FOREST

Master’s Thesis 2018

70 pages, 57 figures, 5 table.

Examiners: Professor Tuomo Kauranne, Dr. Matylda Jabło´nska-Sabuka

Keywords: electricity prices, regulation (balancing) market, random forest

Forecasting electricity prices became a very vital issue after the deregulation of the market because the predicted prices enable the market players to make appropriate decision. Price forecasts can be of great importance to grid operators whose responsibility is to keep the grid balanced. Forecasting electricity prices on the regulation power market is quite distinct from that of the spot market in that regulation prices could either be up or down regulated. Practically, only one of this regulation price would possibly be different from the spot price per time. However, irregular breakdown in the market system, makes the up and down regulation prices to diverge from the spot prices especially during the up regulation. This thesis focuses on forecasting electricity prices in regulation market using a machine learning technique random forest. Forecast was made both for regulation price direction and price difference base on different seasons such as Winter, Spring, Summer and Fall. The result was quite reliable as the forecast obtained captures original price differences although the forecast for spring and fall could not capture the sudden jumps in the prices. It is concluded that if the random forest algorithm is deeply tuned, there is high chances of obtaining a more accurate forecast performance.

(3)

PREFACE

I return all glory to Yahweh the author and finisher of my faith. I like to appreciate the scholarship board and rector (Juha-Matti Saksa) of Lappeenranta University of Technol- ogy for the opportunity given to me to study in this great citadel of learning. My profound gratitude goes to my supervisor Dr. Matlyda Jabło´nska-Sabuka for her constant support, commitment, advice and guidance in achieving the goal of this project.

My heartfelt appreciation also goes to my parents Pastor and Mrs A.O Oduwole and my siblings for their continuous support in all areas and words of motivation which has kept me going in pursuing my career goals. I love you all. I am also grateful to my pal Oyeniyi Oluwafemi for his love and encouragement. To my irreplaceable sister Adedipe Adewumi Taiwo and all friends whose names space would not allow me to mention I say a big thank you for your support.

Lappeenranta, May 25, 2018

(4)

List of Figures

1 The flowchart of the proposed methodology [1]. . . 15

2 Commercial participants and electricity exchange (Source: [2]). . . 17

3 Electrical power system represented by systems of water tanks (Source: [2]). 19 4 Seasonal profiles. . . 20

5 Weekly and intra-day profiles. . . 21

6 Shows area with surplus production/Low price where PL denote area price with maximum use trading capacity and Pcap = 0 is the area price when no transition capacity is available [3]. . . 23

7 Shows area with deficit production/High price,where PH denote area price maximum use of trading capacity and Pcap = 0 is the area price when no transition capacity is available [3]. . . 23

8 Graphical display of the features of regulating power market [4]. . . 25

9 How the regulating market works [4]. . . 25

10 An illustration of price setting in regulation power market [4]. . . 27

11 Nord Pool Spot prices area (Source: Nord Pool Spot, 2017). . . 29

12 Scatter plot of variables. . . 30

13 Graphical view of selected variables. . . 31

14 Random forest (Source: [5]). . . 32

15 Out of bag error obtained for price direction. Showing the fraction of observation belonging to all the trees against the number of grown trees implemented in the random forest algorithm. . . 40

16 Ensemble regression with the least estimated cross-validation loss for Winter 2017, with the value of the observed objective function being 3.098and the value for estimated objective function was3.198. . . 42

17 Predictive ensemble for Winter 2017, ensemble with learning rate 1.0 was used for the prediction. . . 43

18 Forecast made for regulation price difference for Winter 2017. . . 43

19 Ensemble regression with the least estimated cross-validation loss for spring 2017, with the value of the observed objective function being2.456 and the value for estimated objective function was2.456. . . 44

20 Predictive ensemble for Summer 2017, ensemble with learning rate 0.5 was used for the prediction. . . 44

21 Forecast made for regulation price difference for Spring 2017. . . 45

22 Ensemble regression with the least estimated cross-validation loss for Winter 2016. . . 46

(7)

24 Ensemble regression with the least estimated cross-validation loss for

Spring 2016. . . 47

25 Predictive ensemble for Spring 2016. . . 47

27 Ensemble regression with the least estimated cross-validation loss for Summer 2016. . . 48

28 Forecast made for regulation price difference for Summer 2016. . . 49

29 Predictive ensemble for Fall 2016. . . 49

30 Forecast made for regulation price difference for Fall 2016. . . 50

33 Ensemble regression with the least estimated cross-validation loss for Spring 2015. . . 52

34 Predictive ensemble for Spring 2015. . . 52

37 Predictive ensemble for Summer 2015. . . 54

43 Ensemble regression with the least estimated cross-validation loss for Spring 2014. . . 57

50 Predictive ensemble for Winter 2013. . . 61

(8)

52 Ensemble regression with the least estimated cross-validation loss for

Spring 2013. . . 62

(9)

List of Tables

1 Estimates for the four seasons in 2017 . . . 42

(10)

LIST OF ABBREVIATIONS

ANN Artificial Neural Networks.

GARCH Generalized Autoregressive Conditional Heteroskedasticity.

CHP Combined heat power.

LASSO Least absolute shrinkable selection operation.

RF Random forest

ELBAS Electrical Balancing Adjustment System Elspot Electrical Spot

s.t such that

LLN Law of Large numbers

(11)

1 Introduction

1.1 Background of study

Electricity price forecasting has become a vital area of study after the global restructuring, deregulation and the introduction of competitive market in the power industries. Initially before deregulation, electricity prices were normally regulated, and predetermined tariffs were provided to the buyers. The effort to design an effectively operating competitive market that gives participants an accurate incentives was to enhance production efficiency and put a limit to market power. Thus, in the deregulated electricity market, greater autonomy is given to the market players [6].

Presently about 20 countries trade on the markets in the Nordic and Baltic regions where electricity is sold in a common power exchange controlled by Nord Pool Spot AS [7].

This resulted from the deregulation of the Nordic market which commenced in Norway in early 1990s [8]. The Nord Pool Spot major market place is Elspot also known as the day-ahead auction market. In accordance to Nord Pool Spot [9], ”the power exchange serves the society by ensuring that the wholesale electricity trade has transparent pricing, the spot price is used as a reference price in the electricity derivative market and the quotes for long-term contracts shows the expectation for future prices” [10].

Since electrical power is a distinct commodity that it is not economically storable, there arises a need to ensure that there is an equilibrium between the quantity of power supplied into the grid and the demand, such that we have a stabilized power system [11]. The rate of consumption of electricity is dependent on various factors, which could vary from weather (which is a major driving force of demand) to intensity of business and the activity of the day (working days and weekends, holidays, on and off-peak hours). These factors make electricity prices exhibit seasonality, major volatility and unanticipated significant price spikes. Therefore, traders, producers, and retailers rely on the forecasted values of electricity price to make decisions on the strategy "buy" and "sell" bids to broadcast for each trading hour in the power trading market.

At the end of the bidding, the bid prices are fed into the prices determination system [12].

For selling offers, prices are selected in a descending order while for the buying offers, prices are selected in an ascending order until there is an equilibrium between the volume and price. Thus, accurate forecasting of electricity prices is vital for each entity to be involved in the bidding process. Industries in the power exchange also need an accurate

(12)

forecast of electricity price to be able to optimize the use of their portfolio through bidding and hedging against price volatility and making maximum profit. According to studies, electricity price forecast is classified into three types depending on purpose of use, time horizon and resolution [13]:

1) Long term price forecasting lasst for several years. It can be used for investment planning and profitability analysis. Such as determining a new site of power plant.

2) Mid-term price forecasting is mostly used for balance sheet forecast and derivative pricing. It focuses on producing probability distributions of future estimated prices over a certain period. It spans from 7 days to a year. [14].

3) Short-term price forecasting which is the category which this study belongs to.

It is a type of price forecast that is of crucial interest to players of auction-type spot markets whose bids quotation are expressed in terms of quantity and price. It forecasts prices in less than 24 hours [15].

There are several approaches that have been proposed by researchers for forecasting electricity prices. The methods include linear regression models [6], artificial intelligence methods which capture non-linear and complex effects [16], autoregressive moving average exogenous variables [17], fuzzy methods [18], Kalman filtering technology which has been applied to short-term price forecasting [19], GARCH models have been used in original price series to simulate price spikes and hybrid models [20]. However, the problem with these statistical approaches is that they require more time to handle a huge amount of variables and they might not effectively represent the non-linear characteristics of complex prices. There are still some short comings using artificial neural networks (ANN), because there is a need to manually determine parameters and structure additionally, its convergence is very slow in training. The GARCH model failed to incorporate spikes like the height which are always seen in the original prices.

Although there has been quite a huge number of studies carried out in the day-head electricity price forecast, so far there has not been much research in the intra-day electricity price forecast. As such, this study contributes to the few existing researches in intra-day market. In this research, we aim at predicting electricity price in Nord Pool’s regulated market using Random forest model which is a type of machine learning technique that is useful for both classification and regression. It is a randomized ensemble of decision trees which works by combing multiple weak classifiers in order to build a strong one. We shall incorporate into our model adequate variables that influences electricity prices and with a prior knowledge that weather has a key effect on electricity price we consider using

(13)

weather data from weather stations rather than an aggregated data that has been used of in previous research. Random forest is advantageous in feature selection: it randomly creates various small decision trees to explain the data [21]. Its random characteristic assemblage boosts the diversity of the system and enhances the capabilities [22].

1.2 Objective of study

In this study, the goal is to make a forecast of regulation prices in the Nord Pool’s regulated Elbas ( intra-day) market using a random forest model. With this model we aspire to contribute to the operation and trading choice in the market. Thus, the objectives are to:

1. Identify the key variables influencing regulation price.

2. Implement a random forest classification algorithm to forecast the regulation price direction in each trading hour.

3. Implement a random forest regression algorithm to forecast the regulation price difference in each trading hour.

1.3 Literature review

Electricity pricing has gradually become a new emerging area of research, relative to electricity demand forecasting and electricity load forecasting problems.

Cheng applied random forest based ensemble system to make prediction of short-term electricity load using the real load data set from New York and PJM Interconnection. In contrast to other researches where only relevant features are considered for modelling load forecast, predictors were trained using the dataset with various features randomly selected from the original feature set. With the use of average fusion method, decisions are merged and used to build their model to ensure no information was lost. They made a comparison of performance of the method with the generalized regression neural network (GRNN) and back propagation neural network (BPNN). The analysis of their result shows that based on accuracy and stability, random forest performs better than other methods [22].

Singhal used artificial neural networks (ANN) to forecast electricity prices in deregulated open power markets. A neural network method was implemented to predict day-ahead

(14)

market clearing prices (MCPs) for power markets. "The structure of the neural network was a three-layer back propagation (BP) network". In their work they made use of prices from history and other estimated features (such as temperature and load forecast) in the future to "fit" and "extrapolate" the prices and quantities. An artificial intelligent method which combines fuzzy and recurrent neural networks (RNN) was used to forecast location marginal price (LMP). Eight-month historical electricity prices were collected, and the ANN was trained using six-month data. Days with normal trend, small spikes and large spikes were tested for each month. Their result showed that the model was efficient for normal trend days but could not capture days with price spikes [12].

Zhang et al. made use of game-theory based models, their focus was on the effect of bidder attitude on electricity prices. The result showed that electricity prices are related to the "pricing" and "bidding" scheme of market players [23].

Barlow studied a diffusion model for electricity pricing. He proposed a version of the modified geometric Brownian motion as a jump diffusion model for stochastic modeling of electricity prices. The finding shows that in terms of performance, the geometrical mean reverting jump-diffusion models gives an accurate result and models with no jumps seems to be unsuitable for electricity price modelling. However, the drawback of this method is that it is tedious to incorporate physical characteristics of a power system into mathematical models as it can lead to contradiction between the real power market and model outputs [24].

Voronin worked on deriving an approach which will be able to predict a day-ahead electricity prices with the incorporation of price spikes. The price forecasting methodology was formed using a hybrid model shown in Figure 1

(15)

HISTORICAL ELECTRICITY

PRICE DATA HISTORICAL AND FORECASTED

EXTERNAL DATA

DETERMINE SPIKES WITH STATISTICAL METHODS

DATA PRE-PROCESSING

GMM MODEL

(S)AR(I)MAX+GARCH MODEL

(S)AR(I)MAX+GARCH MODEL FORECAST

NN

KNN MODEL

OVERALL ELECTRICITY PRICE FORECAST NORMAL RANGE WITH

PRICE VALUE PREDICTION MODULE

PRICE SPIKES VALUE PREDICTION

MODULE

Figure 1.The flowchart of the proposed methodology [1].

The reclusive search method used in the approach was formed to fine tune the variables and choose the best input set of independent variables. Their numerical solution showed that the approach displays a more precise behavior, in comparison to earlier methods that have been carried out. It provides substantial and useful information for players of the day-head energy market [25].

Lu et al. examined the cause for price spikes using the approach of Bayesian catego- rization and similarity searching. A framework of data mining based electricity price forecast was proposed, capable of predicting prices along side price spikes. The study paid more attention to spike directions and investigated the cause of price spikes taking into account supply–demand balance index (SDI) and relative demand index (RDI). The approach shows the connection between the demand and supply of electrical power. From the mining result, a price spike forecast model was formed, the model was used to produce

(16)

predictions for price spikes, level of spikes and associated forecast confidence levels [1].

Ludwig et al. considered least absolute shrinkable selection operation (LASSO) and random forest for feature selection to forecast electricity prices. They proposed a model that can be used for hourly day-ahead electricity prices. In their analysis, attention was on selecting appropriate exogenous variables thus, LASSO and random forest was used for selecting the variables from different weather stations. Their model was applied to Ger- man weather data, and it was able to improve forecast accuracy by 16.9 %in terms of mean error [26].

Knapik studied the impact of basic price drivers in the forecast of price jumps in Nord Pool intra-day market. They formed a categorical time series model putting into consideration prices drivers, persistence and seasonality of electricity prices. Their result revealed that high loads have a notable effect on the occurrence of price spikes and an increase in price drop can be linked to low loads [27].

1.4 Structure of the study

The remaining chapters of this study are arranged as follows:

Section 2 gives information about the Nordic electricity market. Section 2.1 introduces the commodity (electricity) under study. Section 2.2 provides information about how the Nord Pool works, with detailed information about the Nordic power grid, electricity consumption, the Elspot market and how the electricity price is calculated. Section 2.3 introduces the regulation power market, the functionality of the Elbas market and gives instances of up and down regulation of electrical power. In Section 3 the source of the data used in the study is discussed, the methodology used in the study is discussed in Section 4. Section 5 presents the result of the forecast and Section 6 provides the concluding part of the study.

(17)

2 The Nordic electricity market

2.1 Electricity- a distinct commodity

After the liberation of electricity market, electricity became a commodity just like cocoa, grains etc. In the beginning like every other market, there exists wholesale and retail market with three market participants which include producers, retailers and consumers. But in the case of electricity, the market rapidly expanded into an advanced trading system and thus some new participants have emerged such that traders and brokers were introduced into the market where electricity is owned by the traders at the process of trading, brokers operate as an intermediary between the trader and consumers [2]. A graphical view is shown in Figure 2.

PRODUCERS Energy Companies

Generators TRADERS

CONSUMERS Power Intensive idustry

Distributors

BROKERS

Trade power on physical and financial market

Electricity Exchange

Figure 2.Commercial participants and electricity exchange (Source: [2]).

There are various characteristics that make electricity distinct from every other commodity. It is not economically storable and as such it has to be consumed concurrently as it is been produced. Hence, it becomes vital to maintain an immediate balance between the electricity that has been produced and demanded for in the electricity market. The non-storable characteristic of electricity also makes reserve margin a necessity for an electricity power system [25]. Unlike in the classic economic theory where commodi- ties can be distinguished in reference to the brand or product, electrical energy can only be distinguished with respect to the source (such as hydro, thermal wind, power), power

(18)

quality features or voltage level, yet there is still no way to physically trace the producers that generated a particular quantity of electricity delivered to a consumer. Thus, electricity can be said to be a uniform commodity. Additionally, electricity has an inelastic demand that is a sudden rise in the price would not have an effect on its demand because there is no other commodity that can serve as a substitute.

2.2 Nord Pool

Nord Pool runs the leading electricity market in world, it functions in both Nordic and Baltic zones in Europe. After the deregulation in early 1990s, Nordic electricity market was established in 1993 and it developed from different national power markets to a multi-national power market [28]. In 1991 upon the introduction of competition and different electricity transmission from generation of electricity by the Norwegian Energy Act, Norway took the first action towards deregulation. Thus, in 1992 Norwegian market for electric power exchange was established [29].

In 1996, ratification of competition in electricity was made effective in Sweden and they became part of the Norwegian electric power market which stood as the first international electricity exchange Nord Pool. Finland became a full part of Nord Pool 1999 after the removal of border tariffs, Western Denmark also joined in the same year while Eastern Denmark stayed out until 2000. However, the Danish market was only fully integrated in Nord Pool after the removal of border tariff in 2002 [28]. The main aim of Nord Pool is to maintain an equilibrium between the production of electricity and the demand for electricity at an accurate and optimal price. Nordic power exchange consists of Elspot (day-ahead) and Elbas (intraday).

2.2.1 Nordic power grid

Nordic power grid can be divided into national main grids, regional transmission network and local distribution networks. Power system of countries at close proximity are connected using transmission line between main grids [10]. The grid is owned by the Nordic transmission system operators (TSOs): Statnett SF, Svenska kraftn¨at, Fingrid Oy, Energinet.dk and the Baltic transmission system operators Elering, Litgrid and Augst- sprieguma tikls (AST) [7]. Transmission system operators (TSO) are non-commercial companies, neutral and independent of commercial players [2] that manage and ensure there is stability in the transmission grid.

(19)

The Nordic power market can be considered as a connection of water tanks, in which electrical power is depicted by water and the connecting pipes represent the transmission lines see Figure 3 . Electricity producers channel power into the tank and consumers tap power from it. For instance there is a possibility for a retailer at Northern part of Sweden to purchase electrical power from a producer at Southern part of Sweden, in this case all that would be needed is for the producer to channel into the grid the unit of electricity that matches the quantity demanded by the retailer [3]. As long as the water surface remains at a steady level, it implies that there is balance between quantity demanded and quantity supplied.

In other to deal with transmission congestion, the market uses the price areas as an economic tool with each individual area having sufficient transmission and distribution capacity. However, if there arises transmission bottlenecks between these pricing areas, Nord Pool raises the price of electricity in the area with lower electricity supply by an amount that will cause a drop in demand and match up with the available electricity supply [10].

Figure 3. Electrical power system represented by systems of water tanks (Source: [2]).

2.2.2 Consumption

Consumption of electricity in Nordic the countries is high due to the popularity of electrical heating as a result of winter and power intensive companies [30]. The biggest sources of consumption differ from one country to another, however it mostly follows variation in temperature and economic growth. In 2016, the total amount of electrical energy consumed in Sweden, Finland, Norway, Denmark was about 355 TWh [31] with industries having the main consumption. In Finland and Sweden, pulp and paper are the highest power consuming companies in contrast to Norway where petrochemical and metal companies are the largest consumers.

As depicted in Figure 4 and 5 from the Swedish data, the consumption shows a periodical

(20)

pattern. It can be seen from the figure that there exist seasonal trends due to large electrical heating usage which is dependent on changes in temperature, so as the temperature decreases, the rate of consumption of electricity also increases and vice versa. At working hours the consumption is lower than over the weekends because most companies are not active. The same pattern can be seen during public holidays. Also, there exists a strong intraday profile. The morning and evening peaks are seen due to the arrival of people at offices and a rise in household electricity consumption upon the arrival of people from the various offices [30].

Jan 2017 Feb 2017 March 2017 30 April 2017 Months

1500 2000 2500 3000 3500 4000

Consumption (GWH/week)

-30 -25 -20 -15 -10 -5 0 5

Temperature(Celcius)

Seasonal profile

Consumption Temperature

Figure 4.Seasonal profiles.

(21)

20.3.15 21.3.15 22.3.15 23.3.17 24.3.17 25.3.17 26.3.17 27.3.17 Days

900 1000 1100 1200 1300 1400

Consumption (GWH/week)

Weekly and Intra-day profiles from 20.02. - 27.03.2017

Figure 5. Weekly and intra-day profiles.

2.2.3 Elspot market

Elspot serves as the major market in Nord Pool. It is also know as the day-ahead market which connotes that each day the delivery of electricity of every hour of the succeeding day is being traded, each market player submits their bids stating the volume and price of electricity the are interested to purchase or sell in the succeeding day’s trading hours.

Players in the Elspot do not have any knowledge as regards bids submitted by other players in the market since the market is a closed auction. There is possibility of submitting three types of bids in the day-ahead market [7]:

1. Hourly bids: Specifies the intention of a player to purchase or sell a quantity of electrical power from an agent in a particular area at various bid prices for specific hour of the day. Most of the trading conducted in the Elspot market are matched depending on the single hourly bids.

2. Block bids: Specifies the intention of a player to purchase or sell a quantity of electrical power at a specific amount for at least three consecutive hours. It is required

(22)

that the player submits the price and volume of electrical power proposed for the block and periods when the trading might occur. Block bid is also subdivided into four:

• Profiled

• Linked

• Regular

• Curtailable

3. Flexible hour bids: It avails the market player ample chances to make extra bids of sale (there is no chance to buy) of electricity at any hour of the day depending on the bid price and situation of the market at each hour [32].

2.2.4 Nord Pool price formation

TSOs divides the Nordic and Baltic area into different bidding regions so as to manage congestion in the transmission grid [4]. The bidding regions can either have an excess, shortage or balance supply of electrical power. Electrical power would flow from regions with lower prices to regions with higher prices due to a high demand.

The insufficiency of the transmission grid capacity in the Nordic region causes an hin- drance in the uniformity of prices. Taking into consideration the capacity of the transmission grid between various bidding areas, area pricesare determined base on the demand and supply curves aggregated for the particular bidding regions as shown in Figure 6 and 7.

There is a surplus production in the area in Figure 6 and shortage of production in Figure 7 when the price of electrical power is equivalent to that of the system price. If the quantity of electrical power needed to be exported from a surplus area to a deficit area is greater than the transmission capacity, then there is no possibility of meeting the entire power demand in the deficit area. In this situation, the selling curve (Figure 7) is shifted to the right such that the new area price is read on the vertical line at the point of intersection of the buying and the new selling curve. Thus, the price in the deficit area would be greater than the system price. If a situation occurs in an area with surplus production, where the needed quantity of electrical power export is more than the transmission capacity, the area price of the surplus area would fall lower than the system price. The import and export of electricity between the deficit and surplus area are always equal.

(23)

Area prices are always identical whenever the flow of electricity between the bidding regions are within the transmission capacity boundary set by the TSOs.

Buy Sell

PL PCap = 0

Turnover including export Volume (MW) Price

Figure 6. Shows area with surplus production/Low price where PL denote area price with maximum use trading capacity and Pcap = 0 is the area price when no transition capacity is available [3].

Price

Buy Sell

Turnover including Import PCap = 0

PH

Volume (MW) Figure 7. Shows area with deficit production/High price,where PH denote area price maximum use of trading capacity and Pcap = 0 is the area price when no transition capacity is available [3].

(24)

Upon the completion of each trading cycle in the Elspot market, system price is being calculated for each hour of succeeding day. The system price is an unrestricted reference price that does not take into consideration the transmission limitation that exist between bidding regions. If the transmission capacity that is available does not limit the delivery of electrical power, the system prices are equal to the real prices in every bidding region. To obtain the system price for each trading hour, a combination of bids submitted for buying and selling for every hour of the succeeding day is used to build the demand and supply curve [32].

2.3 The Nordic regulation power market

As the proportion of wind and various other renewable power generation methods grow in the European power supply there arises uncertainty in the predictability of power production. In most cases there always arises some unforeseen fluctuations between production and consumption causing the market to be unstable. But for the effectiveness of the market there always arises a necessity for stability in the power system. The Nordic spot market closes daily at noon. At each closing time the demand and supply are balanced and arrangements for hourly delivery are made for the following day. There is at least a time interval of 12 hours between when bids are submitted and when the real trade takes place. The regulated power market is a well stabilized market which is capable of handling all unpredictable differences between real and planned exchange in the delivery phase at a moment’s notice. The stability demand on the spot market guarantees that the predictable aspect of the difference between the participant’s energy supply and obliga- tions are outbalanced in the spot market [9]. Bids for the delivery of regulating power are compiled in the Nordic NOIS-list, thereafter, they are arranged systematically such that increasing prices are used for up-regulation (greater than the spot price) and cheapest prices for down-regulation (lesser than the spot price). The bids are used to strike a balance between consumption and production. The regulation market is flexible in that there is possibility to submit, adjust and remove bids till 60 minutes prior to operation hour.

Regulating power market is controlled by Transmission System Operation. A graphical illustration of how the regulating power market works is shown in Figures 8 and 9.

(25)

Bids for up and down regulation

Common Nordic merit order list

Swedish and Norwegian TSOs maintain the grid

Activate bids Request for

up or down regulation Consumption

Production

50 Hz Local TSO

treat bids

Market players with regulating resources

Local TSOs pass bids on

Figure 8.Graphical display of the features of regulating power market [4].

Operational reliability Ancillary services Regulayiong power market Balance control and settlement Organization and opeartion

Finanncial market Elspot

Elbas

Electricity exchange TSOs

Bids to Elspot and Elbas markets

Reguest for up and down regulation Producers

Distributors Grid owners Traders Brokers Large industries

Production schedules Bids to the regulating power market Market players

Grid Capacity

Spot trading

Figure 9. How the regulating market works [4].

(26)

2.3.1 The Elbas market

Elbas market also known as the intra-day market serves as a secondary market for the Elspot. It is a continuous market where power trading takes places till an hour prior the delivery time. It makes it possible for market players to minimize the expected imbalances by adjusting their power generation or consumption strategy close to delivery. The intra- day trading is regarded to be more profitable by market players based on various reasons such as:

1. There is possibility of minimizing imbalance cost which market players are sub- jected to when producing a greater or lesser power than they have planned. The disparity of costs can be a vital incentive needed for the entire market players to suitably compute precise predictions for production and consumption, also to plan and trade relying on these predictions. Additionally, they can hedge against unforeseen price disparity which can swerve remarkably from the spot prices by minimizing imbalance volumes [33]. From the study carried out amid Sewdish balance accountable panels, it was discovered that the minimization of imbalance cost can be considered as the major inspiration for intra-day trading in Sweden [34].

2. To reduce production cost of producer generating power in their personal plant which can be expensive to run, they can consider buying power from other sources so as to optimize their consumption and production schedules.

3. It can also be used to proffer flexibility in private power generation or consumption to other market players that are ready to offer more than the cost of having to run and reschedule the corresponding power plant.

In Europe, there are basically two main types of intra-day trading namely:

1. Continuous trading: It allows trades to be settled anytime an offer is accepted between two or more market players. Thus, there is variation of prices depending on the trade. This is distinct from discrete auctions which are cleared at discrete times. This type of trading is advantageous because it gives the market player ben- efit to trade whenever there is a chance to make profit [35]. For instance, it can be beneficial to a risk-averse player that is interested in reducing price risk as soon as possible from any anticipated imbalances. Also, it permits market players who encountered increasing price the later to reschedule and offer a flexible intra-day

(27)

price at earlier time [33]. The disadvantage of this type of trading is its first-come first-serve principle that brings about inefficiencies.

2. Discrete trading: It limits trading to pre-established times.

2.3.2 Up and down regulation

Up-regulation occurs when generated power is less than consumption. In this situation, the frequency of the electric current in the grid presented in Figure 3 drops below50Hz and there arises need for the TSO to ensure that more electrical power producers supplies more electricity to the grid such that the TSO purchases more power from the producers who has announced surplus generation capacity. Also, there can also be a down regulation scenario in which the generated power exceeds consumption. Here, the frequency in the grid exceed50Hz and in order for the TSO to strike a balance, he therefore needs to make sure some producers reduce their production. To achieve this he sells electrical power to the producers [2].

An example of how prices are set in the regulating power market is demonstrated in Figure 10 where the blue and magenta rectangles depicts the up-regulation and down regulation demand respectively.

Volume (MW) 500 MW Up-regulation

Market Price Up

Regulation price

Price (EUR/MWh)

Up-regulation Down-regulation

Figure 10.An illustration of price setting in regulation power market [4].

(28)

If there is a demand for 500 MW up-regulation, the demands with the least prices are activated till500MW is attained then the up-regulated price is fix to be equal to the price of the last up-regulated MW. The same procedure applies to the down-regulated price. It can be assumed that when demand for electrical power is higher than production, hydro power and gas stations can be easily regulated to offer a large amount of regulating power.

At any given timet, the demand for regulating power, RP_t can be mathematically calculated as the sum sum of the difference between the actual production plan and the new production plan

RP_t=X

p∈P

AP_p,t−N P_p,t, (1)

P denotes all producers in the systemAP_p,twith the actual production at timetandN P_p,t denoted the new production plan at timet.

(29)

3 Data

The source of the data used for this work is gotten from Nord Pool ’s Spot. The Nordic market is subdivided into fifteen bidding regions, four in Sweden, five in Norway, one in Eastern Denmark and Western Denmark, Estonia, Finland, Lithuania and Latvia respectively. The present geographical structure of the Nord pool Spot market is shown in Figure 11.

Figure 11.Nord Pool Spot prices area (Source: Nord Pool Spot, 2017).

Specifically the electricity prices that are considered for our analysis is the Swedish weather data and Swedish intra-day prices comprising of the four bidding areas from January 1^st, 2013 to June 30^th, 2017 with a total number of 39480 observations. Missing values in the data set were replaced by preceding values. The Swedish power system is an integrated part of the Nordic and there exists a continuous interchange of energy among the counties. Over eighty percentage (80%) of electrical power production in Sweden is based on hydro power and nuclear power while less than ten percent (10%) is from wind power and combined heat and power (CHP) [36].

3.1 Choice of variables

Since our data comprises a huge number of predictor variables, there is a need to ascertain the variables which are vital to obtain an accurate forecast. We fit the model and based on

(30)

the correlation coefficients and the p-values (with a significance level of 0.05), variables which are seen to correlate with regulation price difference are:

1. Electricity production prognosis 2. Production difference

3. Wind power production 4. Temperature

Figure 12 shows a scatter plot between the price difference and these variables with the correlation coefficients and p-values and Figure 13 shows a visualization of the selected variables.

Figure 12.Scatter plot of variables.

(31)

0.5 1 1.5 2 2.5 3 3.5 10⁴ 0

50 100 150

Regdiff

0 1000 0

2 10⁴

0.5 1 1.5 2 2.5 3 3.5

10⁴ 2000

4000

Prodprog

2000 0 5000

0.5 1 1.5 2 2.5 3 3.5

10⁴ -1000

0 1000

Proddiff

0 0 2

10⁴

0.5 1 1.5 2 2.5 3 3.5

10⁴ -20

0 20

Temp

-20 0 20 0

5000 10000

0.5 1 1.5 2 2.5 3 3.5

10⁴ 0

5 10

Wind

0 5 10 0

10000

Figure 13.Graphical view of selected variables.

(32)

4 Methodology

4.1 Random Forest

Similar to the literal definition of a forest which is a collection of a large group of individual trees. Random forests (RF) is a type of machine learning technique which can be used for both classification and regression. It is composed of an ensemble of individual trees. It was developed by Leo Breiman [37] and Adele Cutler for constructing an ensemble based predictor with a collection of un-correlated decision trees which grows in a subspace of features that are chosen randomly. Random forest is a classifier that is composed of an ensemble tree-shaped classifiers{g(w, φ_q),∀q= 1,2,3, . . .},{φ_q}denotes random vec- tors independent and identically distributed with polls casted by every single tree in the forest for the most occurring class at inputw[38]. Figure 14 gives a pictorial illustration of how Random Forest works.

Figure 14.Random forest (Source: [5]).

(33)

4.1.1 Characteristic of random forest

• It is efficient in handling a big data set and visualizing for high dimensional data.

• It can manage a substantial amount of input variables without removal.

• It has an outstanding accuracy and admits several current algorithms.

• It automatically detects useful predictors.

• It provides applicable internal error estimates, correlation,robustness and pinpoint important variables [39].

• They can be simply parallelized as a result of the independent nature of the trees.

• It has an effective technique capable of finding an estimate for a large portion of omitted data values without affecting its accuracy.

• It provides an experimental approach for detecting interaction between variables.

4.2 Convergence of random forest

From a collection of different classifiers say g₁(v), g₂(v), g₃(v), . . . , g_q(v), and with randomly selected training trees from the distribution of random vectorsW, the margin function is defined byV is

mg(V, W) =av_qI(g_q(V) =W)−max

l6=W av_qI(g_q(V) =l).

I(·)denotes an indicator function. The rate to which the average number of polls from the exact classes surpasses the average poll from other classes is measured by the margin function. Thus, the greater the margin, the more we become assured of the classification.

The generalized error is given by

GE^∗ =G_(W,V₎(mg(W, V)<0)

subscript W, V denotes that the probability is over the W, V space [40]. In a random forest, g_q(V) = g(V, φ_q). For a huge ensemble of trees, using the Strong Law of Large Numbers (LLN) and tree shape it follows that:

(34)

Theorem 1. The larger the number of trees, all sequencesφ₁, . . . , GE^∗ converge almost surely to

G_(W,V₎(G_φ(g(V, φ) = W)−max

l6=W G_φ(g(V, φ) =l)<0)

Proof. We want to prove that there exist a set of probability zero U on the sequence space φ₁, . . .for allw, outside ofU ,

1 N

N

X

n=1

I(g(φn, v) =l)−→Gφ(g(φ, v) =l).

For a constant number of training data set and constant φ, for all ws.t g(φ, v) = l is a union of hyper-rectangles. For all g(φ, v) 3 a limited number of Q of such unions of hyper-rectangles, represented asX₁, X₂, . . . , X_q. Letδ(φ_n) = qif{w : g(φ, v) = l} = V_q. LetN_qdenote the occurring scenarios whenδ(φ_n) = lin the firstN number of trials.

Thus,

1 N

N

X

n=1

I(g(φ_n, v) = l) = 1 N

X

q

N_qI(v ∈X_q)

By LLN ,

N_q = 1 N

N

X

n=1

I(δ(φ_n) = q)

converges almost surely to G_φ(δ(φ_n) = q). For some values q the union of all set of nonoccurence of convergence results in a setU of zero probability s.t outsideU,

1 N

N

X

n=1

I(g(φ_n, v) =l)−→X

q

G_φ(δ(φ) = q)I(v ∈X_q).

the R.H.S isG_φ(g(φ, v) = l[38]

4.3 Correlation and strength of random forest

For RF, an upper bound can be obtained for the generalization error in terms of two variables which are estimates of the accuracy of each classifier and dependence between them. The interaction between these two variables gives the basis for understanding how

(35)

random forest works.

Definition 4.1. Margin Function [40]

For RF, margin function is defined as:

mf(V, W) =G_φ(g(V, φ =W)−max

l6=W G_φ(g(V, φ=l),

and the strength of the classifier{g(V, φ)}is given by x=E_V,Zmf(V, W).

With the assumption thatx≥0, Chebychev’s inequality yields GE^∗ ≤ var(mf)

x² . (2)

Definition 4.2. Raw margin function [40] Raw margin function is defined by rmf(φ, V, W) = I(g(V, φ) = W)−I(g(V, φ) = ˆl(V, W)).

Hence, the expectation ofrmf(φ, V, W)with respect toφismf(V, W). For any function z the identity

[E_φz(φ)]² =E_φ,φ^‘z(φ)z(φ^‘)

holds with φ, φ^‘ been independent of each other and from similar distribution, this then implies that

mf(V, W)² =E_φ,φ^‘rmf(φ, V, W)rmf(φ⁰, V, W) (3) From Equation (3) variance of mf gives

Var(mf) =E_φ,φ‘(Cov_V,Wrmf(φ, V, W)rmf(φ⁰, V, W)) (4)

=E_φ,φ^‘(ρ(φ, φ^‘)sd(φ)sd(φ⁰)

ρ(φ, φ^‘)denote the correlation betweenrmf(φ, V, W)andrmf(φ⁰, V, W), keepingφ, φ^‘ constant andsd(φ)denotes the standard deviation ofrmf(φ, V, W)keepingφ constant.

Thus,

Var(mf) = ¯ρ(E_φsd(φ))²

≤ρ(E¯ _φVar(φ) (5)

(36)

¯

ρdenote the mean value of the correlation i.e.

¯

ρ= E_φ,φ‘(ρ(φ, φ^‘)sd(φ)sd(φ^‘) E_φ,φ‘(sd(φ)sd(φ^‘)

Write

E_φVar(φ)≤E_φ(E_V,Wrmf(φ, V, W))² −x²

≤1−x² (6)

Combing Equation (2), (5) and (6) gives

Theorem 2. An upper bound for the generation error is given by GE^∗ ≤ ρ(1¯ −x²)

x² (7)

Although there is a possibility of of loosing the bound, it fulfills the same suggestive function for RF as a Vapnik Chervonenkis (VC) type bounds do for other types of classifiers.

It signifies that the strength of each classifier in the forest and the correlation between them in terms of raw margin function are required in the generalization error [38] .

4.4 Bagging

Bagging which can also be referred to as bootstrapping aggregation was first introduced and studied by Leo Breiman [37]. Bagging was the first approach that successfully merged an ensemble of learning algorithms to improve performance over a single such algorithm [41]. With a forest size of N and a training data set m, the bagging would create from the training data setsN a data set with sizem⁰. These data sets are created by random sampling with replacement. Each single tree in the forest would be trained on it’s equivalent sample, then the obtained results are aggregated up. The size of each sample in the bagging is a hyper-parameter of the forest [42]. The probability of a certain object from the training set is absent in a specific bootstrap can be written as

(1− 1

m)^m = 1 e

∼= 0.368 (8)

Thus, the probability of an object occurring in a specific bootstrap sample would be1− 0.368 = 0.632. It is therefore possible (due to the fact that the bootstraps do not depend

(37)

on each other) to evaluate this probability by making use of binomial distribution given by

P(X ≥j) =

N

X

i=j

N j

! 1− 1

e)^j 1 e)^N−j

(9)

Bagging appears to be the most effective if the predictor is imbalanced. When the learning approach is stable, the bagged predictor is balanced, the bagged predictors will not be different from the single predictor and it can in a sense also affect the performance because there is a possibility that the bootstrap sample for each tree contains very few objects from the minority classes while in the extreme case it contains none.

There are various methods to bagging modifications to control imbalanced dataset such as oversampling of the minority classes or under-sampling of the majority classes. Oversam- pling causes an increase in the replication of the objects in the bootstrap samples which in turn lead to an increase in the weight of the object in classification. With oversampling, a more correlated tree is obtained. A similar approach to oversampling is SMOTE [43], it produces synthetic object for the underrepresented classes. Under-sampling of the majority class could lead to bad utilization of the objects in the majority classes. It occurs more when there are more samples in the majority class than there are in the minority class.

However, for our data we shall use a balanced bagging which sample a sufficient number of objects from each class so as to maintain a balance in the bootstrap sampling result.

4.5 Out-of-bag estimates (OOB)

There two major features that characterize RF: out-of-bag error and measure variable importance. OOB is the average of prediction errors of variables in the training set. Each variable is evaluated on trees which did not contain the object in their bootstrap samples [42]. It as been shown that out-of-bag error gives a precise estimate of the generalization error. Wolpert et.al. [44] also showed that the results obtained from out-of-bag error are more reliable than results obtained from k-fold cross validation and it is com- putationally effective as it does not need to restrain the model k times. To assess the predictive accuracy of random forest, out-of-bag error can be calculated as

OOB = 1 N

N

X

k=1

(Y_k−ˆ(Y_k))²

(38)

Yˆ_i represent the mean prediction for kth observation from every tree that has been has been out-of-bag.

To compute the measure of variable importance, we proceed by taking a permutation of a feature and mean of difference in OOB before and after permuting over each tree. If we consider a randomly selected training sample (this sample is also referred to as bootstrap) of sizemdenotedX_m^φ¹ and the observations excluded inX_m^φ¹ that is OOB. To compute the out-of-bag error of the constructed trees fromX_m^φ¹ , for a constantiamidst theqfeatures, the values of the ith variable are permuted randomly over OOB_i to obtain a distributed sample denotedOOB. Then the new OOB of the obtained distributed sample is evaluated˜ and this process is repeated for every bootstrap sampleX_m^φ¹. The importance of theith feature represented as F(Xⁱ) is obtained as the difference between the mean error of distributed and original OOB [45]:

F(Xⁱ) = 1 m

m

X

k=1

(OOB˜ _k−OOB_k) (10)

Thus, if there is an increase in the error generated by the random permutation of theith feature, it signifies that the feature is vital. In general, the higher the value ofF(Xⁱ), the more significant the feature.

4.6 Cross validation

In fitting a classification and regression tree, there is a need to input some variables. Aside the choice of splitting criterion, the amount of pruning is very important. It can be done by either setting a maximum depth of tree (a maximum number of splits before reaching a leaf) or a minimum number of data points below which a node does not get splitted anymore (and thus becomes a leaf). The process of exploring variables that yield better performance is referred to as validation. It involves randomly splitting the dataset into a training and test set (validation set). The model is then fitted using the training set and then the fitted model is used to make prediction for the response of the observations in the test set. It helps in avoiding the stress of having to try different values of our parameters and then test the performance of each on the dataset which will of course eventually result in overfitting on our test set. To prevent this, a frequently used approach is thek-foldcross- validation [42]. This approach divides the training data set into k sections, it proceeds to fit k trees for specific values of our parameter making use of only k −1 section of the training data set. The performance is then measured on the section that was not included

(39)

in the training which was used to grow the tree. The mean of the k-performance values will be found such that we have just one performance value with which we can select the parameter with the best performance value. Finally, it will then be possible to test the performance of the built tree on the test data set.

4.7 Random forest algorithm

It is an extension of bagging classification trees developed by Leo Breiman. Random forest has the additional feature of random variable selection at every node with no pruning or terminating rules. Every tree is grown using a bootstrapped sampleSi from the original learning sampleS[46]. Nonetheless, at every node of the tree,nof theqindependent features are selected at random from which to choose to split. This random selection of variables at every node reduces the correlation among the trees in the forest therefore minimizing the forest error rate. The random subspace selection approach has been shown to have a better performance than only bagging when there are many redundant features adding to discrimination between classes [47], [48], [49]. The Random Forest algorithm as given by [46] is given below: Fory = 1, . . . , Y, samplexobservations with replacement fromS. This is called to as the bootstrap sampleSy. UseSy to build a classification tree:

1. At each nodev, uncorrelated variablespare selected at random.

2. For every selected variables k = 1, ..., m, identify the perfect split s_k from all feasible splits for thekthvariable.

3. Select the perfect split s^∗ out of the k = 1, . . . , m perfect splits s_k on which to split the node t . This j th variable at its identified split point c^∗_s is directing the i = 1, . . . , nvariables with x_ij < c^∗_s to the left descendant node and all variables withx_ij ≥c^∗_sto the right descendant.

4. Repeat steps 1 - 4 for all succeeding nodes to grow a maximal sized tree,T_y.

For the selection of the best split, the Gini ctiterion which has the lowest impurity was used at every node. For every individual trees in the RF, predicted class was obtained for every of the observation. ”The class which has the highest votes amidst theY trees in the forest is the predicted class of an observation” [46].

(40)

5 Results and Discussion

For implementing the random forest algorithm both for classification and regression in Matlab, the data was divided into training and test sets incorporating all the chosen variables as discussed in Chapter 3. Since the major tuning parameters in random for- estalgorithm are the number of trees in the forest and the number of variables, we did a random selection of the variables to be used for the individual splits in everyone of the grown trees and depth of each tree. A sufficient selection of the variables will yield an outstanding improvement in the performance accuracy. The higher the number of trees used, the lower the errors generated. We made the prediction of the regulation price difference for the Sweden data based on the different types of Swedish climate; Spring running from March/April to May, June to August summer, September to October/November fall, November/December to March/February winter.

5.1 Regulation price direction

The forecast for the regulation price direction in each trading hour was made, the out of bag (OOB) estimated error is shown in Figure 15.

0 2 4 6 8 10 12 14 16 18 20

Number of grown trees 0.16

0.18 0.2 0.22 0.24 0.26 0.28

Out-of-bag classification error

Figure 15. Out of bag error obtained for price direction. Showing the fraction of observation belonging to all the trees against the number of grown trees implemented in the random forest algorithm.

(41)

The out-of-bag error (i.e prediction error) was about16.32% as shown in Figure 15 with a performance accuracy of83.05%.

5.2 Regulation price difference

For the regulation price difference prediction was made separately for the years2013− 2017and categorized in accordance to the Swedish climate such that for each year, predictions was made for Spring, Summer, Fall and Winter. In order to obtain an accurate predictive ensemble for the regulation price difference, an estimate of generalized error of boosting was made using10−fold cross-validation and tune the hyper parameters (optimize regression ensemble), then a predictive ensemble was made tuning the decision tree-complexity level in the forest using cross-validation. In order to search for this optimal decision tree-complexity level:

1. A cross-validation of a set of ensembles was made. An exponential increment of the tree complexity level was made for the successive ensembles from decision stump to at most n−1splits, n denoting the sample size and also we vary the learning rate between0.1and1.

2. An estimate of the cross validated MSE (mean-square error) was made for each of the ensembles.

3. A comparison of the cumulative, cross validattion, mean-square error was made by plotting them against the learning rate.

4. The curve with the minimum mean-square error (MSE) was chosen for the predictive ensemble.

(42)

5.2.1 Regulation price difference forecast for 2017

The estimated generalized errors, minimum MSE, number of trees used for the ensemble, learning rate and the maximum number of splits for the four seasons is shown in Table 1

Table 1.Estimates for the four seasons in 2017

Seasons Estimate Error Minimum MSE Trees Maximum split Learning rate

Winter 27.27 18.60 72 128 0.5

Spring 12.64 8.46 22 8 0.1

0 5 10 15 20 25 30

Function evaluations

3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9

Min objective

Min objective vs. Number of function evaluations

Min observed objective Estimated min objective

Figure 16. Ensemble regression with the least estimated cross-validation loss for Winter 2017, with the value of the observed objective function being3.098and the value for estimated objective function was3.198.

(43)

50 100 150 Number of trees 10

20 30 40 50

Cross-validated MSE

MaxNumSplits = 1

50 100 150

Number of trees 10

20 30 40 50

Cross-validated MSE

50 100 150

Number of trees 10

20 30 40 50

Cross-validated MSE

Learning Rate = 0.10 Learning Rate = 0.25 Learning Rate = 0.50 Learning Rate = 1.00 Deep Tree Stump Min. MSE

Figure 17.Predictive ensemble for Winter 2017, ensemble with learning rate 1.0 was used for the prediction.

Nov 2017 Dec 2017 Jan 2017 Feb 2017 Mar 2017

Months -40

-30 -20 -10 0 10 20

Regulation Price

Forecasted Regulation price from Nov - Mar 2017

Original Regulation price diff Predicted Regulation price diff

Figure 18.Forecast made for regulation price difference for Winter 2017.

Forecasting electricity prices in the Swedish regulation market using random forest

FORECASTING ELECTRICITY PRICES IN THE SWEDISH REGULATION MARKET USING RANDOM FOREST

ABSTRACT

PREFACE

CONTENTS

List of Figures

List of Tables

LIST OF ABBREVIATIONS

1 Introduction

1.1 Background of study

1.2 Objective of study

1.3 Literature review

1.4 Structure of the study

2 The Nordic electricity market

2.1 Electricity- a distinct commodity

2.2 Nord Pool

2.3 The Nordic regulation power market

3 Data

3.1 Choice of variables

4 Methodology

4.1 Random Forest

4.2 Convergence of random forest

4.3 Correlation and strength of random forest

4.4 Bagging

4.5 Out-of-bag estimates (OOB)

4.6 Cross validation

4.7 Random forest algorithm

5 Results and Discussion

5.1 Regulation price direction

5.2 Regulation price difference