SCOPE - LTE network performance in latency-sensitive control applications

1. INTRODUCTION

1.4. SCOPE

For the experimental part of the research another latency-sensitive application had to be selected due to the reasons mentioned in previous section. A position control application was selected due to the availability of a test setup at LUT, to the fact that it is a latency-sensitive application, and, finally, to the similarity with microgrid protection application in terms of latency requirements.

Another scope limitation for this research is that no custom-built devices are used. All the hardware involved in the experimental part is off-the-shelf industrial automation equipment without any modifications. Same limitation is present for the software part – no custom-written protocols or drivers were used to carry out experiments.

Last limitation to be mentioned is that 4G LTE network was used as a “black box”. In other words, the research team had no access to enB settings, and no specific wireless network configuration has been performed.

18 1.5. STRUCTURE

In section 2, an applicable IEC standard is described, general quality metrics of a network are presented and quantitative requirements to microgrid protection application communication network are summarized.

In section 3, detailed descriptions of experimental setups and mathematical models used in this research are given. Also, experiment data and simulation results are presented.

In section 4, all the results of the research are pointed out, answering goals and challenges mentioned above.

In section 5, all the results of the research are pointed out, replying to questions posed in goals and challenges sections. Also, further research directions and possibilities are highlighted.

In section 6, goals and results of this research are summarized.

2. APPLICATION REQUIREMENTS TO COMMUNICATION NETWORK

As it was mentioned above, 4G LTE network has to meet certain requirements in order to be used as a communication medium for microgrid protection and control systems. At the moment there is no standard developed specifically for the systems mentioned above.

Nevertheless, it is suggested by numerous sources to use IEC 61850 family of standards as a source for the requirements mentioned. IEC 61850 standards family was originally designed for substation automation systems, but its development continued further when a necessity appeared to support microgrid automation and to integrate microgrids into the utility grid structure.

IEC 61850 includes numerous standards regulating various aspects of substation automation systems. Standards that are most important from the point of view of this research are presented in the Table 2 below.

For instance, authors of [22] are using IEC 61850-7-420, a standard describing the presentation of microgrid elements as logical nodes, to model a centralized microgrid protection system. In another article, it is stated that IEC 61850-5 can be applied directly to microgrid control applications and that it simplifies the interplay between protection and control systems of a microgrid, and allows easier integration of a microgrid into the utility grid [23]. One more article briefly describes how IEC 61850 standards family got adjusted to be used for microgrids. It is pointed out that it happened mainly to unify technologies used and avoid vendor-specific technologies making microgrid automation systems unnecessary complicated [24].

The topic of IEC 61850-compatible data acquisition seems to be popular nowadays as well.

For instance, in a recent master’s thesis the topic of sending Generic Object-Oriented Substation Event (GOOSE) and Sampled Measured Values (SMV) messages – IEC 61850-defined protocols – over Wide Area Network (WAN) based on Technical Report (TR) IEC 61850-90-5 is discussed in detail, as was already mentioned above [15]. Similar possibilities, particularly sending GOOSE messages over WAN, are said to be investigated in [25] as well. The research, where smart meters data acquisition by IEC 61850 MMS protocol over LTE is considered, was also mentioned above [16].

Thus, it is assumed to be possible to use the abovemetioned standards family as a source of requirements for microgrid protection and control systems, namely for communication networks of such systems.

Next step would be to determine the quality metrics that characterize the performance of a communication channel. In [15] and [13] it is proposed to use the following parameters:

- Network Throughput,

- Latency or travel time or delay,

- Reliability or Packet Delivery Ratio (PDR) or Packet Loss Ratio (PLR), - Availability,

- Packet Delivery Variation (PDV) (or jitter).

Table 2. IEC 61850 standards mentioned in this thesis.

IEC 61850-5:2013 Communication requirements for functions and device models IEC 61850-7-420:2009 Basic communication structure - Distributed energy resources

logical nodes

IEC 61850-7-2:2010 Basic communication structure - Abstract communication service interface (ACSI)

IEC TR 61850-90-5:2012

Use of IEC 61850 to transmit synchrophasor information according to IEEE C37.118

PDV is sometimes also referred to as jitter, but the latter can cause confusion because it has different meaning in different scientific areas. A brief description of all the parameters is presented below.

Network throughput represents the number of successfully delivered packages per second in a certain network, and can be estimated using the equation [26]:

𝑇ℎ𝑟𝑜𝑢𝑔ℎ𝑝𝑢𝑡 =𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑢𝑐𝑐𝑒𝑠𝑠𝑓𝑢𝑙𝑙𝑦 𝑑𝑒𝑙𝑖𝑣𝑒𝑟𝑒𝑑 𝑏𝑖𝑡𝑠

𝑡𝑖𝑚𝑒 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙 . (1)

Since the amount of transmitted information is measured in bits, measurement unit for network throughput is bits per second (bps).

In the scope of this research network throughput was not considered due to the fact that latency is a higher priority characteristic in terms of enabling 4G LTE network to be used as a communication medium for microgrid protection and control. If the aforementioned

network is not able to fulfill latency requirements, there will probably be no point to investigate it in terms of network load.

Latency, or transfer time, is defined as a time required for a packet to be transferred from its source to its destination. Either One-way Trip Time (OTT) or Round-Trip Time (RTT) can be measured to estimate network latency. Measurement unit for this quality metric is usually milliseconds. The definition of OTT and RTT latencies in terms of equations is presented as:

𝐿𝑎𝑡𝑒𝑛𝑐𝑦_𝑂𝑇𝑇 = 𝑡_𝑟𝑥2− 𝑡_𝑡𝑥1, (2) 𝐿𝑎𝑡𝑒𝑛𝑐𝑦_𝑅𝑇𝑇 = (𝑡_𝑟𝑥2− 𝑡_𝑡𝑥1) + (𝑡_𝑟𝑥1− 𝑡_𝑡𝑥2), (3) where 𝑡_𝑡𝑥1 is the time when packet was sent by device 1 and 𝑡_𝑟𝑥2 – when it was received by device 2. For RTT equation 𝑡_𝑡𝑥2 and 𝑡_𝑟𝑥1 stand for the time when the same packet was sent back by device 2 and received by device 1, respectively.

Latency is the main quality metric in the scope of this research, because it defines whether the 4G LTE network in question is suitable for microgrid protection and control application. More information on the latency measurement process during experiment stage of this research is given in section 3.3.

PDV is closely connected with latency, and is according to [27], “the difference in end-to-end one-way delay between selected packets in a flow with any lost packets being

ignored.” In other words, PDV shows how much does latency value change during a certain time period. For instance, if latency of a connection is constant, its PDV equals zero. It is worth noting here, that, according to [28] and [29], for a latency-sensitive application the effect of variable latency is more significant than that of a constant one.

Therefore, this quality metric should be taken into account as well.

Reliability of a network is the ratio of successfully received packet quantity to the total number of packets sent over a certain period of time, as shown in the equation [30]:

𝑅𝑒𝑙𝑖𝑎𝑏𝑖𝑙𝑖𝑡𝑦 = 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑢𝑐𝑐𝑒𝑠𝑠𝑓𝑢𝑙𝑙𝑦 𝑑𝑒𝑙𝑖𝑣𝑒𝑟𝑒𝑑 𝑝𝑎𝑐𝑘𝑒𝑡𝑠

𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑝𝑎𝑐𝑘𝑒𝑡𝑠 𝑠𝑒𝑛𝑡 𝑑𝑢𝑟𝑖𝑛𝑔 𝑠𝑎𝑚𝑒 𝑝𝑒𝑟𝑖𝑜𝑑 𝑜𝑓 𝑡𝑖𝑚𝑒. (4) Therefore, reliability metric represents how many packets are lost in the network. Packet loss may occur due to a number of reasons, for instance, network congestion and network unavailability. Reliability is crucial for latency-sensitive applications because lost packages mean lost information and consequently possible application failures.

Since the project aims at deploying 4G LTE networks in rural areas, the question of network coverage is very important, and reliability metric can represent the connection between wireless network signal strength and its PDR, potentially defining an area where it is safe to install microgrid protection equipment.

Finally, availability can be defined as a probability that at a certain point in time 4G LTE Network will be operational [31]. Although it is an important network quality metric and should be taken into consideration in the case of using 4G LTE network for microgrid protection and control, it is out of the scope of this research. The reason for that is this thesis aims to investigate the latency constraint of a 4G LTE network, and it is assumed that all devices used in the research and test processes comply with availability requirements. Another reason is that availability in scope of the Fusion Grid project depends also on the rural area where the network is to be deployed, power producing and storing devices, weather and human factors, which makes its estimation rather complicated.

To sum up, only latency, PDV and reliability are considered in this thesis as quality metrics that characterize the performance of a communication channel.

Next, as it was mentioned above, the requirements to a network used for communication in microgrid protection and control applications are listed in [32]. Firstly, a definition of latency is given as OTT from sending to receiving device plus transfer processing time at both of them, illustration presented in Figure 3.

Figure 3. IEC 61850-5 transfer time definition.

Then, six message types are defined, as well as three performance classes that depend on the function of particular device with which communication is happening. Next, for every combination of message type and performance class, a maximum allowed transfer time is defined in milliseconds, as can be seen in the Table 3 below.

Table 3. IEC 61850-5 communication network transfer time requirements.

Transfer time, ms Performance

class 1

Performance class 2

Performance class 3 Type 1 –

fast messages

1A - Trip < 10 < 3 < 3 1B - Others < 100 < 20 < 20 Type 2 – medium speed messages < 100 < 100 < 100 Type 3 – low speed messages < 500 < 500 < 500 Type 4 – raw speed messages < 10 < 3 < 3 Type 5 – File transfer functions >= 1000 >= 1000 >= 1000 Type 6 – Command messages and file

transfer with access control

- -- -

Therefore, 4G LTE test network quality metrics have to be measured and compared to the aforementioned requirements. There were no clear requirements found for PDV and reliability values in [32], which makes latency the most important quality metric in the scope of this research.

3. COMMUNICATION NETWORK PERFORMANCE

Best option for latency estimation of a 4G LTE network would have been a test microgrid setup with protection devices. However, such an installation was not available at LUT during this thesis work, so other options had to be found. One opportunity could have been to perform a series of latency and PDV tests between two devices connected to the network in question. This option is simple to carry out, but results may be not accurate enough because those depend on many device-related factors, for example, on the OS and protocol used. In addition, the above mentioned option would not allow to clearly demonstrate how 4G LTE network latency affects the performance of latency sensitive applications. Finally, it was decided to utilize another latency-sensitive application, with latency requirements similar to those of microgrid and with a test setup available at LUT (see section 3.3).

3.1. TEST LTE NETWORK DESCRIPTION

Test LTE network is promoted as a wireless solution for communication in industrial and digital automation applications. The package includes one or multiple enBs, and an industrial PC to allow local data processing functionality (without core network participation) and cloud connectivity via Internet. Also, devices that allow User Equipment (UE) to connect to test LTE network are needed. Finally, a frequency spectrum band has to be licensed in order to be used by test LTE network system.

A test private LTE network was installed at LUT to be used for research and testing. It includes two enBs – one installed outdoors and another one - indoors. Other positions included are industrial PC and eight LTE modems to provide UE the ability to connect to the network. Frequency band was acquired from a licensing organisation, and application-specific SIM cards were issued by mobile service provider. LTE network is private, so only users with the SIM cards mentioned above are allowed to be registered and connected.

In terms of network adresses, test LTE network looks like a typical subnetwork as can be seen in Figure 4.

Figure 4. Test 4G LTE enB network IP structure from UE point of view.

Test LTE network structure and functionality is similar to the one that is involved in the Fusion Grid project, and will possibly be used in microgrid protection application.

3.2. TEST CASE 1 – LUT GREEN CAMPUS (GC) NETWORK

Main purpose of this test case was to help the research team to become familiar with using test 4G LTE network, with setting up LTE routers and with establishing VPN client-server connections.

LUT GC in the context of this thesis is a number of renewable energy plants installed at the university site [33]. Energy produced by these installations is used to partly cover the energy consumption of the university. Energy production data for every plant is collected every second and stored on a database server for further use and analysis.

There are five renewable energy sources in GC network: one wind turbine and four solar PV plants.Wind turbine nominal power is 20 kW at 11 m/s wind speed.

Solar plants have different rated output power. Solar plant on Carport has the highest output of 108 kW. Roof plant follows with 90.5 kW in total, including 51.5 kW provided by flat roof setup and 39.0 kW – by wall-mounted setup. Solar tracker accounts for 5 kW peak power, and manual tracker has only one 250W solar panel installed.

All data acquisition for renewable power plants is done by Siemens PAC3200 energy meters. Modbus Transmission Control Protocol (TCP) protocol is used for data acquisition [34]. Database server generates queries for every PAC3200 device every second, and stores received data.

Initial structure of the GC network is presented in Figure 5. As it can be seen, there is a total of seven PAC3200 devices dealing with energy production measurement. Wind turbine and solar tracker energy meters are installed inside LUT energy laboratory, other devices are located outdoors in electrical enclosures. All PAC3200 devices use a wired connection to exchange data with database server.

It is important to mention that in order to be able to get data from a PAC3200, local IP address of the latter must be visible to the database server. At the same time, server local IP address must also be reachable for every power analyzer. Therefore, a set of routing rules was added to every router or switch in between PAC3200 devices and the server to allow visibilities mentioned above.

The idea of this test case was to replace the wired network connection that all energy meters had initially with a wireless connection via test 4G LTE network.

Figure 5. Initial GC network structure.

In order to do that, every energy meter installation site was equipped with a LTE router.

Then, every PAC3200 device was connected to the router with a cable and router itself was connected wirelessly (green lines in Figure 6 below) to the enB. Due to the circumstance that all the devices still had to be in the same subnet, and that network security had to be maintained after the transition to wireless medium, it was decided by LUT IT department to use a VPN tunnel to protect the wireless part of the connection.

There are numerous software solutions that allow setting up VPNs. In this case, OpenVPN was used due to the reasons that it is freeware, it is available for many operating systems, it

has high configuration flexibility and finally that IT department is already familiar with it [35].

Thus, a VPN tunnel was set between LTE routers and a virtual VPN server, the latter having a wired connection to the database server. Routing rules were added to LTE routers, and OpenVPN server, and existing ones were changed at the database server.

Figure 6. Resulting GC network structure.

As a result, three Digi WR31 routers were installed on sites 1,2 and 4 in Figure 6 [36].

However, this LTE router model is not suitable for site 3 due to the fact that it has only two Ethernet ports, while three PAC3200 devices should be connected on site 3. Thus, another router model with three Ethernet ports is planned to be installed on site 3 - Telewell SF301-G [37]. At the moment, it is investigated why this router model does not expose its local subnet addresses to OpenVPN server. As soon as this problem is solved, this router will be installed on site 3 to move remaining PAC3200 devices under test LTE network.

Table 4. Test case 1 equipment list.

LTE router 1, 2, 4 DIGI WR31

LTE router 3 Telewell SF301-G

Power analyzer Siemens PAC3200

Indoor enB - -

Outdoor enB - -

Unfortunately, there was no specific research done to compare reliability and availability of the network before and after LUT GC network partial transition to wireless medium.

Nonetheless, an availability data request was sent to private LTE network provider. As a result, availability measurements for both enodeBs were provided from 29^th July to 27^th August, and from 17^th to 24^th September 2018. Measurements were taken every 15 minutes, and belong to 0..100 range, where 0 refers to “not available during last 15 minutes”, 50 – “available for 7.5 out of last 15 minutes” and 100 – available throughout last 15 minutes. Hence, the number refers to time percentage when enodeB was available.

Data analysis has shown that both enodeBs were available 100% of the time period in August. During one week in September indoor enodeB was not in operating state from 20:45 18^th to 03:25 19^th , reducing the availability percentage to 96.1%. However, during this time period no experiments were carried out, so only data acquisition routines from PAC3200 devices were interrupted.

Therefore, based on the information presented above, and also on the feedback of researchers working with GC database, private LTE network is assumed to be suitable for data acquisition purposes in GC structure. At the same time, test LTE network can be used in other university projects, especially those that involve remote control of mobile setups, for instance, in mobile robot applications.

3.3. TEST CASE 2 – THREE-AXIS MANIPULATOR POSITION CONTROL 3.3.1. TEST SETUP GENERAL DESCRIPTION

As it was mentioned in section 1.4, a position control application was selected as a test setup to determine if private LTE network is suitable to be used as a communication channel in a microgrid protection application. Utilizing a position control loop as a test setup allows to measure the latency introduced by test LTE network, and to investigate the relation between latency magnitude and a number of LTE network parameters. In addition, microgrid protection and position control applications both have strict requirements to communication channel latency, so the positioning quality can be considered as an indirect estimate of microgrid protection execution quality.

Hence, in the second test case, a manipulator setup is used, as shown in Figure 7. It is also referred to as “cartesian pick-and-place robot” in [38]. The manipulator is able to move in

three dimensions, as well as to pick up and drop objects using its grip mechanism.

Functional diagram of this setup is presented on Figure 8, and equipment pieces involved in tests are listed in Table 5.

Figure 7. Three-axis manipulator used in test case 2 [39].

Operation principle is similar for all manipulator axis. A Permanent Magnet Synchronous Machine (PMSM) is connected to a tooth-belt drive, the latter converting rotary movement of motor shaft into belt linear movement. It should be mentioned that X axis movement is executed by two tooth-belt drives X1 and X2 similar to those described above and moving

In document LTE network performance in latency-sensitive control applications (sivua 17-0)