Basic Principles

Neural networks (NN), also known as artificial neural networks (ANN), are machine learning algorithms that are widely used to process different kinds of data. These models have been used “from pattern recognition to optimization and scheduling” (Maren et al., 2014, pp.1). Neural networks are models that are able to analyze linear and nonlin-ear datasets. These models also have the ability to lnonlin-earn, adapt and generalize the pro-cessed data. As with other statistical models, also neural networks require appropriate and sufficient data in order to receive accurate results. (Yu et al., 2007, pp. 27). The aim of NN models is to automate the information analysis and, with complex algorithms, make these models more efficient and capable of detecting changes.

Neural networks are inspired by the structure of the human brain. The basic idea is that network consists of layers that consist of computational units called neurons. These neu-rons are individual processing units that are able to receive information and then trans-mit this information to other neurons. Therefore, neurons are always interconnected and they pass signals through the network. More precisely, an artificial neuron consists of the input values, weights, sum function, activation function, and output value. Figure 1 presents the most basic structure for a network.

Figure 1. Neural Network (Haykin, 2009, pp. 10-11)

The figure shows that the network receives one or more input values (𝑥₁, 𝑥₂… 𝑥_𝑛) which it then processes with a set of coefficients (𝑤₁, 𝑤₂… 𝑤_𝑛 ). These coefficients are also called synaptic weights and each weight either amplifies or weakens the corresponding input. The network calculates a sum of all the received inputs and then multiplies each input with the corresponding coefficients (∑). Threshold value (𝑏) determines the appropriate value that the summation must exceed for the neuron to activate. 𝑉 describes the activation potential and therefore it describes the difference between the sum function and the threshold. The neuron is activated when 𝑉 > 0. After the activation potential is passed, the input is transmitted to the activation function (𝜑(∙

)) which is designed to keep the values inside certain limit values. The output value (𝑦) is the final value corresponding to the received inputs and their weights. (Haykin 2009, pp. 10-11).

The architecture of a neural network determines the structure, the number of neurons, the number of different layers as well as the direction of the signals that pass through the model. Figure 2 presents the general architecture of a neural network.

Figure 2. Basic architecture of neural network (Yu et al., 2007, pp. 27)

The basic architecture consists of an input layer, one or several hidden layers, and an output layer. These layers consist of neurons which are presented as nodes in the figure.

All of these nodes interact with each other and therefore this connection is illustrated in the figure by arrows between the nodes. The function of the input layer is to receive data and signals outside the network. Often input layer also scales and normalizes the received data. The hidden layers, on the other hand, are responsible for the data analy-sis. The number of hidden layers varies depending on the complexity of the problem and the amount and quality of the data which is being processed. Finally, the function of the output layer is to produce and present the final outputs of the neural network. (Yu et al., 2007, pp. 27).

However, a neural network is not automatically an accurate model which knows how to process input data to a final output data. Neural networks gain the ability to provide accurate output values from a dataset with a training process. In this training process, the model is trained with examples that consist of the input value and a corresponding output value as well as the synaptic weights. With the synaptic weights, the aim is to minimize the difference between the target output value and the actual output value.

The learning process continues until the model is able to identify patterns and therefore is able to provide estimate values for even input signals whose outputs are still un-known. Therefore, networks are able to learn from example data and then apply this knowledge to similar cases.

This learning ability is one of the main reasons why neural networks have been used to solve complex data problems. However, there are also several other benefits in neural networks. According to Maren et al. (2014) and Haykin (2009) the main advantages of neural networks are the following properties:

1. Input-output mapping 2. Adaptivity

3. Nonlinearity 4. Fault Tolerance

Input-output mapping refers to the ability to learn from examples. Then from these ex-amples, the model is able to detect patterns that can be utilized when processing new input data. Nonetheless, networks are also adaptable so in case the environment changes the model is able to readjust. This adaptivity is one of the biggest benefits of neural networks. More particularly, adaptivity describes how the synaptic weights are able to change according to the environment with simple retraining. (Haykin, 2009, pp.

2-3; Maren et al., 2014, pp. 7-8).

The reason why neural networks have been especially utilized in the research of foreign exchange rates is due to their ability to handle complex and diversified data. One of the main properties of neural networks is flexibility as the model can process both linear or nonlinear data. Lastly, a neural network is fault tolerance which means that the perfor-mance of the model does not depend on individual information. In other words, the model may notice that some information is invalid or missing and despite this produce a valid output. (Haykin, 2009, pp. 3-4; Maren et al., 2014, pp. 7-8).

Given the above benefits, it is no wonder that neural networks are being used to process complex data and problems. The demand for time-effective machine learning methods is increasing and while it is possible to utilize these models in a more efficient and ver-satile way, also problems related to these models are being more and more detected. A

common issue related to neural networks is the format of the data that the model re-quires. NNs require numerical data and in some cases, it might be extremely difficult to translate information to numerical values. (Mijwel, 2018).

Additionally, common disadvantages of neural networks are for instance the disability to explain the behavior of the model as well as the determination of an accurate network structure. The first is probably the most important since when it comes to neural net-works the model does not provide information on how and why it has received certain output values. Therefore, in some cases, there might be uncertainty related to values that the network provides. However, the more the model is trained, the better results should be provided. The latter problem is related to this training process since there is no certain way to construct a neural network. Thus, only with trial and error it is possible to learn what kind of structure suits certain kinds of datasets. (Mijwel, 2018).

To conclude, there are many advantages with neural networks which make them a com-petitive option for different kinds of studies. There are naturally some challenges as well and therefore neural networks should be utilized in studies that are able to exploit its positive features and, on the other hand, control its challenges. These features that have so far been presented are general features related to NN. Thus, it is crucial to understand that there are different types of neural networks which have even more advantages and disadvantages. Hence, the next section of this paper will go through the most common types of neural networks.