# Fundamentals of Neural Networks: Multi-Layer Perceptrons(MLPs)

VOLUME 4, APRIL 2011

Fundamentals of Neural Networks: Multi-Layer Perceptrons(MLPs)

By : Khalid Isa (PhD Student)

Multiple layer perceptrons (MLPs) represent a generalization of the single-layer perceptron (SLP) as described in previous article. MLPs can form arbitrarily complex decision regions and can separate various input patterns. The capability of MLP stems from the non-linearities used within the nodes. If the nodes were linear elements, then a single-layer network with appropriate weight could be used instead of two- or three-layer perceptrons. Figure 1 shows a typical MLP neural network structure.

The input/output mapping of a network is established according to theweights and the activation functions of their neurons in input, hidden and outputlayers. The number of input neurons corresponds to the number of inputvariables in the neural network, and the number of output neurons is the same asthe number of desired output variables. The number of neurons in the hiddenlayer(s) depends upon the particular NN application. For example, consider thefollowing two-layer feed-forward network with three neurons in the hidden layer and two neurons in the second layer:

Figure 1 Example of MLP

As is shown, the inputs are connected to each neuron in hidden layer via their corresponding weights. A zero weight indicates no connection. For example, if it is implied that no connection exists between the second input and the third neuron . Outputs of last layer are considered as the outputs of the network.The activation function for one neuron could be different from other neurons within a layer, but for structural simplicity, similar neurons are commonly chosen within a layer. The input data sets (or sensory information) are presented to the input layer. This layer is connected to the first hidden layer. If there is more than one hidden layer, the last hidden layer should be connected to the output layer of the network. At the first phase, we will have the following linear relationship for each layer:

where A, is a column vector consisting of m elements, is an m x n weight matrix and X is a column input vector of dimension n. For the above example, the linear activity level of the hidden layer (neurons n1 to ) can be calculated as follows:

The output vector for the hidden layer can be calculated by the following formula:

whereF is a diagonal matrix comprising the non-linear activation functions of the first hidden layer:

References

1. M. Tim Jones, Artificial Intelligence A Systems Approach,Infinity Science Press, 2008.

2. Ham, F. and Kostanic, I., Principles of Neurocomputing for Science and Engineering, McGraw Hill, New York, NY, 2001.

# Artificial Neural Networks

VOLUME 4, MARCH 2011

Artificial Neural Networks
By : Zulkifli Zainal Abidin

2. Simulation

All the tests in the simulation are conducted using 16 flies. With respect to the surrounding area, 10 evaluations will be used to determine the direction (smelling process) while 16 to 21 evaluations will be used to determine how far the path will be in that direction (shooting process). In other words, on average, 2 evaluations will be made by each fly in each of the iteration. Both the smelling radius and shooting radius will decay with iterations. Thus, these two parameters are crucial in determining the results. The number of evaluations in each shooting process is also crucial in determining the converging speed.

The tests are made based on several benchmark functions for which their equations are listed in Table 1. De Jong’s function is a maximization problem while the rest of the functions are minimization problems (Fig. 1). Rosenbrook’s function has a long ridge and tends to cause initialization yields point far from the optimums (Fig. 2). It is actually the inverted version of De Jong function but with smaller range. Both functions have 3 peaks and optimum point at (1, 1). Only the best point from the initialization will be chosen for further exploration. Goldstein (Fig. 3) and Martin were chosen as the third and forth functions for testing the algorithm. The test functions and their optima are shown in Table 1 and Table 2.

Fig. 1 De Jong function with a long ridge at the middle and it tends to trap the fly at the beginning iterations.

Fig. 2 Rosenbrook’s Function which is the minimization problem.

Fig. 3 3D surface of Goldstein Problem

## Table 1: Boundary and equation for each test case.

 Function Name Boundary Equation De Jong [-2.048, 2.048] Rosenbrook i) [-1.2,1.2] ii) [-10,10] Goldstein [-2,2] Martin [0, 10] [-10, 5]

## Table 2: Expected Result of Each Benchmark Functions.

 Function Name Expected Location Optimum Fitness De Jong (1, 1) 3905.93 Rosenbrook (1, 1) 0 Goldstein (0,-1) 3 Martin (5,5) 0 Branin 0.3977

# Acoustics Capacitive Sensing: The Analogy (Mechanical)

VOLUME 4, MARCH 2011

Acoustics Capacitive Sensing: The Analogy (Mechanical)
By: Mohd Faizal Abd Rahman

In sensing technology, capacitive sensing is one of the techniques applied in detecting some physical variables. The basic structure design of such sensor consists of two parallel plate with a dielectric medium sandwiched between these plates. The top plate is normally referred as membrane or diaphragm which will be acting as an active sensing element. In acoustic engineering, the vibration of the medium particles carrying the signal will cause the membrane to vibrate accordingly, changing the initial gap thickness and thus producing the variation in capacitance according to the well known equation: The variation then will be detected by a specific circuit to reflect the measured variable[1]. In order to understand the behaviour of such sensor, researchers are working based on the equivalent model that closely resembles the actual operation. In [1]-[2], the sensor is considered as analogous to spring-mass-damper, while in [3], the sensor is modelled in the form of electrical equivalent circuit.

SPRING-MASS-DAMPER(MSD System)

CMUT can be thought of as having the same working principles as SMD system. So, the mechanical equation for SMD system (a) is a valid representation for CMUT as explained in [1].

where:

m is the mass in kg, k is the spring constant in N/m, and c is the viscous damping coefficient in N-s/m. The time dependent variables, x(t) and u(t) represents the position and displacement component. Analytical representation of this parameter has been discussed in detail in [1]-[2] and are summarised below:

and Q is the resonance frequency and the quality factor of the system respectively.

REFERENCES

[1] D Ozewin, D.W Grewe, I.J Oppenheim, S.P Pessiki, Resonant Capacitive MEMS Acoustic Emission Transducers Smart Mater. Struct., 15, 2006, 1863-1871.,

[2] G.G Yaralioglu, A.S Ergun and B.T Khuri-Yakub ,Finite Element Analysis of CMUT,IEEE Trans. on Ultrasonics, Ferroelectric and Frequency Controls, Vol 52,No 12, Dec 2005

[3] G.Caliano et al, PSpice Modeling of CMUT, Ultrasonics Symposium 40 (2002),449-455

# Modern Control Systems

VOLUME 4, MARCH 2011

Modern Control Systems

By : Maziyah Mat Noh

Linear Quadratic Regulator (LQR)

LQR is a method in modern control theory that uses state-space approach to analyze such a system. This is the standard optimal control design which produces a stabilizing control law that minimizes a cost function, J that is weighted of sum of squares of the states and input variables. By determines the feedback gain matrix that minimizes J, we can establish the trade-off between the use of control effort, the magnitude, and the speed of response that will guarantee a stable system. Assume that all the states are available for feedback. The cost function is to be minimized is defined as

Where Q is symmetric positive semi-definite state penalty matrix and R is symmetric positive semi-definite control penalty matrix. Choosing Q relatively large than those of R, then deviations of x from zero will be penalized heavily relative to deviations of u from zero. On the other hand, if R is relatively large than those of Q, then control effort will be more costly and the state will not converge to zero as quickly as we wish. The tracking performance of the LQR applied to the system under investigation by setting the value of vector K and N which determines the feedback control law and for elimination of steady state error capability respectively. The LQR structure is shown in figure 1.

Figure 1: LQR structure

Consider a linear time-invariant of an aircraft in (1)[3].

Where = angle of attack, q = pitch rate, = pitch angle (output), and = elevator deflection angle (input). From figure 1, K = control gain matrix, u = -Kx(t) = input, Nbar is a scaling factor. Using MATLAB, we obtain the open loop poles lies at 0, 0.3695 + 0.8860i, and -0.3695 - 0.8860i. The state penalty matrix Q is

set to be and control penalty R is set to be 0.5. Using these values, we obtain K = [-0.65 208.53 10] and Nbar= 10. Figure 2 shows the open loop and closed loop responses in response to input of = 0.2rad.

Figure 2: Response of the system Observer-based controller

References

[1] Ashish Tewari, Modern Control Design with MATLAB and SIMULINK, John Wiley & Sons Ltd, 2002

[2] Herbert Werner. Control Systems 2 lecture notes. Hamburg University of Technology (TUHH), Germany. 2008

[3] Control Tutorial for Matlab. http://www.engin.umich.edu/group/ctm/examples/pitch/SSpitch.html . Accessed Jan 2011.

# Linear and Switching Power supply fundamentals (Part 6)

VOLUME 4, MARCH 2011

Linear and Switching Power supply fundamentals (Part 6)

By: Alireza Nazem

Protection Circuits Built Into IC Linear Regulators

Linear IC regulators contain built-in protection circuits, which make them virtually immune to damage from either excessive load current or high operating temperature. The two protection circuits found in nearly all linear IC regulators are:

a) Thermal Shutdown
b) Current Limiting

CHAIN OF COMMAND

The thermal shutdown, current limiter, and voltage error amplifier make up three distinct and separate control loops that have a definite hierarchy (pecking order) which allows one to "override" the other. The order of command of the loops is:

1) Thermal Limit (IC is regulating junction temperature/power dissipation)
2) Current Limit (IC is regulating load current)
3) Voltage Control (IC is regulating output voltage)

This hierarchy means that a linear regulator will normally try to operate in "constant voltage" mode, where the voltage error amplifier is regulating the output voltage to a fixed value. However, this assumes that both the load current and junction temperature are below their limit threshold values. If the load current increases to the limiting value, the current limiting circuitry will take control and force the load current to the set limiting value (overriding the voltage error amplifier). The voltage error amplifier can resume control only if the load current is reduced sufficiently to cause the current limiting circuits to release control.

A rise in dying temperature approaching the limit threshold (about 160°C for the non-military range) will cause the thermal shutdown and cut drive to the power transistor, thereby reducing the load current and internal power dissipation. Note that the thermal limiter can override both the current limit circuits and the voltage error amplifier. It is important to understand that a regulator holds its output voltage fixed only when it is in constant voltage mode. In current limiting, the output voltage will be reduced as required to hold the load current at the set limiting value.

In thermal limiting, the output voltage drops and the load current can be reduced to any value (including zero). No performance characteristic specifications apply when a part is operating in thermal shutdown mode.