Fundamentals of Neural Networks: Multi-Layer Perceptrons(MLPs)

VOLUME 4, APRIL 2011

Fundamentals of Neural Networks: Multi-Layer Perceptrons(MLPs)

By : Khalid Isa (PhD Student)

Multiple layer perceptrons (MLPs) represent a generalization of the single-layer perceptron (SLP) as described in previous article. MLPs can form arbitrarily complex decision regions and can separate various input patterns. The capability of MLP stems from the non-linearities used within the nodes. If the nodes were linear elements, then a single-layer network with appropriate weight could be used instead of two- or three-layer perceptrons. Figure 1 shows a typical MLP neural network structure.

The input/output mapping of a network is established according to theweights and the activation functions of their neurons in input, hidden and outputlayers. The number of input neurons corresponds to the number of inputvariables in the neural network, and the number of output neurons is the same asthe number of desired output variables. The number of neurons in the hiddenlayer(s) depends upon the particular NN application. For example, consider thefollowing two-layer feed-forward network with three neurons in the hidden layer and two neurons in the second layer:

apr 1a
Figure 1 Example of MLP

As is shown, the inputs are connected to each neuron in hidden layer via their corresponding weights. A zero weight indicates no connection. For example, if apr 1b it is implied that no connection exists between the second input apr 1c and the third neuron apr 1d. Outputs of last layer are considered as the outputs of the network.The activation function for one neuron could be different from other neurons within a layer, but for structural simplicity, similar neurons are commonly chosen within a layer. The input data sets (or sensory information) are presented to the input layer. This layer is connected to the first hidden layer. If there is more than one hidden layer, the last hidden layer should be connected to the output layer of the network. At the first phase, we will have the following linear relationship for each layer:

apr 1e

where A, is a column vector consisting of m elements, apr 1f is an m x n weight matrix and X is a column input vector of dimension n. For the above example, the linear activity level of the hidden layer (neurons n1 to apr 1d) can be calculated as follows:

apr 1g

The output vector for the hidden layer can be calculated by the following formula:

whereF is a diagonal matrix comprising the non-linear activation functions of the first hidden layer:

References

1. M. Tim Jones, Artificial Intelligence A Systems Approach,Infinity Science Press, 2008.

2. Ham, F. and Kostanic, I., Principles of Neurocomputing for Science and Engineering, McGraw Hill, New York, NY, 2001.

Artificial Neural Networks

VOLUME 4, MARCH 2011

Artificial Neural Networks
By : Zulkifli Zainal Abidin

2. Simulation

All the tests in the simulation are conducted using 16 flies. With respect to the surrounding area, 10 evaluations will be used to determine the direction (smelling process) while 16 to 21 evaluations will be used to determine how far the path will be in that direction (shooting process). In other words, on average, 2 evaluations will be made by each fly in each of the iteration. Both the smelling radius and shooting radius will decay with iterations. Thus, these two parameters are crucial in determining the results. The number of evaluations in each shooting process is also crucial in determining the converging speed.

The tests are made based on several benchmark functions for which their equations are listed in Table 1. De Jong’s function is a maximization problem while the rest of the functions are minimization problems (Fig. 1). Rosenbrook’s function has a long ridge and tends to cause initialization yields point far from the optimums (Fig. 2). It is actually the inverted version of De Jong function but with smaller range. Both functions have 3 peaks and optimum point at (1, 1). Only the best point from the initialization will be chosen for further exploration. Goldstein (Fig. 3) and Martin were chosen as the third and forth functions for testing the algorithm. The test functions and their optima are shown in Table 1 and Table 2.



mac 5a
Fig. 1 De Jong function with a long ridge at the middle and it tends to trap the fly at the beginning iterations.




mac 5b
Fig. 2 Rosenbrook’s Function which is the minimization problem.

mac 5c


Fig. 3 3D surface of Goldstein Problem

Table 1: Boundary and equation for each test case.

Function Name Boundary Equation
De Jong [-2.048, 2.048] mac 5d
Rosenbrook i) [-1.2,1.2]
ii) [-10,10]
mac 5e
Goldstein [-2,2] mac 5f 
Martin [0, 10]
[-10, 5]
 
mac 5g


Table 2: Expected Result of Each Benchmark Functions.

Function Name Expected Location Optimum Fitness
De Jong (1, 1) 3905.93
Rosenbrook (1, 1) 0
Goldstein (0,-1) 3
Martin (5,5) 0
Branin mac 5h


0.3977

Acoustics Capacitive Sensing: The Analogy (Mechanical)

VOLUME 4, MARCH 2011

Acoustics Capacitive Sensing: The Analogy (Mechanical)
By: Mohd Faizal Abd Rahman

In sensing technology, capacitive sensing is one of the techniques applied in detecting some physical variables. The basic structure design of such sensor consists of two parallel plate with a dielectric medium sandwiched between these plates. The top plate is normally referred as membrane or diaphragm which will be acting as an active sensing element. In acoustic engineering, the vibration of the medium particles carrying the signal will cause the membrane to vibrate accordingly, changing the initial gap thickness and thus producing the variation in capacitance according to the well known equation: mac 4a The variation then will be detected by a specific circuit to reflect the measured variable[1]. In order to understand the behaviour of such sensor, researchers are working based on the equivalent model that closely resembles the actual operation. In [1]-[2], the sensor is considered as analogous to spring-mass-damper, while in [3], the sensor is modelled in the form of electrical equivalent circuit.

SPRING-MASS-DAMPER(MSD System)

CMUT can be thought of as having the same working principles as SMD system. So, the mechanical equation for SMD system (a) is a valid representation for CMUT as explained in [1].

mac 4b

where:

m is the mass in kg, k is the spring constant in N/m, and c is the viscous damping coefficient in N-s/m. The time dependent variables, x(t) and u(t) represents the position and displacement component. Analytical representation of this parameter has been discussed in detail in [1]-[2] and are summarised below:

mac 4c

mac 4d and Q is the resonance frequency and the quality factor of the system respectively.

REFERENCES

[1] D Ozewin, D.W Grewe, I.J Oppenheim, S.P Pessiki, Resonant Capacitive MEMS Acoustic Emission Transducers Smart Mater. Struct., 15, 2006, 1863-1871.,

[2] G.G Yaralioglu, A.S Ergun and B.T Khuri-Yakub ,Finite Element Analysis of CMUT,IEEE Trans. on Ultrasonics, Ferroelectric and Frequency Controls, Vol 52,No 12, Dec 2005

[3] G.Caliano et al, PSpice Modeling of CMUT, Ultrasonics Symposium 40 (2002),449-455

Modern Control Systems

VOLUME 4, MARCH 2011

Modern Control Systems

By : Maziyah Mat Noh

Linear Quadratic Regulator (LQR)

LQR is a method in modern control theory that uses state-space approach to analyze such a system. This is the standard optimal control design which produces a stabilizing control law that minimizes a cost function, J that is weighted of sum of squares of the states and input variables. By determines the feedback gain matrix that minimizes J, we can establish the trade-off between the use of control effort, the magnitude, and the speed of response that will guarantee a stable system. Assume that all the states are available for feedback. The cost function is to be minimized is defined as

mac 3a

Where Q is symmetric positive semi-definite state penalty matrix and R is symmetric positive semi-definite control penalty matrix. Choosing Q relatively large than those of R, then deviations of x from zero will be penalized heavily relative to deviations of u from zero. On the other hand, if R is relatively large than those of Q, then control effort will be more costly and the state will not converge to zero as quickly as we wish. The tracking performance of the LQR applied to the system under investigation by setting the value of vector K and N which determines the feedback control law and for elimination of steady state error capability respectively. The LQR structure is shown in figure 1.

mac 3b 
Figure 1: LQR structure

Consider a linear time-invariant of an aircraft in (1)[3].

mac 3c

Where mac 3d = angle of attack, q = pitch rate, jan 1d = pitch angle (output), and mac 3e = elevator deflection angle (input). From figure 1, K = control gain matrix, u = -Kx(t) = input, Nbar is a scaling factor. Using MATLAB, we obtain the open loop poles lies at 0, 0.3695 + 0.8860i, and -0.3695 - 0.8860i. The state penalty matrix Q is

set to be mac 3f and control penalty R is set to be 0.5. Using these values, we obtain K = [-0.65 208.53 10] and Nbar= 10. Figure 2 shows the open loop and closed loop responses in response to input of mac 3e = 0.2rad.

mac 3g

Figure 2: Response of the system Observer-based controller

 

References

[1] Ashish Tewari, Modern Control Design with MATLAB and SIMULINK, John Wiley & Sons Ltd, 2002

[2] Herbert Werner. Control Systems 2 lecture notes. Hamburg University of Technology (TUHH), Germany. 2008

[3] Control Tutorial for Matlab. http://www.engin.umich.edu/group/ctm/examples/pitch/SSpitch.html . Accessed Jan 2011.

Linear and Switching Power supply fundamentals (Part 6)

VOLUME 4, MARCH 2011

Linear and Switching Power supply fundamentals (Part 6)

By: Alireza Nazem

Protection Circuits Built Into IC Linear Regulators

Linear IC regulators contain built-in protection circuits, which make them virtually immune to damage from either excessive load current or high operating temperature. The two protection circuits found in nearly all linear IC regulators are:

a) Thermal Shutdown
b) Current Limiting

CHAIN OF COMMAND

The thermal shutdown, current limiter, and voltage error amplifier make up three distinct and separate control loops that have a definite hierarchy (pecking order) which allows one to "override" the other. The order of command of the loops is:

1) Thermal Limit (IC is regulating junction temperature/power dissipation)
2) Current Limit (IC is regulating load current)
3) Voltage Control (IC is regulating output voltage)

This hierarchy means that a linear regulator will normally try to operate in "constant voltage" mode, where the voltage error amplifier is regulating the output voltage to a fixed value. However, this assumes that both the load current and junction temperature are below their limit threshold values. If the load current increases to the limiting value, the current limiting circuitry will take control and force the load current to the set limiting value (overriding the voltage error amplifier). The voltage error amplifier can resume control only if the load current is reduced sufficiently to cause the current limiting circuits to release control.

A rise in dying temperature approaching the limit threshold (about 160°C for the non-military range) will cause the thermal shutdown and cut drive to the power transistor, thereby reducing the load current and internal power dissipation. Note that the thermal limiter can override both the current limit circuits and the voltage error amplifier. It is important to understand that a regulator holds its output voltage fixed only when it is in constant voltage mode. In current limiting, the output voltage will be reduced as required to hold the load current at the set limiting value.

In thermal limiting, the output voltage drops and the load current can be reduced to any value (including zero). No performance characteristic specifications apply when a part is operating in thermal shutdown mode.