Development of Fuzzy Neural Networks: Current Framework and Trends

Fuzzy systems have been demonstrated their ability to solve different kinds of problems in classification, modeling control and in a considerable number of industry applications. It has been shown as a powerful methodology for dealing with imprecision and nonlinearity efficiently (Wang, 1994). However, one of the shortcomings of fuzzy logic is the lack of learning and adaptation capabilities. As we know, neural network (NN) is one of the important technologies towards realizing artificial intelligence and machine learning. Many types of neural networks with different learning algorithms have been designed and developed (Deng et al., 2002; Levin & Narendra, 1996; Narendra P Huang et al., 2004; Jang, 1993; Juang L Leng et al., 2004; Lin L Qiao & Wang, 2008). The technology of FNNs combines the profound learning capability of neural networks with the mechanism of explicit and easily interpretable knowledge presentation provided by fuzzy logic. In a word, FNN is able to represent meaningful real-world concepts in


Introduction
Fuzzy systems have been demonstrated their ability to solve different kinds of problems in classification, modeling control and in a considerable number of industry applications. It has been shown as a powerful methodology for dealing with imprecision and nonlinearity efficiently (Wang, 1994). However, one of the shortcomings of fuzzy logic is the lack of learning and adaptation capabilities. As we know, neural network (NN) is one of the important technologies towards realizing artificial intelligence and machine learning. Many types of neural networks with different learning algorithms have been designed and developed (Deng et al., 2002;Levin & Narendra, 1996;Narendra &Parathasarathy, 1998). Recently, there is an increasing interest to hybridize the approximate reasoning method of fuzzy systems with the learning capabilities of neural networks and evolutionary algorithms. Fuzzy neural network (FNN) system is one of the most successful and visible directions of that effort. FNNs as hybrid systems have been proven to be able to reap the benefits of fuzzy logic and neural networks. In these hybrid systems, standard neural networks are designed to approximate a fuzzy inference system through the structure of neural networks while the parameters of the fuzzy system are modified by means of learning algorithms used in neural networks. One purpose of developing hybrid fuzzy neural networks is to create selfadaptive fuzzy rules for online identification of a singleton or Takagi-Sugeno-kang (TSK) type fuzzy model (Takagi & Sugeno, 1985) of a nonlinear time-varying complex system. The twin issues associated with a fuzzy system are 1) parameter estimation which involves determining parameters of premises and consequences and 2) structure identification which involves partitioning the input space and determining the number of fuzzy rules for a specific performance. FNN systems have been found to be very effective and of widespread use in several fields. In recent years, the idea of self-organization has been introduced in hybrid systems to create adaptive models. Some adaptive approaches also have been introduced in FNNs whereby not only the weights but also the structure can be self-adaptive during the learning process ( Er & Wu, 2002;Huang et al., 2004;Jang, 1993;Juang &Lin, 1998;Leng et al., 2004;Lin &Lee, 1996;Qiao & Wang, 2008). The technology of FNNs combines the profound learning capability of neural networks with the mechanism of explicit and easily interpretable knowledge presentation provided by fuzzy logic. In a word, FNN is able to represent meaningful real-world concepts in comprehensive knowledge bases. The typical approach of designing an FNN system is to build standard neural networks first, and then incorporate fuzzy logic in the structure of neural networks. The key idea is as follows: Assuming that some particular membership functions have been defined, we begin with a fixed number of rules by resorting to either trial-and-error methods or expert knowledge. Next, the parameters are modified by learning algorithm such as backpropagation (BP) algorithm (Siddique & Tokhi, 2001). The BP is a gradient descent search algorithm. It is based on minimization of the total mean squared error between the actual output and the desired output. This error is used to guide the search of the BP algorithm in the weight space. The BP is widely used in many applications in that it is not necessary to determine the exact structure and parameters of neural networks in advance. However, the problem of the BP algorithm is that it is often trapped in local minima and the learning speed is very slow in searching for global minimum of the search space. The speed and robustness of the BP algorithm are sensitive to several parameters of the algorithm and the best parameters vary from problems to problems. Therefore, many adjustment methods have been developed, notably evolutionary algorithms such as genetic algorithms (GAs) or particle swarm optimization (PSO). By working with a population of solutions, the GA can seek many local minima, and thus increase the likelihood of finding global minimum. This advantage of GA can be applied to neural networks to optimize the topology and parameters of weights. The key point is to employ an evolutionary learning process to automate the designing of the knowledge base, which can be considered as an optimization or search problem. The GA is used to optimize the parameters of neural networks (Seng et al., 1999;Siddique & Tokhi, 2001;Zhou & Er, 2008) or identify the optimal structure of neural networks (Chen et al., 1999;Tang et al., 1995,). Moreover, the adaptation of neural network parameters can be performed by different methods such as orthogonal least square (OLS) (Chen et al.,1991), recursive least square (RLS) (Leng et al., 2004), linear least square (LLS) (Er & Wu, 2002), extended Kalman filter (EKF) (Kadirkamanathan & Niranjan, 1993;Er et al., 2010) and so on. The objective of this chapter is to develop FNNs by hybrid learning techniques so that these systems can be used for online identification, model and control nonlinear and time-varying complex systems. In this chapter, we propose two kinds of self-organizing FNN that attempt to combine fuzzy logic with neural network and apply these learning algorithms to solve several well-known benchmark problems such as static function and linear and nonlinear function approximation, Mackey-Glass time-series prediction and real-world benchmark regression prediction and so on. The chapter is organized as follows. The general frame of self-organizing FNN is described in Section 2. The first learning algorithm combined FNN with EKF is presented in Section 3. It is simple and effective and is able to generate a FNN with high accuracy and compact structure. Furthermore, a novel neuron pruning algorithm based on optimal brain surgeon (OBS) for self-organizing FNN is described in Section 4 in detail. Simulation studies on several well-known benchmark problems and comparisons with other learning algorithms have been conducted in each section. The summary of FNN associated with conclusions and future work are discussed in Section 5.

General frame of self-organizing FNNs
The self-organizing fuzzy neural network system primarily implements TSK or TS type (Sugeno & Kang, 1988) fuzzy model. The general architecture is depicted in Fig. 1. This fivewww.intechopen.com layer self-organizing FNN implements a TSK type fuzzy system. Without loss of generality, we consider a multi-input-single-output (MISO) fuzzy model with input vector X=(x 1 ,x 2 ,…,x r ) and output variable y. Layer 1 is an input layer and transmits values of input linguistic variable (1 , 2 , ) i xi r = to layer 2 directly, where r is the number of input variables. Each input variable i x has l membership functions (MFs) ( 1,2, , 1,2, , ) ij ir j l μ = = as shown in layer 2. The MFs can be triangular function (G. Leng et al., 2009) or Gaussian function (Wu and Er, 2004)  (2) www.intechopen.com Layer 4 is the normalized layer. The number of neurons in this layer is equal to that of layer 3. The output of the jth neuron in this layer is 1 ,1 , 2 , , Layer 5 is output layer and each node represents an output linguistic variable. The output of this layer is the weighted summation of incoming signals given by where ψ ij is the output of the ith neuron in the normalized layer when jth training data arrives.

Self-constructing Fuzzy Neural Networks with Extended Kalman Filter (SFNNEKF)
In this section, a self-constructing fuzzy neural network employing extended Kalman filter (SFNNEKF) is designed and developed. The learning algorithm based on EKF is simple and effective and is able to generate a FNN with a high accuracy and compact structure. The proposed algorithm comprises of three parts: (1) criteria of rule generation; (2) pruning technology and (3) adjustment of free parameters. The EKF algorithm is used to adjust the free parameters of the SFNNEKF. The performance of the SFNNEKF is compared with other learning algorithms in the tasks of function approximation, nonlinear system identification and time-series prediction. Simulation studies and comparisons with other algorithms demonstrate that a more compact structure with high performance can be achieved by the proposed algorithm.

Learning algorithm of SFNNEKF 3.1.1 Error Reduction Ratio (ERR)
Suppose that for n observations, the FNN has generated uRBF neurons. The output matrix of the hidden layer is given by (8) then (6) can be written as where n TR ∈ is the desired output vector and E is the error vector, T Φ =Ψ can be rewritten as follows: where K is a n × u matrix with orthogonal columns and A is a u × u upper triangular matrix. Substituting (10) into (9), we obtain The orthogonal lease squares solution G is given by An error reduction ratio (ERR) due to k i as defined in (Shen et al., 1991) is given by Substituting (12) into (13) yields 2 () 1 The ERR offers a simple and effective way of seeking a subset of significant regressors. In this chapter, we use it as a growing criterion to evaluate the network generalization capability. We define If η < k err , where k err is a prespecified threshold, the FNN needs more hidden nodes to achieve good generalization performance and a neuron will be added to the network. Otherwise, no neurons will be added.

Criteria of growing neurons
For the ith observation(P i ,t i ), calculate the output error of the SFNNEKF, e i , the distance d i (j) and the ERR as follows: a new neuron is added to the SFNNEKF network and the center, width and the output layer weight for the newly generated neuron are set as follows: (24) where k is an overlap factor that determines the overlap of responses of the RBF units.

Criteria of pruning neurons
To facilitate the following discussion, we rewrite the output matrix of the hidden layer, as follows: In order to evaluate the contribution of each hidden neuron to the network output, the root mean square error (RMSE) of each hidden node is calculated as follows: /, 1 , , is the column vector of matrix Φ . If ir m s e hk < where rmse k is a prespecified threshold, the ith hidden neuron is inactive and should be deleted. www.intechopen.com

Adjustment of weights
When no neurons are added or pruned from the network, the network parameter vector W EKF is adjusted using the EKF method (Kadirkamanathan & Niranjan, 1993) Here, a n is the gradient vector and has the following form where R n is the variance of the measurement noise and P n is the error covariance matrix which is updated by where Q is a scalar that determines the allowed random step in the direction of gradient vector and I is the identity matrix. When a new neuron is allocated, the dimensionality of the P n increases to where P 0 is an estimate of uncertainties in the initial values assigned to the parameters.

Function approximation
First, a very popular function, Hermite polynomial, is used to evaluate the performance of the proposed SFNNEKF. The function is given by  Fig. 1 shows the growth of hidden neurons and Fig. 2 shows the RMSE with respect to the training samples. A comparison of structure and performance with MRAN (Lu et al., 1997) and DFNN (Er & Wu, 2002) is summarized in Table 1. It can be seen that the DFNN algorithm achieves the best RMSE performance in this case, but it needs more training time, because the TSK model makes the DFNN network structure more complicated. The SFNNEKF algorithm can obtain almost the same RMSE performance as the DFNN with faster learning speed and it has better performance than the MRAN algorithm in terms of training time and RMSE.

Nonlinear plant identification
The plant be identified is described by the second-order difference equation (Narendra & Parathasarathy, 1990) (1 ) [ ( ) , (1 ) ]( ) y tf y t y tu t Parameters of the SFNNEKF are set as follows: The input u(k) is assumed to be independent identically distributed (i.i.d) random signal uniformly distributed in the interval [−2, 2] and a total of 2000 samples are randomly chosen for the training process. After this, the sinusoidal input signal () s i n ( 2 /2 5 ) ut t π = is used to test identified model. Fig. 3 depicts the identified results using the SFNNEKF algorithm. In order to observe the result clearly, only the first 100 samples are shown in Fig. 3. Here, the solid-line curve is the desired plant output and the dash-line curve is for the identified model output. It can be seen that the SFNNEKF algorithm can identify the plant very well. The RMSE for training samples is shown in Fig. 4. The SFNNEKF algorithm generated a total of 15 neurons at the end of the training process.

Mackey-Glass time-series prediction
The Mackey-Glass time-series prediction is a benchmark problem which has been considered by a number of researchers (Lu et al., 1997;Wu & Er, 2000;Cho &Wang, 1996;Juang & Tin, 1998;Kadirkamanathan & Niranjan, 1993). The time series in our simulation is generated by for a=0.1, b=0.2 and τ=17. A sampled version of the series is obtained by the above equation.
The prediction model is given by For the purpose of training and testing, 6000 samples are generated between t=0 and t=6000 from (33) with initial conditions x(t)=0 for t<0 and x(0)=1.2. Hence, these data are used for preparing for the input and output pairs in the (34).  Fig. 5 and Fig. 6. Figures illustrate the simulation results of the SFNNEKF in the case that p is 6 and the number of rules is 34. It indicates that the SFNNEKF possesses remarkable generalization capability. If we selected p=50 and Δt=6, for the convenience of comparison, the generalization capability is evaluated by the normalized root mean squared error (NRMSE) or the non-dimensional error index (NDEI), which is defined as the RMSE divided by the standard deviation of the target series (Platt, 1991). Generalization comparisons between the SFNNEKF, DFNN (Er & Wu, 2002), RAN (Platt, 1991) RANEKF (Kadirkamanathan & Niranjan, 1993), and MRAN (Lu et al., 1997) is listed in table 3. It is shown that SFNNEKF can obtain better performance even it has generated more rules than DFNN. However, the SFNNEKF shows superiority compared with the RAN and RANEKF algorithms in terms of the performance of the NRMSE, which measures the average prediction performance. The final value of the NRMSE is 0.0327 for SFNNEKF with 41 rules compares with 0.071 for the RAN and 0.056 for the RANEKF.

Conclusions
A new self-constructing fuzzy neural network has been proposed in this part. The basic idea of the proposed approach is to construct a self-constructing fuzzy neural network based on criteria of generating and pruning neurons. The EKF algorithm has been used to adapt the consequent parameters when a hidden unit is not added. The superior performance of the SFNNEKF over some other learning algorithms has been demonstrated in three examples in this part. Simulation results show that a more effective fuzzy neural network with high accuracy and compact structure can be self-constructed by the proposed SFNNEKF algorithm.

A neuron pruning algorithm based on optimal brain surgeon for selforganizing fuzzy neural networks with parsimonious structure
In this section, a novel learning algorithm for creating a self-organizing FNN to implement TSK type fuzzy models is proposed. The optimal brain surgeon (OBS) (Hassibi & Stork, 1993) is employed as a neuron pruning mechanism to remove unimportant neurons directly during the training procedure. Distinguished from other pruning strategies based on the OBS, there is no need to calculate the inverse matrix of Hessian, we simplify the calculation by using LLS method to obtain the Hessian matrix. To acquire precision model, the LLS method is performed to obtain the consequent parameters of the network. The proposed algorithm has a parsimonious structure and is generated with high accuracy. The effectiveness of the proposed algorithm is demonstrated in several well-known benchmark problems such as static function approximation, two-input nonlinear function approximation and real-world non-uniform benchmark regression problems. Simulation studies are compared with other existing published algorithms. The results indicate that the proposed algorithm can provide comparable approximation and generalization performance with a more compact structure and higher accuracy.

Learning algorithm of the FNN
The learning procedure of the proposed self-organizing FNN comprises two stages. In the first stage, the structure of the FNN is generated based on the growth criteria and the proposed OBS-based pruning method. In the second stage, parameters of newly created neurons are assigned and consequent parameters of all existing neurons will be updated by the LLS method.

Structure design of the FNN
To determine the proper number of neurons or fuzzy rules of FNNs, we adopt two growth criteria to generate neurons. In order to obtain a compact structure, the OBS algorithm (Hassibi & Stork, 1993) is employed as a pruning criterion. An initial structure of the FNN is first constructed and the importance of each hidden neuron or fuzzy rule is evaluated by the OBS algorithm, the least important neuron will be deleted if the performance of the entire network is accepted after deleting this unimportant neuron. This procedure will repeat until the desired accuracy can be satisfied. We will describe the learning algorithm in detail in the sequel. The learning stage is based on a data set composed by n input-output pairs: When the kth observation (X t ,t k ) arrives, the overall FNN output of the existing structure using (2)-(4) is denoted by y k , the system error can be defined by (16) where e max is the maximum error chosen, e min is the desired accuracy of the FNN output and β∈(0,1)is a convergence constant.
The second growth criterion is the accommodation criterion. It can be described as follows: when the kth observation (X t ,t k ) arrives, calculate the distance d kj between the observation X k and the center C j of the u existing neurons, i.e. ,1 , 2 , , ,1 , 2 , , the minimum distance between the kth observation and the nearest center is described as min argmin( ), 1,2, , , is the decay constant. If these two growth criteria are satisfied simultaneously, a new hidden neuron will be generated. When growth procedure is complete, a pruning algorithm is employed to optimize the structure of the FNN to obtain a compact network structure. For this algorithm, we use the OBS as a pruning strategy. The OBS algorithm uses the second-order derivative information of the error function to find the unimportant neuron to get a tradeoff between the network complexity and good performance of entire network. The basic idea of the OBS algorithm is described as follows. Firstly, we assume that the pruning algorithm is used after the training process has converged or is fully trained. It means that the network reaches a local minimum of the error surface. The functional Taylor series of the cost function E with respect to parameters w is given by is the Hessian matrix which contains all second-order derivative information. For a network trained to a local minimum in error, and we also ignore the third and all higher order terms. The goal is to set one of the parameters w q to zero to minimize the increase in error given by (41). Eliminating w q can be expressed as Then the optimal model of OBS can be formulated as an optimization problem as follows where I q is the unt vector in parameter space corresponding to parameter w q . Solving the constrained problem using the Lagrangian method Note that the OBS algorithm needs to calculate the inverse of the Hessian matrix. In fact, for very small dimension of the Hessian matrix, its inverse can be obtained by standard matrix methods while inverting a matrix of hundreds or thousands of terms seems computationally intractable. For our algorithm, we use the LLS method to avoid calculating the inverse of the Hessian matrix. The cost function used in our algorithm is defined as the training error () In the following part, we will discuss how to use the Hessian matrix H to measure the increase in cost function () t E Θ due to deleting some parameters and then present the pruning algorithm. From (11) and (53) the Hessian matrix can be rewritten as After obtaining the Hessian matrix H , we can calculate the increase in cost function () t E Θ due to a change in parameters Θ . Since deleting the ith parameter in Θ is equivalent to setting the value of the corresponding parameter to zero, then the change in () t E Θ due to the removal of a subset G of parameters is After all, the importance of each hidden neuron or fuzzy rule can be evaluated by the change in cost function or training squared error ΔE t (Θ), the smaller the value of the change in the cost function, the less important is the neuron. Based on these descriptions, the proposed OBS-based pruning algorithm can be summarized as follows , remove this neuron, and go to step 7) for the second least important neuron, and so on. Otherwise, do not remove any neurons and proceed to the weight adjustment stage. Therefore, the structure (number of hidden neurons or fuzzy rules) of FNN is obtained based on the growth and pruning criteria.

Update of parameters and weight adjustment
After the network structure is established, the network enters the second stage: parameters of newly created neurons are assigned and the consequent parameters of all existing neurons will be updated by the simple but efficient LLS method. σ is the center and width of the first neuron respectively, and 1 w is the connective weight of the first neuron. The objective of weight adjustment is as follows We try to find an optimal weight vector (1 ) * ur WR × + ∈ such that the error energy T EE is minimized. By using the LLS method, we can find the optimal * W is in the form of Since the structure of our FNN is very compact after structure learning process, the dimension of W and Ψ is small, the LLS method provides a computationally simple and efficient weight adjustment process.

Static function approximation
The first example is on static function approximation. The underlying function to be approximated is a three-input nonlinear function f(x,y,z) which is widely used to verify the effectiveness of proposed algorithms of Khayat et al., 2009;Wu et al., 2001;Qiao & Wang, 2008;Chen et al., 1991.
where n is the number of data pairs, t(k) and y(k) are the kth desired output and actual output of the FNN, respectively.
Simulation results are shown in Fig. 7, where we see that a four-hidden-neuron structure is generated. The RMSE curving of training data with 216 epochs is shown in Fig. 8. The input variables x,y and z define four membership functions respectively. As we can see from the resulting figures, the trained network obtains a high precision and the RMSE is also confined to a very small value.   Comparative studies of the proposed algorithm with other published works, such as the OLS (Chen et al., 1991), GDFNN (Wu et al., 2001) and SOFNN (Leng et al., 2005) are shown in Table 4. APE trn % and APE chk % are APE of training data and testing data, respectively. As it can be seen from Table 4, the proposed algorithm has the least number of parameters which means that it achieves the most compact structure compared with other algorithms while the approximation performance is worse than that of the SOFNN and GDFNN and nearly the same as that of the OLS. It should be noted that the proposed algorithm has a satisfactory generalization performance as its APE of testing is only 1.55 which is less than OLS and almost the same as that of the GDFNN. In brief, the proposed algorithm achieves the most parsimonious structure with good accuracy.

Two-input nonlinear sinc function
This example is used to demonstrate the performance of the proposed algorithm in noisy and noise-free environments. This function is also used in Leng et al., 2004 andLeng et al., 2009. It is defined as follows A total of 5 hidden neurons (i.e. fuzzy rules) FNN is generated. The number of membership functions associated inputs x and y is five. When the training data is without noise, the RMSE for training is 0.0073 whilst the RMSE for testing is 0.0057. In order to determine the affect of noise, the training data are mixed with Gaussian white noise sequences which have zero mean and different variances as shown in Table 5 It can be observed from Table 5, the number of hidden neurons increases with the level of noise. That is because more neurons will be required to filter the noise. Both the RMSE for training and testing are small values though the data mixed with high level of noise. It means that the proposed algorithm is robust even when the training data is perturbed by noise. It demonstrates that our algorithm is suitable for noisy environment.

Boston housing prediction
The Boston housing prediction problem is to predict the median value of owner-occupied homes in suburbs of Boston. The database has 506 observations and each observation consists of 13 inputs (12 continuous inputs and one binary-valued input) and one continuous output (median value of owner-occupied homes The proposed algorithm 2 0.0916 0.0583 481 25 Table 6. Comparisons of the proposed algorithm with other methods (Boston housing) As observed from Table 6, the proposed algorithm obtains the smallest RMSE for training and testing, which means that the approximation and generalization performance is better than that of other learning algorithms. It is highlighted that the average number of hidden neurons generated by the system is only 2, which is less than that of other learning algorithms. Therefore, the proposed algorithm has the most compact structure of all the learning algorithms. In other words, our algorithm needs the least number of neurons to conduct the Boston housing prediction with comparable accuracy simultaneously.

Auto-mpg prediction
The auto-mpg problem is to predict the fuel consumption (miles per gallon) of different models of cars based on the displacement, horsepower, weight and acceleration of cars. A total of 392 observations are collected for the prediction problem. Each observation consists of seven inputs (four continuous inputs: displacement, horsepower, weight, acceleration, and three discrete inputs: cylinders, model year and origin) and one continuous output (the fuel consumption). For simplicity, the seven input attributes and one output have been normalized to the range [0, 1]. A total of 320 training data and 72 testing data are randomly chosen from the auto-mpg data set in each trial of simulation studies. The training parameters are selected as the previous Boston housing problem except the expected RMSE for training set is as k RMSF =0.008. Performance comparisons among RAN (Platt, 1991), RANEKF (Kadirkamanathan & Niranjan, 1993), MRAN (Lu et al., 1997), GAP-RBF (Huang et al., 2004), OS-ELM (Liang et al., 2006), FAOS-PFNN (Wang et al., 2009) and our proposed algorithm are presented in Table 7. It can be seen that the RMSE for testing our algorithm is the smallest among these learning algorithms while the RMSE for training is less than other algorithms except the FAOS-PFNN. It can be summarized that our proposed algorithm has the best generalization ability of all the learning algorithms though the approximation performance is a little worse than the FAOS-PFNN. Moreover, the number of fuzzy rules of our algorithm is only 2 and is less than other algorithms so that it has the most parsimonious structure among these learning algorithms. In brief, the overall performance of our algorithm is superior to other algorithms.

Discussions
In this part a novel learning algorithm for creating a self-organizing fuzzy neural network (FNN) to implement the TSK type fuzzy model with a parsimonious structure was proposed. The OBS is employed as a pruning algorithm to remove unimportant neurons directly during the training process. Apart from other pruning strategies based on the OBS, there is no need to calculate the inverse matrix of Hessian; we simplify the calculation by using the LLS method to obtain the Hessian matrix. The effectiveness of the proposed algorithm has been demonstrated in four well-known benchmark problems: namely static function approximation, nonlinear dynamic system identification, two-input nonlinear function and real-world non-uniform benchmark problems. Moreover, performance comparisons with other learning algorithms have also been presented in this part. The results indicate that the proposed algorithm can provide comparable approximation and generalization performance with a more compact parsimonious structure and higher accuracy.

Conclusions and future work
In this chapter, the development of fuzzy neural networks has been reviewed and the main issues for designing fuzzy neural networks including growing and pruning criteria and different adjustment methods of consequent parameters have been discussed. The general frame of fuzzy neural networks based on radial basis function neural networks has been described in Section 2. Two self-organization FNNs have been developed. For the first FNN, the SFNNEKF algorithm employs ERR as a generation condition in constructing the network which makes the growth of neurons smooth and fast. The EKF algorithm has been used to adjust free parameters of the FNN to achieve an optimal solution. Simulation results show that a more effective fuzzy neural network with high accuracy and compact structure can be self-constructed by the proposed SFNNEKF algorithm. For the second FNN, it is composed of two stages: the structure identification stage and the parameter adjustment stage. The structure identification consists of constructive and pruning procedures. An initial structure starts with no hidden neurons or fuzzy rule sets and grows neurons based on the criteria of neuron generation. Then the OBS is employed as a pruning strategy to further optimal the obtained initial structure. At last, the well-known LLS method is adopted to tune the free parameters in the parameter adjustment stage for sequentially arriving training data pairs. Simulation studies are compared with other algorithms. The simulation results indicate that the proposed algorithm can provide comparable approximation and generalization performance with a more compact structure and higher accuracy. In a word, fuzzy neural networks are hybrid systems that combine the advantages of fuzzy logic and neural networks, there existed many kinds of FNN developed by researchers. Recently, the idea of self-organizing has been introduced in FNN. The purpose is to develop self-organizing fuzzy neural network systems to approximate fuzzy inference through the structure of neural networks to create adaptive models, mainly for approximate linear and nonlinear and time-varying systems. FNNs have been widely used in many fields. For our future work, studies will focus on the structure learning since appropriate number of fuzzy rules or find proper network architecture and developing optimal parameter adjustment methods. The grandest accomplishments of engineering took place in the twentieth century. The widespread development and distribution of electricity and clean water, automobiles and airplanes, radio and television, spacecraft and lasers, antibiotics and medical imaging, computers and the Internet are just some of the highlights from a century in which engineering revolutionized and improved virtually every aspect of human life. In this book, the authors provide a glimpse of the new trends of technologies pertaining to control, management, computational intelligence and network systems.