The present paper shows mathematical foundations of ICA (independent component analysis) and related subjects of signal representations. Information geometry plays a basic role for elucidating the structure of the problem underlying signal representation and decomposition. The method of estimating function is used for the analysis of errors and stability for various ICA algorithms. The nonholonomic method is of particularly interest.

Blind source separation (BSS) and related methods such as independent component analysis (ICA) and their extensions or sparse component analysis (SCA) refers to wide class of problems in signal and image processing, when one needs to extract the underlying sources from a set of mixture. The goal of BSS can be considered as estimation of true physical sources and parameters of a mixing system, while objective of generalized component analysis (GCA) is finding a new reduced or hierarchical and structured representation for the observed (sensor) multidimensional data that can be interpreted as physically meaningful coding or blind signal decompositions. These methods are generally based on a wide class of unsupervised learning algorithms and they found potential applications in many areas from engineering to neuroscience. The recent trends in blind source separation and generalized component analysis is to consider problems in the framework of matrix factorization or more general signals decomposition with probabilistic generative and tree structured graphical models and exploit some priori knowledge about true nature and structure of latent (hidden) components or sources such as spatio-temporal decorrelation, statistical independence, sparsity, nonnegativity, smoothness or lowest possible complexity. The key issue is to find a such transformation or coding which has true physical meaning and interpretation. In this paper we discuss some promising approaches and algorithms for BSS/GCA, especially for ICA and SCA in order to analyze, enhance, perform feature extraction, removing artifacts and denoising of multi-modal, multi-sensory data.

We employ unsupervised clustering techniques for the analysis of dynamic contrast-enhanced perfusion MRI time-series in patients with and without stroke. "Neural gas" network, fuzzy clustering based on deterministic annealing, self-organizing maps, and fuzzy c-means clustering enable self-organized data-driven segmentation w.r.t.fine-grained differences of signal amplitude and dynamics, thus identifying asymmetries and local abnormalities of brain perfusion. We conclude that clustering is a useful extension to conventional perfusion parameter maps.

Exploratory data analysis techniques are applied to the segmentation of lesions in MRI mammography as a first step of a computer-aided diagnosis system. ICA and clustering techniques are tested on biomedical time-series representing breast MRI scans. This techniques enable the extraction of spatial and temporal features of dynamic MRI data stemming from patients with confirmed lesion diagnosis. By revealing regional properties of contrast-agent uptake characterized by subtle differences of signal amplitude and dynamics, these methods provide both a set of prototypical time-series and a corresponding set of cluster assignment maps which further provide a segmentation with regard to identification and regional subclassification of pathological breast tissue lesions.

In this paper, we investigate the application of independent component analysis (ICA) to remotely sensed hyperspectral image classification. We focus on the performance of Joint Approximate Diagonalization of Eigenmatrices (JADE) algorithm, although the proposed method is applicable to other popular ICA algorithms. The major advantage of using ICA is its capability of classifying objects with unknown spectral signatures in an unknown image scene, i.e., unsupervised classification. However, ICA suffers from computational expensiveness, which limits its application to high dimensional data analysis. In order to make it applicable to hyperspectral image classification, a data preprocessing procedure is employed to reduce the data dimensionality. Noise adjusted principal component analysis (NAPCA) is used for this purpose, which can reorganize the original data information in terms of signal-to-noise ratio, a more appropriate criterion than variance when dealing with images. The preliminary results demonstrate that the selected major components from NAPCA can better represent the object information in the original data than those from ordinary principal component analysis (PCA). As a result, better classification using ICA is expected.

We present techniques for performing image reconstruction based on deconvolution in the Radon domain. To deal with a variety of possible boundary conditions, we work with a corresponding generalized discrete Radon transform in order to obtain projection slices for deconvolution. By estimating the projections using wavelet techniques, we are able to do deconvolution directly in a ridgelet domain. We also show how this method can be carried out locally, so that deconvolution can be done in a curvelet domain as well. These techniques suggest a whole new paradigm for developing deconvolution algorithms, which can incorporate leading deconvolution schemes. We conclude by showing experimental results indicating that these new algorithms can significantly improve upon current leading deconvolution methods.

We determined the overall disparity from stereo images using multiscale products of disparity values. Using the wavelet transform, we used a multiresolution analysis to calculate disparity values from stereo images at different scales. Forming the product of disparities and rescaling resulted in a disparity map that was directly related to depth. Using a multiresolution approach allowed us to more accurately determine disparity by examining the consistency of results through different scales. Using this approach could form the basis of an effective method for depth estimation.

We propose a WAVENET method for feature extraction of high-range resolution (HRR) radar profiles. Because HRR signals constantly vary with incremental changes in time and target aspect, the inverse problem we address is that of extracting a subset of discriminatory features from a set of HRR profiles that are unique to each target class. Based on, we construct a neural net technique built on wavelets for determining the discriminating features separating each target class. The method involves choosing a suitable set of child wavelets, such that the transformation of the original data (the training set of HRR profiles) will enhance the nonlinear separability of different classes of target signals while significantly reducing the dimension of the data.

Feature extraction methods based on the discrete wavelet transform and multiresolution analysis are used to develop a robust classification algorithm that reliably discriminates between conventional and simulated chemical/biological artillery rounds via acoustic signals produced during detonation. Distinct characteristics arise within the different airburst signatures because high explosive warheads emphasize concussive and shrapnel effects, while chemical/biological warheads are designed to disperse their contents over large areas, therefore employing a slower burning, less intense explosive to mix and spread their contents. The ensuing blast waves are readily characterized by variations in the corresponding peak pressure and rise time of the blast, differences in the ratio of positive pressure amplitude to the negative amplitude, and variations in the overall duration of the resulting waveform. Unique attributes can also be identified that depend upon the properties of the gun tube, projectile speed at the muzzle, and the explosive burn rates of the warhead. In this work, the discrete wavelet transform is used to extract the predominant components of these characteristics from air burst signatures at ranges exceeding 2km. Highly reliable discrimination is achieved with a feedforward neural network classifier trained on a feature space derived from the distribution of wavelet coefficients and higher frequency details found within different levels of the multiresolution decomposition.

Adaptive wavelets are important when the signals or environment are changing with time. Interferometric sensors (moire, Michelson, Mach-Zehnder, or shearing interferometers) are critical to many Smart Structures applications. Adaptive wavelets will be developed to achieve phase distortion correction. Phase distortion correction will be achieved by means of wavelet ridge extraction and phase conjugation.

The health of a sensor and system is monitored by information gathered from the sensor. A normal mode of operation is established. Any deviation from the normal behavior indicates a change. An RC network is used to model the main process, which is defined by a step-up (charging), drift, and step-down (discharging). The sensor disturbances and spike are added while the system is in drift. The system runs for a period of at least three time-constants of the main process every time a process feature occurs (e.g. step change). To extract time information and shape isolation the Wavelet Transform is used. The results are analyzed using continuous as well as discrete wavelet transforms. The results indicate distinct shapes corresponding to each process. The Wavelet Transform results are compared to the signal average power using hamming window and Fourier Transform. The Fourier Transform analysis of the signal is carried out by selecting each point of the signal with a window of trailing data collected previously. Two trailing window lengths are selected; one equal to two time constant of the main process and the other equal to two time constant of the sensor disturbance. Next, the DC is removed from each set of data and then the data are passed through a window followed by calculation of spectra for each set. In order to extract features, the signal power, peak, and spectral area are plotted vs. time

Wireless sensor networks provide an opportunity of innovations while also brings unique challenges. Collaborations between sensor nodes are generally required for complicated applications. As a key component for node collaboration, robust routing protocols that are able to effectively communicate among multiple nodes become necessary. In wireless sensor networks, these protocols are energy efficient and provide low latency. In this paper, we develop optimal distance geographic routing (ODGR), an application-independent protocol using power control in the transmission scheme and the available geographic information to dynamically explore the optimal routing path in order to reduce the total transmission energy and routing latency. ODGR is built on the optimal distance theory derived exclusively from the fundamental transmission energy model. Detail procedures of ODGR are presented for practical implementations. Case study of a two-dimensional mesh network shows that ODGR is able to reduce total transmission energy by 66.41% and 43.89%, and average latency by 76.45% and 26.27% respectively compared to traditional MTE and cluster algorithms, thus ODGR can improve the system-life and the performance of wireless sensor networks.

In this paper we describe the implementation of a vision system based on an optoelectronic neural network architecture which is based on an optical broadcast interconnection scheme. The architecture of the neural network processor has been designed to exploit the computational characteristics of electronics and the communication characteristics of optics, thus it is based on an optical broadcast of input signals to a dense array of processing elements. In the proposed vision system, a CMOS sensor capture the image of an object, the output of the camera is introduced to the optoelectronic processor which compares the input image with a set of reference patterns, the optoelectronic processor provides the reference pattern that best match with the input image. The processing core of the system is an optoelectronic architecture that has been configured as a Hamming neural network.

Neural network based image processing algorithms present numerous advantages due to their supervised adjustable weight and bias coefficients. Among various neural network architectures, dynamic neural networks, Hopfield and Cellular neural networks have been found inherently suitable for filtering applications. These kind of neural networks present two important features; supervised learnable and optimization properties. Using these properties, dynamic neural filtering technique has been developed based on Hopfield neural networks. The filtering structure involves adjustable a filter mask and 2D convolution operation instead of weight matrix operations. To improve the supervised training properties, Widrow-recurrent learning algorithm has been proposed in this paper. Since the proposed learning algorithm requires less computation, consumption time in the training stage has been decreased considerably compared to previous reported supervised techniques for dynamic neural filtering.

Recent research on Neocognitron-like neural feed-forward architectures, which have formerly been successfully applied to recognition of artifical stimuli like paperclip objects, is promising application to more natural stimuli. Several authors have shown high recognition performance of such networks with respect to translation, rotation, scaling and cluttered surroundings. In this contribution, we introduce a variation of existing hierarchical models, that is trained using a non-negative matrix factorization algorithm. In contrast to previous work, our approach can not only classify objects but is also capable of rapid object detection in natural scenes. Thus, the time-consuming and conceptually unsatisfying split-up into a localization stage (e.g. using segmentation) and a subsequent classification can be avoided. Though in principle an exhaustive search by classification of every sub-window of an image is performed, the process is nevertheless highly efficient. The network consists of alternating layers of simple and complex cell planes and incorporates nonlinear processing schemes that have been proposed in recent literature. Learning of receptive field profiles for the lower layers of the network takes place by unsupervised learning whereas a final classification layer is trained supervised. Detection is achieved by attaching an additional network layer, whose simple cell profiles are learned from the final classification units that were acquired during the training phase. We test the classification performance of the network on images of natural objects which are systematically distorted. To test the ability to detect objects, cluttered natural background is used.

Recent advances in intelligence technology have boosted the development of micro- Unmanned Air Vehicles (UAVs) including Sliver Fox, Shadow, and Scan Eagle for various surveillance and reconnaissance applications. These affordable and reusable devices have to fit a series of size, weight, and power constraints. Cameras used on such micro-UAVs are therefore mounted directly at a fixed angle without any motion-compensated gimbals. This mounting scheme has resulted in the so-called jitter effect in which jitter is defined as sub-pixel or small amplitude vibrations. The jitter blur caused by the jitter effect needs to be corrected before any other processing algorithms can be practically applied. Jitter restoration has been solved by various optimization techniques, including Wiener approximation, maximum a-posteriori probability (MAP), etc. However, these algorithms normally assume a spatial-invariant blur model that is not the case with jitter blur. Szu et al. developed a smart real-time algorithm based on auto-regression (AR) with its natural generalization of unsupervised artificial neural network (ANN) learning to achieve restoration accuracy at the sub-pixel level. This algorithm resembles the capability of the human visual system, in which an agreement between the pair of eyes indicates "signal", otherwise, the jitter noise. Using this non-statistical method, for each single pixel, a deterministic blind sources separation (BSS) process can then be carried out independently based on a deterministic minimum of the Helmholtz free energy with a generalization of Shannon's information theory applied to open dynamic systems. From a hardware implementation point of view, the process of jitter restoration of an image using Szu's algorithm can be optimized by pixel-based parallelization. In our previous work, a parallelly structured independent component analysis (ICA) algorithm has been implemented on both Field Programmable Gate Array (FPGA) and Application-Specific Integrated Circuit (ASIC) using standard-height cells. ICA is an algorithm that can solve BSS problems by carrying out the all-order statistical, decorrelation-based transforms, in which an assumption that neighborhood pixels share the same but unknown mixing matrix A is made. In this paper, we continue our investigation on the design challenges of firmware approaches to smart algorithms. We think two levels of parallelization can be explored, including pixel-based parallelization and the parallelization of the restoration algorithm performed at each pixel. This paper focuses on the latter and we use ICA as an example to explain the design and implementation methods. It is well known that the capacity constraints of single FPGA have limited the implementation of many complex algorithms including ICA. Using the reconfigurability of FPGA, we show, in this paper, how to manipulate the FPGA-based system to provide extra computing power for the parallelized ICA algorithm with limited FPGA resources. The synthesis aiming at the pilchard re-configurable FPGA platform is reported. The pilchard board is embedded with single Xilinx VIRTEX 1000E FPGA and transfers data directly to CPU on the 64-bit memory bus at the maximum frequency of 133MHz. Both the feasibility performance evaluations and experimental results validate the effectiveness and practicality of this synthesis, which can be extended to the spatial-variant jitter restoration for micro-UAV deployment.

Electronic mails are adopted worldwide; most are easily hacked by hackers. In this paper, we purposed a free, fast and convenient hybrid privacy system to protect email communication. The privacy system is implemented by combining private security RSA algorithm with specific chaos neural network encryption process. The receiver can decrypt received email as long as it can reproduce the specified chaos neural network series, so called spatial-temporal keys. The chaotic typing and initial seed value of chaos neural network series, encrypted by the RSA algorithm, can reproduce spatial-temporal keys. The encrypted chaotic typing and initial seed value are hidden in watermark mixed nonlinearly with message media, wrapped with convolution error correction codes for wireless 3rd generation cellular phones. The message media can be an arbitrary image. The pattern noise has to be considered during transmission and it could affect/change the spatial-temporal keys. Since any change/modification on chaotic typing or initial seed value of chaos neural network series is not acceptable, the RSA codec system must be robust and fault-tolerant via wireless channel. The robust and fault-tolerant properties of chaos neural networks (CNN) were proved by a field theory of Associative Memory by Szu in 1997. The 1-D chaos generating nodes from the logistic map having arbitrarily negative slope a = p/q generating the N-shaped sigmoid was given first by Szu in 1992. In this paper, we simulated the robust and fault-tolerance properties of CNN under additive noise and pattern noise. We also implement a private version of RSA coding and chaos encryption process on messages.

In the paper, we consider a new method of pulse-echo image analysis and its application for seismic prospecting of oil reservoir. The method has a high sensitivity in comparison with other well known techniques of seismic data analysis and consists of the following seven steps: 1) A moving window (3x3) slides on a seismic image and compares each pixel of analyzed image to unwrapped in a spiral order window. 2) Brightness of pixels into the window are considered as information features and ones is presented in a form of a same vector a of size (1x9). 3) Its complex analog b, obtained using a gradient transformation, which is compared to this vector. This step is needed for increasing a sensitivity of future segmentation procedure. 4) Coefficients of the vector b are considered as complex coefficients of some characteristic polynomial H(z). 5) Complex zeroes calculation of the polynomial H(z). This step gives a possibility for forming a new 3-D complex space of information features. 6) A new information basis is formed on the base of independent components, obtained after applying independent component analysis (ICA) to a magnitude of complex space-zero array. 7) The information fusion is used by means of Kohonen’s self-organizing map (SOM) algorithm. This step gives possibility to obtain one new resulting image. Experiments were made on the real-world pulse-echo image with known position of oil reservoir. The new method permitted to detect oil reservoir, while the known methods did not give any positive results.

A principled theoretical framework for a novel probabilistic source separation model for lattice data is derived using an invariant extension of the Extreme Physical Information (EPI) theory. The pdf’s of the estimated sources are parameterized by a unique ansatz that satisfies the invariant EPI (IEPI) game corollary. The mixing and unmixing matrices are described by unitary operators that diagonalize the Fisher information matrix, expressed in dimensionless form through application of the ansatz. The discrete IEPI source separation model is ascribed a quantum mechanical connotation.