This work deals with many techniques to accelerate motion compensation routines based on matching approaches. These routines comprise the most expensive and heavy part of many video coding standards, such as H.264, that have been evaluated, analyzed, and profiled. For achieving this, an exhaustive viability study of three very well-known matching-based motion compensation algorithms, widely used in multimedia coding, has been performed. These algorithms have been implemented in a low-cost FPGA using on the one hand three different architectural approaches, and on the other hand several algorithmic features. The approach technique deployed is based on optimizing the SoC performance using an efficient combination of On-chip memory and SDRAM regarding the reset vector, exception vector, stack, heap, read/write data (.rwdata), read only data (.rodata), and program text (.text) in the design. Additionally this approach addresses the enhancing of the algorithms referred previously with the incorporation of custom instructions in the Nios II ISA. The present approach present the multi-cycle customized instruction where an efficient combination of both methods is then developed to build the final embedded system. The results of these approaches suggest the accomplishment of a small sensor which processes 50×50 @ 180 frames per second, meaning almost real-time motion compensation for the common video coding QCIF format at 19 frames per second. The present work thus, opens the door to motion coding for the low-cost soft-core microprocessors, particularly the RISC architecture of Nios II implemented entirely in the programmable logic and memory blocks of Altera FPGAs. This work gets together contributions to different research fields like Computer Vision, Multimedia Coding, Block-Matching Techniques, and FPGA based Embedded Systems.

PNG (Portable Network Graphics) is a lossless compression method for real-world pictures. Since its specification, it continues to attract the interest of the image processing community. Indeed, PNG is an extensible file format for portable and well-compressed storage of raster images. In addition, it supports all of Black and White (binary mask), grayscale, indexed-color, and truecolor images. Within the framework of the Demat+ project which intend to propose a complete solution for storage and retrieval of scanned documents, we address in this paper a hardware design to accelerate the PNG encoder for binary mask compression on FPGA. For this, an optimized architecture is proposed as part of an hybrid software and hardware co-operating system. For its evaluation, the new designed PNG IP has been implemented on the ALTERA “Arria II GX EP2AGX125EF35” FPGA. The experimental results show a good match between the achieved compression ratio, the computational cost and the used hardware resources.

As the CCD industrial base declines, CMOS imagers are becoming the dominant visible image sensor for space-based applications. The generally lower dynamic range of CMOS imagers can be partly addressed by techniques and algorithms for High/Wide Dynamic Range CMOS (HDR/WDR CMOS). While algorithmic HDR imaging helps bridge the gap between CCDs and CMOS imagers, it is either computationally intensive or requires high bandwidth data transmission. Real-time algorithms help address computation and bandwidth issues for CMOS image sensor systems, but implementing real-time algorithms still requires significant computing power. The system must capture, analyze, and store every pixel according to algorithm constraints without dropping frames. This paper documents the implementation of a real-time multiple samples algorithm to extend dynamic range for a custom-built camera module with commercial CMOS image sensor. The multiple samples technique (in model form) allows arbitrarily large dynamic range with appropriately scaled computational demands. The algorithm is implemented by FPGA to relieve burdens on host processor. These real-time algorithms support applications such as motion tracking in day/night vision environments and help close the gap between CCD and CMOS imagers as well as sensor data generation rates and available bandwidth for satellite imaging.

The semivariogram is a statistical measure of the spatial distribution of data based on Markov Random Fields, similar to other measures of texture like spatial covariance and co-occurrence matrices. Semivariogram analysis is a computationally intensive algorithm that has typically seen applications in the geosciences and remote sensing areas. Recently applications in the area of medical imaging are being investigated, resulting in the need for efficient real time implementation. The semivariogram is a plot of semivariances for different lag distances between pixels. Semi-variance is defined as the half of the expected squared differences of pixel values between any two data locations for a given lag distance. Consequently, the base algorithm complexity for n pixels isO(n^3). This paper presents a technique for the fast computation of the semivariogram using a custom FPGA architecture. The algorithm is benchmarked using VHDL on a Xilinx Spartan-3E FPGA for varying image sizes. Medical image data from MRI scans is utilized for the experiments. Computational speedup is measured with respect to Matlab implementation on personal computer with an Intel i7 multi-core processor. Preliminary results indicate that significant advantage in speed is attained by the architecture making the algorithm viable for implementation in medical devices.

Conversion of available 2D data for release in 3D content is a hot topic for providers and for success of the 3D video applications, in general. It naturally completely relies on virtual view synthesis of a second view given by original 2D video. Disparity map estimation is a central task in 3D content generation but still follows a very difficult problem for rendering novel images precisely. There exist different approaches in DM reconstruction, among them manually and semiautomatic methods that can produce high quality DMs but they demonstrate hard time consuming and are computationally expensive. We propose several hardware implementations of two designed frameworks for an automatic 3D color video generation based on 2D real video sequence. The analyzed frameworks include together processing of neighboring frames using the following blocks: the RGB to CIELa*b* color space conversion, wavelet transform (WT) with edge detection using HF wavelet sub-bands (HF, LH and HH) or pyramidal scheme, color segmentation via k-means on a*b* color plane, up-sampling in wavelet case, disparity map (DM) estimation using stereo matching between left and right images (or neighboring frames in a video), adaptive post-filtering, and finally, the anaglyph 3D scene generation. The designed techniques has been implemented on DSP TMS320DM648, Matlab’s Simulink module over a PC with Windows 7, and using graphic card (NVIDIA Quadro K2000) demonstrating that the proposed approach can be applied in real-time processing mode. The time values needed, mean SSIM and BP values for different hardware implementations (GPU, Single CPU, Multicore CPU and DSP) are exposed in this paper. The data for time consumption and quality metrics in case of stereo pair Tsukuba (384x288), set of images from Middlebury set (27 stereo pairs, size of 1390 x 1110) and complete Tsukuba video sequence (1800 frames) are presented and discussed in this work.

The SIFT algorithm is one of the most popular feature extraction methods and therefore widely used in all sort of video analysis tasks like instance search and duplicate/ near-duplicate detection. We present an efficient GPU implementation of the SIFT descriptor extraction algorithm using CUDA. The major steps of the algorithm are presented and for each step we describe how to efficiently parallelize it massively, how to take advantage of the unique capabilities of the GPU like shared memory / texture memory and how to avoid or minimize common GPU performance pitfalls. We compare the GPU implementation with the reference CPU implementation in terms of runtime and quality and achieve a speedup factor of approximately 3 - 5 for SD and 5 - 6 for Full HD video with respect to a multi-threaded CPU implementation, allowing us to run the SIFT descriptor extraction algorithm in real-time on SD video. Furthermore, quality tests show that the GPU implementation gives the same quality as the reference CPU implementation from the HessSIFT library.We further describe the benefits of GPU-accelerated SIFT descriptor calculation for video analysis applications such as near-duplicate video detection.

Real-time object tracking for moving target auto-focus in digital camera Focusing at a moving object accurately is difficult and important to take photo of the target successfully in digital camera. Because the object often moves randomly and changes its shape frequently, position and distance of the target should be estimated at real-time so as to focus at the object precisely. We propose a new method of real time object tracking to do auto-focus for moving target in digital camera. Video stream in the camera is used for the moving target tracking. Particle filter is used to deal with problem of the target object’s random movement and shape change. Color and edge features are used as measurement of the object’s states. Parallel processing algorithm is developed to realize real-time particle filter object tracking easily in hardware environment of digital camera. Movement prediction algorithm is also proposed to remove focus error caused by difference between tracking results and target object’s real position when the photo is taken. Simulation and experiment results in digital camera demonstrate effectiveness of the proposed method. We embedded the algorithm in digital camera. Position and distance of the moving target object is obtained accurately by object tracking from the video stream. SIMD processor is applied to enforce parallel real-time processing. Processing time less than 60ms for each frame is obtained in the digital camera with its CPU of only 162MHz.

For several years, face recognition has been a hot topic in the image processing field: this technique is applied in several domains such as CCTV, electronic devices delocking and so on. We studied the impact of position and scale variation on a wavelet-based face recognition method. The use of wavelet transform has a limited impact on the robustness to pose variation of PCA-based face recognition. We show, for a given database (Yale face database B), that position and scale can vary up to 10% of the ROI size without reducing the recognition rate. These results are obtained when using approximation coefficients of the input image after 3 levels of decomposition. The amount of learned database and processed data are divided by a factor 64. This result confirms the interest of wavelet transform for face recognition on ultra-embedded vision systems. It also allows us to consider the use of a low complexity face detection stage compliant with the limited processing power available on such systems. We already have a PC-based demo using a standard webcam and a homemade GUI, the next step is to implement such an application on an embedded processor platform such as an Android-based device.

This paper presents a novel computing technique and true data plane architecture for real time embedded multimedia computing. The technique involves what is called middle stratum operation (MSO) based computing and multimedia data-plane processor (MDP) based on true data-plane architecture. MDP provides very high acceleration of multimedia algorithms at ultra low power consumption. The real time embedded computing capabilities of MDP are demonstrated by accelerating 3D- depth map computing from stereo video images. The results are compared with a GPU based implementation which highlights MDP's high performance and ultra low power consumption.

A major issue in creating photo-mosaics resides in the appearance of a visible seam at the position where the images overlap. A possible solution to this issue consists of using a multi-band blending (MBB), which requires Laplacian Pyramid (LP) decomposition. The main disadvantage of the MBB is a high required precision of the filter coefficients and long processing time. This results in a reduced processing frame rate, which is unsuitable for real-time processing systems. In this work, we create a set of FIR filters consisting of several custom-designed filters, 5/3 and 9/7 wavelets, and already proposed Gaussian filters. We analyze their influence on image quality and hardware complexity, and compare the results based on blur, edge quality and image naturalness. We show that 5/3 wavelet filter pair results in the best image quality, especially when edge quality is concerned. We propose an FPGA system implementing real-time 2D non-separable systolic filtering scheme, processing approximately 95 fps for 1080p frame resolution. Additionally, the resulting image quality is independent of the system where the algorithm is being implemented on. The presented work allows further research in real-time image processing areas such as surveillance or object recognition and tracking, which are currently limited by either low image resolution or low frame rate.

Iris recognition is one of the best biometric-based authentication methods available today. In an iris recognition system human irises are extracted from a static image or video frame then segmented and unwrapped into a matrix template more suitable for logical analysis. The template is then compared to others in a stored database to determine a match. One of the most commonly used methods to unwrap an iris image into a matrix template is based on the Coordinate Rotation Digital Computer (CORDIC) technique. In this work, an unrolled pipelined and parallel architecture is designed to implement the CORDIC-based coordinate transformation module to increase the throughput of the system. A parallel XOR tree is designed to rapidly compute the hamming distance in the template matching module. The proposed design is targeted for a low-cost FPGA prototype board from Altera for a cost effective iris recognition system. The large databases that will be used to test the entire system contain pictures of human eyes captured in the near infrared spectrum. With an FPGA as the main integration platform our expectations are to improve system processing time and overall power consumption to levels more efficient than current iris recognition systems released today.

Recent developments in image sensors have made cameras in smartphones approaching the abilities of human vision in terms of resolution and sensitivities. However, the miniaturization of image sensors causes tiny pixel size which eventually produces severe noise. On the other side, the use of rolling shutter in these inexpensive devices further degrades final imaging qualities, such as artifacts caused by flickering AC lighting fixtures. In this paper, we present a real-time solution to automatically detect the flicker type and noise level, and alleviate both flicker and sensor noise adaptively. In addition, the computational cost of our algorithm is relatively low in order to reduce its power consumption when implemented on smart devices. To test our method, degraded videos are generated from a set of reference video clips. The sensor noise is simulated as an additive independent distributed Gaussian noise, while the flicker is approximated by sine wave horizontal stripes. Denoising methods are evaluated by the quality gain of the denoised video over the corresponding generated video. Our method achieves comparable PSNR gain and better SSIM gain, in comparison with BM3D. To demonstrate the low computational expense of our approach, we have implemented our method with OpenGL ES on a Lenovo K900 smartphone, which is equipped with the Intel Atom Z2580 processor. We are able to display a denoised and deflickered viewfinder directly on the phone’s screen. Our implementation runs at 30 fps (i.e., the camera’s maximum frame rate).

Observing that low-dimensional linear subspaces are able to effectively model image variation caused by illumination and pose change. On the one hand, most related tracking algorithms using a pre-trained view-based eigenspace representation, it is imperative to collect a large set of training images covering the range of possible appearance variation, and this representation, once learned, is not updated. On the other hand, the conventional subspace learning, in the sense of least-squares approximation, is susceptible to outlying measurements. To address these two important issues, we present a robust subspace learning method that incrementally learns a low-dimensional subspace representation, which can efficiently adapt online to changes in the appearance of the target. Tracking then is led by the state inference within the framework in which a particle filter is used for propagating sample distributions over the time. Experimental results on tracking visual objects in video sequences where the targets undergo large changes in pose, occlusion and illumination, demonstrate the effectiveness and robustness of our tracking algorithm. In this article, we also implement the algorithm on an FPGA and DSP so that the detection is performed in real time and robustly.

The proposed algorithm is fast thumbnail extraction algorithm for HEVC. The proposed method only reconstructs 4x4 boundary pixels which are needed for thumbnail image. The proposed method generates the thumbnail image without full decoding. The proposed algorithm can reduce the computational complexity by performing only calculation needed for thumbnail image extraction. The proposed algorithm can significantly reduce the computational complexity for intra prediction and inverse transform. Also, it does not perform de-blocking filter and sample adaptive offset (SAO). After partial decoding process, it could be concluded that the proposed method significantly reduces the decoding time. In addition, the visual quality of obtained thumbnail images was almost identical to thumbnail images extracted after full decoding process.

In this paper, we present a novel scheme that includes the analysis of the inverse problem using the Finite Element Method (FEM) in order to obtain the conductivity within a unit circle domain, following the estimated conductivity is introduced in the algorithm via a technique of the Pseudoanalytic Function Theory where the Taylor series in formal powers method is used, performing the solution in EIT for electric potential in the boundary. The convergence and stability of this ill posed problem is also analyzed in this work using proposed hybrid procedure based on regularization to increase the convergence performance. Numerous experimental results for estimated electric potential by means of resolving together inverse and direct EIT problems for different types of conductivity distributions are exposed in this study justifying good performance of novel method. These results are performed using the FEM and numerical approximation of the Taylor series in formal powers approach developed in the structural programming paradigm in C++. An analysis of the Taylor series in formal powers employing a parallel programming paradigm in C++ is included with the objective to modify the algorithm and to develop a possible real-time mode in computing the solution with proposed method. This is performed with help of hardware based on multi-core processors and GPU platform.

Nowadays vision systems are used with countless purposes. Moreover, the motion estimation is a discipline that allow to extract relevant information as pattern segmentation, 3D structure or tracking objects. However, the real-time requirements in most applications has limited its consolidation, considering the adoption of high performance systems to meet response times. With the emergence of so-called highly parallel devices known as accelerators this gap has narrowed. Two extreme endpoints in the spectrum of most common accelerators are Field Programmable Gate Array (FPGA) and Graphics Processing Systems (GPU), which usually offer higher performance rates than general propose processors. In previous work, we have proposed a GPU-based version of a neuromorphic motion estimation algorithm which achieve a ×32 speedup. This algorithm has been developed entirely in the CUDA’s programming model. Despite providing successful performance, its portability is limited to NVIDIA based systems.Moreover, the use of these kind of accelerators involves the efficient exploitation of any parallelism in the target application. This task is not easy because performance rates are affected by (1) the efficient use of the complex memory hierarchy, (2) homogeneous balance workload distributed between many-core’s multiprocessors and (3) fast technology evolution. Moreover, it also worth mentioning the debuggers immaturity which slows any migrating process. Under these premises several initiatives have emerged to facilitate the programming process in order to exploit these accelerators. In the field of GPUs, we consider among the most relevant initiatives in high-level programming the standards OpenACC and OpenMP (from 4.0 version with accelerator support). Both paradigms are based on programming models with directives where programmer specifies loops and regions to be offloaded to accelerator. OpenACC specification have been developed by well-known companies as Cray,NVIDIA, PGI and CAPS. Nowadays, CAPS and PGI commercial compilers support this standard. In this extended abstract, it’s evaluated GPU behavior with the well-known convolution using the OpenACC new paradigm. The speedups obtained in a modern NVIDIA GPU using OpenACC standard varying image and filtering size shows a performance higher of 100x for 1024x1024 frame resolutions. These encouraging results, open the chance to consider this new programming paradigm in real time image processing scenario.

When it comes to extracting meaningful information from a scene, vision algorithms have to cope with varying illumination conditions at both intra-frame and inter-frame levels. Generally speaking, all the reported techniques targeting High Dynamic Range (HDR) deal globally with the image content. There is no special consideration for specific regions in the process of adjusting the capture according to the illumination conditions. However, vision algorithms usually focus their attention in particular so-called Regions Of Interest (ROI). In such a case, the priority should be to adapt the capture for those ROIs while ensuring that new ones can still be detected in case they enter the scene. This paper describes the operation of a prototype smart imager capable of adjusting the photo-integration time of multiple ROIs concurrently, automatically and asynchronously with a single exposure period. Each pixel of this prototype incorporates two photo-diodes. One of them senses the pixel value itself whereas the other, in collaboration with its counterparts in a particular ROI, senses the mean illumination of that ROI. Additional circuitry interconnecting both photo-diodes enables the automatic asynchronous adjustment of the integration time for each ROI. The sensor can be reconfigured on-the-fly according to the requirements of a vision algorithm.

Edge detection is defined as the process of detecting and representing the presence of and locations of image signal discontinuities, which serves as the basic transformation of signals into symbols and it influences the performance of subsequent higher level pattern analysis downstream. In general, the edge detection operation has two main steps: filtering, and detection and localization. In the second step, often edge detectors have been designed to capture simple ideal step functions in image data, but real image signal discontinuities deviate from this ideal form. Another three types of deviations from the step function which relate to real distortions occurring in natural images are examined. These types are impulse, ramp, and sigmoid functions which respectively represent narrow line signals, simplified blur effects, and more accurate blur modeling. The performance analysis on experiments supports that the proposed edge detection algorithm with edge pattern analysis does lead to more effective edge detection and localization with improved accuracies. To expand the proposed algorithm into real-time applications, a parallel implementation on a graphics processing unit (GPU) is presented in this paper. For the various configurations in our test, the GPU implementation shows a scalable speedup as the resolution of an image increases.

Driven by improved bandwidth in both mobile and fixed networks, steadily reducing hardware costs and the development of new technologies, many existing and new classes of commercial and industrial video applications are now being upgraded or emerging. Towards optimal performances, the design or optimisation in each of these applications should be context aware and task oriented with the characteristics of the video stream (frame rate, spatial resolution, bandwidth etc.) chosen to match the use case requirements. This paper presents a novel, conceptually simple and easily implemented method of assessing video quality relative to its suitability for a particular task and dynamically adapting videos streams during transmission to ensure that the task can be successfully completed. Firstly we defined two principle classes of tasks: recognition tasks and event detection tasks. Each with a set of task-oriented attributes (minimum spatial resolution, minimum frame rate etc.). We also define a set of contextual attributes associated with each instance of a running application that include resource constraints imposed by the transmission system employed and the hardware platforms used as source and destination of the video stream. Empirical results are presented and analysed to demonstrate the advantages of the proposed schemes.

Since the delivery of version one of H.265, the Joint Collaborative Team on Video Coding have been working towards standardisation of a scalable extension (SHVC), a series of range extensions and new profiles. As these enhancements are added to the standard the range of potential applications and research opportunities will expand. For example the use of video is also growing rapidly in other sectors such as safety, security, defence and health with real-time high quality video transmission playing an important role in areas like critical infrastructure monitoring and disaster management, each of which may benefit from the application of enhanced HEVC/H.265 and SHVC capabilities. The majority of existing research into HEVC/H.265 transmission has focussed on the consumer domain with the lack of freely available tools widely cited as an obstacle to conducting this type of research. In this paper we present a toolset that facilitates the transmission and evaluation of HEVC/H.265 and SHVC encoded video on an open source emulator. Our toolset provides researchers with a modular, easy to use platform for evaluating video transmission and adaptation proposals on large-scale wired, wireless and hybrid architectures. The proposed toolset would significantly facilitate further research in delivering adaptive video streaming based on the latest video coding standard over wireless mesh and other ad hoc networks.

This paper proposes a new low-complexity dynamic resource allocation engine for cloud-based video transcoding services that are both scalable and capable of being delivered in real-time. Firstly, through extensive experimentation, we establish resource requirement benchmarks for a wide range of transcoding tasks. The set of tasks investigated covers the most widely used input formats (encoder type, resolution, amount of motion and frame rate) associated with mobile devices and the most popular output formats derived from a comprehensive set of use cases, e.g. a mobile news reporter directly transmitting to the TV audience, with minimal usage of resources both at the reporter’s end and of the cloud infrastructure. These benchmark results are then exploited to create a reference database with predefined combinations of inputs and outputs, which is a product of rigorous experiments in defining the conversion process at an optimized level. The proposed cloud-based video transcoding process is platform-independent and codec-agnostic; it is a function of the input and expected video output, dynamically determining the most efficient method for its transcoding based on the reference database

With the surge in Internet video traffic, real-time HTTP streaming of video has become increasingly popular. Especially, applications based on the MPEG Dynamic Adaptive Streaming over HTTP standard (DASH) are emerging for adaptive Internet steaming in response to the unstable network conditions. Integration of DASH streaming technique with the new video coding standard H.265/HEVC is a promising area of research in light of the new codec’s promise of substantially reducing the bandwidth requirement. The performance of such HEVC-DASH systems has been recently evaluated using objective metrics such as PSNR by the authors and a few other researchers. Such objective evaluation is mainly focused on the spatial fidelity of the pictures whilst the impact of temporal impairments incurred by the nature of reliable TCP communications is also noted. Meanwhile, subjective evaluation of the video quality in the HEVC-DASH streaming system is able to capture the perceived video quality of end users, and is a new area when compared with the counterpart subjective studies for current streaming systems based on H.264-DASH. Such subjective evaluation results will shed more light on the Quality of Experience (QoE) of users and overall performance of the system. Moreover, any correlation between the QoE results and objective performance metrics will help designers in optimizing system performance. This paper presents a subjective evaluation of the QoE of a HEVC-DASH system implemented in a hardware testbed. Previous studies in this area have focused on using the current H.264/AVC or SVC codecs and have hardly considered the effect of Wide Area Network (WAN) characteristics. Moreover, there is no established standard test procedure for the subjective evaluation of DASH adaptive streaming. In this paper, we define a test plan for HEVC demonstrate the bitrate switching operations in response to various network condition patterns. The testbed consists of real-world servers (web server and HEVC-DASH server), a WAN emulator and a real-world HEVC-DASH client. -DASH with a carefully justified data set taking into account longer video sequences that would be sufficient to We evaluate the QoE by investigating the perceived impact of various network conditions such as different packet loss rates and fluctuating bandwidth, and the perceived quality of using different DASH video stream segment sizes on a video streaming session and using different video content types. Furthermore, we demonstrate the temporal structure and impairments as identified by previous objective quality metrics and capture how they are perceived by the subjects. The Mean Opinion Score (MOS) is employed and a beyond MOS evaluation method is designed based on a questionnaire that gives more insight into the performance of the system and the expectation of the users. Finally, we explore the correlation between the MOS and the objective metrics and hence establish optimal HEVC-DASH operating conditions for different video streaming scenarios under various network conditions.

Whilst existing public cloud service providers are offering on-demand cloud services with affordable cost for mobile users, there are still a number of challenges to achieve high-quality mobile cloud based video applications. In addition to the bandwidth-constrained and error-prone mobile network connectivity, specific cloud networking architecture and service deployment implementation also play an important role in affecting the quality of service. In this paper, we deploy a real-time video streaming framework consisting of various existing Internet cloud networking architectures (Google Cloud, Microsoft Azure and Eucalyptus Cloud) and a cloudlet based on Ubuntu Enterprise Cloud over wireless mesh networking technology for mobile cloud users. To study the performance under different deployments using different public and private cloud service providers, we employ real-time video streaming over the HTTP/HTTPS standard, and conduct experimental evaluation and in-depth comparative analysis of the impact of different deployments on the quality of service for mobile video cloud users. Empirical results are presented and discussed to quantify and explain the different impacts resulted from various cloud deployments, video application and wireless/mobile network setting, and user mobility.

While Denoising is an extensively studied task in signal processing research, most denoising methods are designed and evaluated using readily processed image data, e. g. the well-known Kodak data set. The noise model is usually a additive white Gaussian noise (AWGN). This kind of test data does not correspond to nowadays real-world image data taken with a digital camera. Using such unrealistic data to test, optimize and compare denoising algorithms may lead to incorrect parameter tuning or suboptimal choices in research on real-time camera denoising algorithms. In this paper we derive a precise analysis of the noise characteristics for the different steps in the color processing. Based on real camera noise measurements and simulation of the processing steps, we obtain a good approximation for the noise variance. We further show how this approximation can be used in standard wavelet denoising methods. We improve the wavelet hard thresholding and bivariate thresholding based on our noise analysis results. Both the visual quality and objective quality metrics show the advantage of the proposed method. As the method is implemented using Look-Up-Tables that are calculated before the denoising step, our method can be implemented with very low computational complexity and can process HD video sequences real-time in an FPGA.

In the paper a novel filtering design based on the concept of exploration of the pixel neighborhood by digital paths is presented. The paths start from the boundary of a filtering window and reach its central element. The cost or penalty of a transition between adjacent pixels is defined in the hybrid spatial-color space. Thus, an optimal path of minimum total cost, leading from pixels of the window's boundary to its center can be calculated. The cost of an optimal path determines the degree of similarity of the central pixel to the samples from the local processing window. Thus, the filter output is calculated as a weighted mean of the central pixel and an estimate constructed using the information on the minimal costs assigned to each image pixel. The experiments performed on a set of standard color images, prove that the efficiency of the proposed algorithm is superior to the state-of-the-art denoising techniques in terms of the objective restoration quality measures, both for low and high noise contamination ratios. The proposed filter, due to its low computational complexity can be applied for real time image denoising and also for the enhancement of video streams.

This paper establishes a real–time auto-exposure method to guarantee that surveillance cameras in uncontrolled light conditions take advantage of their whole dynamic range while provide neither under nor overexposed images. Unlike the state–of–the–art auto-exposure methods, the proposed algorithm considers the shape of the histogram of the whole image. To control the camera settings, we defined a function (F ) that linearly depends on the exposure time and the electronic gain; and is inversely proportional to the square of the lens aperture diameter. The algorithm iteratively computes the value of F that maximize the width of the histogram while the overexposed and underexposed regions are not reached. The proposed algorithm was tested in a video surveillance camera placed at an outdoor parking lot surrounded by buildings and trees which produce moving shadows in the ground. The algorithm was running alternatively together with a representative auto-exposure algorithm in the recent literature. Besides the sunrises and the nightfalls, multiple weather conditions occurred which produced light changes in the scene. Summing up the results, we concluded that the proposed algorithm outperforms the selected alternative and provides a fast and stable auto-exposure method that maintains an optimal exposure for video surveillance applications.

In this paper a system for real-time recognition of objects in multidimensional video signals is proposed. Object recognition is done by pattern projection into the tensor subspaces obtained from the factorization of the signal tensors representing the input video signal. However, instead of taking only the intensity signal the novelty of this paper is to first build the extended structural tensor representation from the intensity signal that conveys information on signal intensities, as well as on higher-order statistics of the input signals. This way the higher-order input pattern tensors are built from the training samples. Then, the tensor subspaces are built based on the Higher-Order Singular Value Decomposition of the prototype pattern tensors. Finally, recognition relies on measurements of the distance of the test pattern projected into the tensor subspaces obtained from the training tensors. Due to high-dimensionality of the input data, tensor based methods require high memory and computational resources. However, recent achievements in the technology of the multi-core microprocessors and graphic cards allows real-time operation of the multidimensional methods as is shown and analyzed in this paper based on real examples with action recognition in the video signals.

Design and implementation of a near-infrared video projection system based on Davinci Digital Signal Processor (DSP) platform and Digital Micromirror Device (DMD) is introduced in this paper. And also describes the block diagram, hardware structure and video data processing algorithms of real-time video retrieval in detail. This system take two numbers (start frame, end frame) via serial port then it decode a part of pre-captured video indexed by start frame and end frame using DM6446EVM kit then upsampling this video by THS8200 daughter card then project it by D4100 kit with DMD in infrared region. We also show the incompatibility between the DM6446EVM kit and the D4100 kit and how we develop a solution for that incompatibility.

A successful night navigation system can help drivers of ground vehicles see clearly in a degraded visual environment due to darkness. Initially, an image was obtained by combining two spectral bands of images, thermal and visible, in an effort to enhance night vision imagery. However, the fused image gave an unnatural color appearance. Therefore, a color transfer based on look-up table (LUT) was used to replace the false color appearance with a colormap derived from a daytime reference image. We obtained the reference image from a public database using the GPS coordinates of the vehicle. We found that database images used for colorization had to be supplied using an automated procedure often on the order of seconds or minutes depending on how much the environment changes to allow the system to operate in real-time. The system has four inputs and two outputs. Two of the inputs are from cameras, thermal and visible, one input is from a GPS sensor, and the other input receives images from a public database. The two outputs consist of a connection to a public database that sends requests for images and the display of the final result of the night scene.

This paper proposes an efficient VLSI architecture for Hexagonal search algorithm for motion estimation at real-time scenario. The FPGA-based design proposes a novel address mechanism for accessing the search points in the reference frame in order to find the best match for the current macro-block whose motion vector has to be determined. Major contribution of the design lies in accommodating the entire search through a single processing element rather than systolic array based structures with multiple processing elements and on-chip buffers. Again, since the number of search points is not fixed and depends on the motion content of the frame, the VLSI architecture adopts a combinatorial logic based design to address the iterative search pattern and access the stored pixels required for motion estimation. This handling of data with simplified control also helps in lowering the critical path. The proposed memory addressing scheme with simplified memory organization helps gain a high frame-rate. A single port on-chip memory with simplified control unit further reduces the area. The architecture can process 118 frames (standard CIF) working at a frequency of 320 MHz when implemented on Virtex-5 technology and area of data-path with control unit is computed as 3.4K gate equivalent.

There are an estimated 3.5 trillion photographs in the world, of which 10% have been taken in the past 12 months. Facebook alone reports 6 billion photo uploads per month. Every minute, 72 hours of video are uploaded to YouTube. Cisco estimates that in the next few years, visual data (photos and video) will account for over 85% of total internet traffic. Yet, we currently lack effective computational methods for making sense of all this mass of visual data. Unlike easily indexed content, such as text, visual content is not routinely searched or mined; it's not even hyperlinked. Visual data is Internet's "digital dark matter" [Perona,2010] -- it's just sitting there! In this talk, I will first discuss some of the unique challenges that make Big Visual Data difficult compared to other types of content. In particular, I will argue that the central problem is the lack a good measure of similarity for visual data. I will then present some of our recent work that aims to address this challenge in the context of visual matching, image retrieval, visual data mining, and interactive visual data exploration.