This paper discusses the operation of wireless sensor networks using radio tomographic imaging (RTI) techniques. The paper explores the physical model for RTI measurements that incorporate the effects of sparse, moderate, and dense foliage. The non-line of sight (NLOS) paths, such as multiple weak reflections from foliage and terrain, will result in lower network signal strengths in the presence of an obstruction. The NLOS effects are characterized in terms of terrain and foliage characteristics. The physical model includes the number and strength of NLOS paths, their coherence with the LOS path, and their mitigation by directional antennas on the sensors.

To make a plan to fix walls’ wear inside a coke oven at about 1,200 degree centigrade, we have developed a measurement method to measure both radiation from hot wall bricks and shapes of the whole surface. For a compact design, one unit of the method include only one area camera and a specially designed optical filter divide field of view the camera into two regions that are for radiation and shapes respectively.

Non-uniformity of mid-wave infrared focal plane arrays (FPAs) is a critical factor affecting the detection range of infrared imaging systems, especially in detecting small and dim objects. Moreover, false alarm probability of the system gets higher as the non-uniformity of the FPA increases. Therefore, to improve the performance of infrared imaging systems, non-uniformity of the array should be corrected. Conventionally, to achieve this goal, either scene or calibration based non-uniformity correction (NUC) methodologies have been used. To achieve reasonable performance, scene based NUC techniques may require an impractical amount of time considering the operational duration of high speed platforms. On the other hand, calibration based NUC performance degrades as scene temperature observed by infrared imaging systems of high speed platforms varies. The method presented in this work relies on multiple NUC tables to compensate for the temperature variations in the scene. To compare this method with conventional methods, standard deviation of the NUC images and target detection probability were used as metrics. NUC images were obtained by capturing images with an infrared imaging module. To obtain images with target, either synthetic or pinhole target images were added to the non-uniformity corrected background images. According to these metrics, we found that multiple point NUC correction method is superior to the conventional calibration based method using single NUC table.

Synthetic aperture radar (SAR) is a microwave imaging equipment based on the principle of synthetic aperture, which has all kinds of characteristics such as all-time, all-weather, high resolution and wide breadth. It also has high research value and applied foreground in the area of military and civilian. In particular, worldwide, a great deal of researches on SAR target classification and identification based Deep Learning are ongoing, and the obtained results are highly effective. However, it is well known that Deep Learning requires a large amount of data, and it is costly and inaccessible to acquire SAR samples through field experiment, so image simulation research for expanding SAR dataset is essential. In this paper, we concentrated on generating highly realistic SAR simulated images for several equipment models using Generative Adversarial Network (GAN) without construction of terrain scene model and RCS material mapping. Then we tested the SAR simulated images on a specialized SAR classification model pretrained on MSTAR dataset. The results showed that simulated targets could be identified and classified accurately, demonstrating the high similarity of SAR simulated images with real samples. Our work could provide a greater variety of available SAR image for target classification and identification study.

The challenges of night target tracking come from insufficient light, appearance change, motion blur, etc. Since infrared and visible video data could utilize the complementary information sufficiently and efficiently, we explore a novel framework by combining Correlation Filter based visible tracking and MCMC based infrared tracking. We calculate accuracy scale from corresponding infrared video, also aim to generalize the best location within the co-operative fusion framework. Meanwhile, we built an accurately labeled infrared and visible night target tracking dataset for experiments. The result shows our proposed approach can improve the state-of-the-art visible trackers, and significant improve re-tracking when tracking drift.

The Robust Reading research area deals with the detection and recognition of textual information in scene images. In particular, the natural scene text detection and recognition techniques have gained much attention from the computer vision community due to their contribution to multiple applications. Common text detection and recognition methods are often affected by environment aspects, image acquisition problems, and the text content. In this work, a method for text detection and recognition in natural scenes is proposed. The method consists of three stages: 1) the phase-based segmentation, obtained by applying the MSER algorithm to the local phase image; 2) the text localization, where segmented regions are classified and grouped as text and non-text components; and, 3) the word recognition, where characters are recognized utilizing Histograms of Phase Congruency. Experimental results are presented using a known dataset and evaluated under precision and recall measures.

The study of diatoms has become relevant because it is one of the algal groups with more applications at present. In the present work, descriptors are obtained in the plane of frequencies through the use of the analytical Fourier Mellin transform, the binary masks of concentric rings and the fractional Fourier transform to carry out the automatic identification of images of 46 diatom species with different morphology. This methodology has as main characteristics to be robust before changes of scale, rotations, translations, and lighting.

A novel method is proposed in this paper to accurately reconstruct the three-dimensional scenes by using a passive single-shot capture with a lenslet light field camera. First, the light field data is used to refocus and shift viewpoints. Next, initial disparity map is estimated by building the cost volume using multi-view images. Through multi-view images, correspondence depth cues are also obtained by performing correspondence measure. Then, the focal stack and initial disparity map are used to compute defocus depth cues by focus measure based on gray variance. The initial disparity map is used to control the measure area. Finally, MRF global energy function is built to integrate both defocus and correspondence depth cues, obtaining accurate depth map. Using this accurate depth map and all-in-focus image, the 3D structure in real world are accurately reconstructed.

In case of interactive light-field visualization, the arbitrary rotation of content enables efficient orientation changes without the need for actual user movement. However, the preference of content orientation is a subjective matter, yet it is possible to be objectively managed and assessed as well. In this paper, we present a series of subjective tests we carried out on a real light-field display that addresses static content orientation preference. The state-of-the-art objective methodologies were used to evaluate the experimental setup and the content. We used the subjective results in order to develop our own objective metric for canonical orientation selection.

Steered Mixture-of-Experts (SMoE) is a novel framework for representing multidimensional image modalities. In this paper, we propose a coding methodology for SMoE models that is readily extendable to any dimensional SMoE model, thus representing any image modality of any dimension. We evaluate the coding performance of SMoE models of light field video, a 5D image modality, i.e. time, two angular, and two spatial dimensions. The coding consists of the exploiting the redundancy between the parameters of SMoE models, i.e. a set of multivariate Gaussian distributions. We compare the performance of three multi-view HEVC (MV-HEVC) configurations that differ in terms of random access. Each subaperture view from the light field video is interpreted as a single view in MV-HEVC. Experiments validate that excellent coding performance compared to MV-HEVC for low- to midrange bitrates in terms of PSNR and SSIM with bitrate savings up to 75%.

Pseudo-sequence based light field image compression methods use state-of-the-art video codecs like HEVC. Although video codecs have been designed to compress video sequences, they have good performance for light field images compression. Considering the light field images represented by their multi-views representation, a sequence with different image views is aligned following an appropriate strategy. However, there are some differences between video sequences and light field pseudo videos that can be utilized to improve the codec adaptation to the different view images compression. The pseudo-sequence images have spatial distances and predictable behaviors when compared to video sequences that have temporal distance and unpredictable behavior respectively. Considering these differences unnecessary operations can be avoided, while its performance can be improved. The video codecs compute the motion vectors using block matching motion estimation algorithms, which is computationally the most complex operation of any video codec. To reduce the motion estimation complexity many codecs use prediction models. In this paper, HEVC motion vectors search models are applied to the light field image views aligned as pseudo-sequences to analyze and find their repetitive and predictable patterns. These new patterns are then utilized for changing the HEVC motion estimation algorithm for codec complexity reduction using a state-of-the-art pseudo sequence compression method. Moreover, the use of parallel computing for the pseudo sequence compression method is addressed.

State-of-the-art light field (LF) image coding solutions rely, usually, in one of two LF data representation methods: lenslet or 4D LF. Recent JPEG Pleno guidelines were defined on the objective evaluation of LF image coding, relying on the 4D LF representation to facilitate codec assessment and benchmarking. However, any codec that does not support the 4D LF representation needs to undergo additional processing steps in order to generate a compatible reconstructed LF for comparison. The number of additional steps can be further extended if color format and bit depth conversions are also required. Since these conversions may have a significant impact when comparing different LF codecs, an exhaustive comparative analysis between these two LF representation methods is presented. This paper also includes the individual impact of each additional processing block on the objective quality.

In recent years, light field imaging has attracted the attention of the academic and industrial communities thanks to its enhanced rendering capabilities that allow to visualise contents in a more immersive and interactive way. However, those enhanced capabilities come at the cost of a considerable increase in content size when compared to traditional image and video applications. Thus, advanced compression schemes are needed to efficiently reduce the volume of data for storage and delivery of light field content. In this paper, we introduce a novel method for compression of light field images. The proposed solution uses graph signal processing to encode the similarities in the data. Then, a dictionary-based approach is used to efficiently transmit the dependencies among different views. Experimental results show that our method is a promising alternative to pseudo-temporal sequencing approaches that are currently very popular in compression of light field images. In particular, notable gains can be observed at low bitrates with respect to the state of the art.

In this paper, we present the results of an experiment carried out to investigate the perceptual differences between different angular and spatial resolution parametrization of a light-field video service. The study highlights how the two resolution values affect each other regarding perceived quality, and how the combined effects are detected, perceived and experienced by human observers. By achieving an understanding of the related visual phenomena, especially degradations that are unique for light-field visualization, the design and development of resource-efficient light-field video services and applications become more straightforward.

When humans observe stereoscopic images, visual discomfort may be experienced in the form of physiological symptoms such as eyestrain and headaches. These symptoms arise in cortical mechanisms related to early visual processing. In this work, we study the role of natural scene statistics (NSS) of the disparity maps of stereoscopic images and their relationship to 3D visual discomfort. In particular, we focus on the bivariate NSS models. We also build a new prediction model that combines information from binocular vision and the NSS models of disparity maps to accurately predict 3D visual discomfort, and we demonstrate that our model outperforms other existing predictors.

Point Clouds have been gaining importance as a solution to the problem of efficient representation of 3D geometric and visual information. They are represented for large amounts of data and compression schemes are important for their manipulation transmission and storing.However, the selection of appropriate compression schemes requires effective quality evaluation. In this work Point Clouds subjective quality evaluation using a surface representation is analyzed. Using a set of point clouds data objects encoded with the popular octree-pruning method with different qualities, a subjective evaluation was designed. The point cloud geometry was shown to subjects without any textural information and after Poisson surface reconstruction in a 2D motion sequence, initially rotating 360 degree~around its vertical axis and then 360 degree~around its horizontal axis. Subjective evaluations were performed in five different laboratories in different countries. Scores obtained from each test were correlated and no statistical differences were observed. Moreover, the results were correlated with state of the art point cloud objective metrics revealing poor correlation. Likewise, the correlation with a subjective test using a different representation of the point cloud data also showed poor correlation. These results suggest the need for more reliable objective quality metrics and further studies on adequate point cloud data representations.

Recent trends in multimedia technologies indicate a significant growth of interest for new imaging modalities that aim to provide immersive experiences by increasing the engagement of the user with the content. Among other solutions, point clouds denote a practical representation that allows users to visualize static or dynamic scenes in a more immersive way, mimicking the real-world perception. As in any type of imaging, the visual quality of a point cloud content is of crucial importance, as it directly affects the user experience. Although a significant amount of effort has been devoted recently from the scientific community, indicated also by the current activities of the standardisation bodies, subjective and objective quality assessment of point cloud imaging remains an open problem. In this paper, we report and analyse the results of an extensive benchmarking of the state-of-the-art objective metrics using subjective scores as the ground truth. Our experiments have been conducted considering realistic types of distortions applied on an extended data set of point clouds with diverse characteristics. Thus, our results allow the interpretation of the limitations and suitability of the current metrics in typical use cases in which point cloud are employed, based on their performance after a rigorous analysis. The authors aim to release this database and make it publicly available.

Digital holography is a growing field that owes its success to the provided three-dimensional imaging representation. This is achieved by encoding the wave field transmitted or scattered by an object in the form of an interference pattern with a reference beam. While in conventional imaging systems it is usually impossible to recover the correct focused image from a defocused one, with digital holography the image can be numerically retrieved at any distance from the hologram. Digital holography also allows the reconstruction of multiple objects at different depths. In a previous study, the benchmark of the main available image coding standard solutions JPEG, JPEG-XT, JPEG~2000 and the HEVC intra mode were performed for digital holographic data represented on the object plane. The HEVC intra main coding profile outperforms the other standards while JPEG 2000 results in very similar compression performance. In the current work, a scheme based on the HEVC intra mode codec for holographic information compression on the object plane is proposed. In the base layer, a 2D version of the object (amplitude information on object plane) is coded with HEVC intra main coding profile. In a previous study was was observed that the phase information requires much higher bitrates than the amplitude information, as the standards codec methodologies are not adapted to code this information. In this paper we propose a model where the amplitude information is encoded with the HEVC intra mode codec, while the phase is represented by encoding the real information and the signal of the imaginary information. The real information is also encoded using the HEVC intra mode as it already revealed appropriate for compression of this type of information. The imaginary information signal is encoded with JBIG. The advantage of this scheme is that the amplitude information provides a direct 2D representation of the hologram while the phase information can be considered as a 3D enhancement layer. The results show that the proposed scheme outperforms the state of the art in holography compression, while allowing compatibility with the current standards and direct 2D visualization.

Full-reference (FR) image quality assessment (IQA) models assume a high quality reference against which to measure perceptual quality. However, FR IQA on a compressed but previously distorted "reference" may produce unpredictable results. Hence we propose 2stepQA, which integrates no-reference (NR) and FR measurements into the quality prediction process. A simple, efficient multiplication step fuses these into a single score. We deploy MS-SSIM as the FR and NIQE as the NR component, while it can also be generalized by combining other FR and NR models. A new database containing authentically distorted reference images is built for test. 2stepQA is shown to achieve standout performance compared to other IQA models.

This paper presents an efficient resolution adaptation framework for video compression. It dynamically applies spatial re-sampling to frames exhibiting relatively low spatial detail, trading off the relationship between spatial resolution and quantization. A learning-based Resolution Quantization Optimization (RQO) module determines the optimal spatial resolution based on video features. A modified CNN-based single image super resolution method is employed to reconstruct full spatial resolution content at the decoder. The proposed framework has been integrated with HEVC HM codec and has been shown to offer significant overall bit rate savings and lower encoding complexity (based on PSNR and VMAF quality metrics).

The video quality assessment (VQA) technology has attracted a lot of attention nowadays due to an increasing demand of video streaming services. The existing VQA algorithms are designed to predict video quality aiming at a better mean opinion score (MOS) of video calibrated by subjective experiments. However, such a design cannot predict the satisfaction user ratio (SUR) of an aggregated viewer group and provides little guidance to video coding parameter selection, e.g. the Quantization Parameter (QP) of a set of consecutive frames, in practical video streaming services. The just-noticeable-difference (JND) based VQA methodology has been proposed recently. It has been verified experimentally that the JND point is normally distributed. In this work, we attempt to explain this phenomenon based on a user model. In this model, we try to predict the capability of a user to discern the difference between the same video content but encoded by different QPs. If the QP difference is larger, the user can see the difference more easily. Besides the QP difference, this model will include other factors – video content (e.g. masking and visual saliency) and user vision capability. We will build a mathematical model based on the data collected in the VideoSet. Next, we will use this model to predict the JND distribution range. Finally, we will discuss applications of this model.

Tiles and slices provide different frame partitioning options with tiles offering better parallelization potential and slices aiding video transmission. Since slices in HEVC can be defined as a series of tiles, properly balancing the two is of importance. In this paper we study the problem of tile/slice partitioning with the goals of maximizing parallelism speedup, while minimizing size difference among slices. We develop algorithms that perform tile resizing and slice definition to optimize a composite weighted target function. Experiments with HM reveal that compared to alternatives focusing on either of the two optimization targets, the proposed algorithms achieve valid trade-offs.

Several compression method have been proposed for multi-view and free-viewpoint videos, but the quality assessment of DIBR-synthesized view is very challenging owing to its new types of distortions induced by inaccurate depth map which the conventional 2D quality metrics may fail to assess. In this paper, we test the performance of existing objective metrics on free-viewpoint video with different depth coding algorithms. Results show that all the existing objective metrics perform not well on this database including the full-reference and the no-reference. There is certainly room for further improvement for the algorithms.

This paper starts by describing a set of subjective assessment experiments (using the Oculus Rift HMD) conducted to evaluate the perceptual impact, on quality, of the temporal and spatial omnidireccional video resolutions and HEVC compression parameters. The subjective assessment results are then used to show the efficiency of omnidireccional video specific metrics (S-PSNR, WS-PSNR, and VP-PSNR) as well their advantages and disadvantages for different compression levels and spatial resolutions. Finally, a new quality metric is proposed, that relies on intrinsic video features, such as temporal and spatial activities. As it will be shown, this metric allows to predict, with a reasonable accuracy, the impact on quality of the selected video temporal and spatial resolution, as well as compression factor.

We compare coding performance of state-of-the-art video codecs, such as H.264/AVC, HEVC, VP9 and AV1, on long full-title video sequences from Netflix's vast catalog. Key elements in this study are the use of shot-based encoding, where quality and resolution is allowed to vary on each shot, while utilizing the VMAF full-reference video quality metric to assess perceptual quality. Using the dynamic optimizer framework, we achieve optimality across video shots against the VMAF metric. We further choose near-exhaustive settings in each encoder engine, which allows us to push coding performance to the limits permitted by these video codec implementations.

HTTP-based video streaming techniques have now been widely deployed to deliver video streams over communication networks. With these techniques, a video player can dynamically select a video stream from a set of pre-encoded representations of the video source based on its available bandwidth and viewport size. The bitrates of the encoded representations thus determine the video quality presented to viewers and also the averaged streaming bitrate which is highly related to streaming cost for massive video streaming platforms. Our work minimizes the average streaming bitrate on a per-chunk basis by modeling the probability that a player observes a particular representation. Since popularity of videos is regional, this paper exploits a further optimization that uses regional statistics of client bandwidth and viewport instead of the global statistics. Simulation results suggest that using regional statistics provides significant egress saving than using the global statistics.

Motion estimation is a key component of any modern video codec. Our understanding of motion and the estimation of motion from video has come a very long way since 2000. A quick look at the current evaluation of optic flow techniques first presented in 2016 by Baker, Scharstein,Lewis et al and the corresponding online league table http://vision.middlebury.edu/flow/ shows a comparison of 135 different algorithms. Block based algorithms come out at rank 41 at best, for frame interpolation accuracy. While this does not mean that better motion estimation techniques combined with rate control within a codec would have real impact, we believe its worth the experiment. This paper will therefore present the start of a series of experiments to test different motion estimation algorithms within existing codecs. One key challenge is how to easily combine the block based rate control algorithm with a different motion estimator. We propose to simply use these recent algorithms on macro-pixels and so avoid issues of pel-wise-dense flow encoding. Our contribution to the existing body of literature in motion estimation for encoding is to inform the ongoing debate of the impact of motion estimation performance on encoding performance, a debate ongoing since 1995.

The development of video quality metrics and perceptual video quality metrics has been a well established pursuit for more than 25 years. Most of the work in this area has been seen to be most relevant for improving the performance of visual compression algorithms. However, modeling the human perception of video with an algorithm of some sort is notoriously complicated. This is not only because the human visual system itself is complex, but also because of the variability of the local environment in which the content is experienced. As a result the perceptual coding of video remains challenging and no standards have incorporated perceptual video quality metrics within their specification. In this paper we present the use of video metrics at the system level of a video processing pipeline. We show that it is possible to combine the artefact detection and correction process by posing the problem as a classification exercise. We also present the use of video metrics as part of a classical testing pipeline for software infrastructure, but here it is sensitive to the perceived quality in picture degradation.

The joint video experts team (JVET), originally created in October 2015, is a partnership of the ITU-T video coding experts group (VCEG) and the ISO/IEC moving picture experts group (MPEG) toward the development of a new video coding standard. In April 2018 it evaluated responses to a joint call for proposals for video coding technology that has a compression capability substantially beyond that of the High Efficiency Video Coding (HEVC) standard. This paper will review the current state of work in JVET for development of a new international standard for video compression coding.

This paper compares the coding performance of several video codes: ITU|ISO/IEC standard HEVC, and the formative JVET codec, as well as the Alliance for Open Media codec, AV1. The comparison compares the core tools, as well the performance on standardized test data, without and with rate control. For rate control for HEVC, we use an open source model of HEVC called x265, while the formative JVET codec does not currently have a rate control.

360-degree video is emerging as a popular way of providing immersive viewing experience. In the existing workflow for 360-degree video compression, the input 360-degree video needs to be first projected onto a 2D plane using a selected projection format, and then the projected video will be compressed using a conventional video codec. Depending on the selected projection format, the samples on the projected 2D plane may correspond to different spherical sampling densities. If a constant quantization is applied across the entire projected video frame, this could result in non-uniform reconstructed qualities for different spherical regions. In this paper, an adaptive quantization method is proposed to enhance the coding efficiency of 360-degree video by providing a more uniform reconstruction quality distribution on the sphere. This is achieved by adaptively adjusting the quantization parameter of each region in the 2D projection plane to modulate the reconstructed distortion based on its spherical sampling density. The proposed method is implemented in the JVET 360-degree video coding reference software JEM-6.0-360Lib-3.0 and shows significant coding performance improvement for 360-degree video represented in various projection formats.

We explore the use of different partitioning structures for the design of next generation video codecs. The proposed methods are evaluated relative to the Quad-Tree Binary-Tree (QTBT) partitioning framework currently implemented in the Joint Exploratory Model (JEM) software and under study in the Joint Exploration Video Team (JVET), which is joint collaboration between MPEG and the ITU-T. In the paper, the performance of different partitioning tree types is measured for sequences including those used for the Call for Proposals issued by JVET and trends are analyzed. Objective performance is reported using the Bjøntegaard Delta (BD) bitrate, and visual observations are also provided. To show the efficacy of using different partitions, bitrate savings are computed using simulations and show significant improvement relative to state-of-the-art. It is asserted that the coding efficiency improvement is especially pronounced in sequences with occlusions or dynamic changing content.

HDR has been a hot topic in recent years, and recently there are technology breakthroughs in multiple areas including HDR video, HDR photography, and HDR display. However, if users want to show an HDR image or play an HDR video on their device, a system-level framework is needed to make it happen. This paper gives a system level introduction on end-to-end HDR ecosystem from capture to compression to display, which can benefit not only HDR related module designs in chipset, but also the standard committee for more efficient algorithm and metadata proposal.

High Dynamic Range and Wide Color Gamut (generally called HDR) video is one of the key application challenges in modern video coding. This paper builds on FastVDO's previous work in video coding of HDR data, adds elements from machine learning as appropriate, and applies it to the formative JVET coding, now in development in the ITU|ISO/IEC standardization committees.

This paper explores techniques from machine learning applicable to video coding. These techniques the use of trained neural networks for video prediction, filtering, resampling, and image restoration after decoding. These techniques can be applied generically to any modern video codec, and can be normative or non-normative. We will study the impact on HEVC, JVET, and potentially AV1 as opportunity permits.

Today’s hybrid video coding systems typically perform an intra-picture prediction whereby blocks of samples are predicted from previously decoded samples of the same picture. We propose new intra-picture prediction modes which employ a non-linear feature extraction step as a key property. Our design-method is based on optimizing weights of a neural network. The objective function used incorporates aspects of the whole hybrid video coding system. Furthermore, we describe methods to limit the complexity of the newly designed predictors. Using the reference software implementation of HEVC and JEM, we demonstrate that our proposed neural network-based predictors can significantly increase compression performance.

One fundamental component of video compression standards is Intra-Prediction. Intra-Prediction takes advantage of redundancy in the information of neighboring pixel values within video frames to predict blocks of pixels from their surrounding pixels and thus allowing to transmit the prediction errors instead of the pixel values themselves. The prediction errors are of smaller values than the pixels themselves, thus allowing to accomplish compression of the video stream. The most commonly used compression protocol today is H.264, which is in the process of migrating to H.265 (or HEVC – High Efficiency Video Coding). AV1 is another protocol from the Alliance of Open Media (AOMedia). Both standards take advantage of intra-frame pixel value dependencies to perform prediction at the encoder end and transfer only residual errors to the decoder. The standards use multiple “Modes”, which are various linear combinations of pixels for prediction of their neighbors within image Macro-Blocks (MBs). In this research, we have used Deep Neural Networks (DNN) to perform the predictions. Using twelve Fully Connected Networks, we managed to reduce Mean Square Error (MSE) of the predicted error by up to 3 times. This substantial improvement comes at the expense of more extensive computations. However, these extra computations can be significantly mitigated by the use of dedicated Graphical Processing Units (GPUs).

Recently, the advances in transform coding have contributed to significant bitrate saving for the next generation of video coding. In particular, the combination of different discrete trigonometric transforms (DTT’s) was adopted in the Joint Video Exploration Team (JVET) solution to efficiently model the residual signal statistics and to improve the overall coding efficiency. However, this combination of transforms necessitates an increase in the memory requirement as well as coding complexity, which could potentially limit their practical use. In this paper, we solve both memory and complexity issues by including up to two transforms from the DTT family, where the rest of the transforms are generated by simple mathematical operations, like sign changing and reverse ordering. The simulation results showed that this approach achieves competitive results with substantial simplification of the transform design.

In this paper, we summarized power and latency challenges of eye tracking systems for mobile XR glasses. Compared to conventional ET applications like psychology or neuroscience experiments, or user interface for people with disability, XR glasses require a lot lower power consumption considering overall XR system power budget, which is less than ~1W with battery power. At the same time, latency is also important since if ET latency is too large so that it is difficult to accommodate the ET and graphics rendering within the target XR’s motion-to-photon latency, then users feel motion sickness. Considering the challenging ET’s power and the latency requirements in XR, we will introduce several factors that could impact on power and latency in both algorithm and system-level in this paper. Also, we will share rough power and latency exploration results to see how challenge it is to meet such two requirements at a same time. In addition, we reviewed commercial eye tracker’s power and speed performances to see if they can provide enough performance for the XR applications. Lastly, we will introduce some promising academy researches using ET embedded image sensor to satisfy both power and latency challenges in XR.

Image demosaicing is a vital part of digital imaging, the performance of which often dictates the visual quality of processed images. Recent work has shown convolutional neural nets (CNN) to perform better than traditional hand crafted methods at this task. Inspired by such research, we examine the suitability and performance of modern network architecture paradigms including VGG, U-NET, RES-NET and Adversarial Networks and propose an architecture that out performs the state of the art.

Identification of 3D objects is proposed using correlation of 2D holograms which record the amplitude and phase/depth information of the 3D object. As a proof-of-principle, holograms of objects which have identical intensity features and different depth profiles are recorded and 2D correlation is applied to distinguish the objects. This method allows to distinguish and correlate the depth information without reconstructing the phase of the object.

Real-time video stitching, which is used to obtain a large field video by some small field cameras, has great significance in real life. The existing video mosaic method based on SIFT features and RANSAC algorithm takes too much time in the processing of the first frame image, and the transformation matrix will have large errors when the number of the feature points matched correctly is small. In this paper, a real-time video stitching method based on ORB features and support vector machine (SVM) using binocular cameras is proposed. Firstly, the distortion of the cameras is corrected. Secondly, the ORB features in the overlapped regions of the first two frame images are extracted. Each pair of the feature points matched is filtered through the pre-trained SVM model. The matching calculation is terminated after 4 pairs of feature points are obtained and the transformation matrix can be calculated. Finally, the video stitching result can be obtained by using the gradually fading out algorithm of image fusion. The experiments show that the real-time seamless wide-field video can be obtained, and the first frame processing time of this method is much shorter than the other methods available, the frame frequency is 30fps. Keywords: video stitching, ORB, SVM, image fusion

In this paper, a real-time perception system for autonomous car is presented. It is based on a highly parallel architecture using state of the art Field Programmable Gate Array (FPGA) to perform both low and intermediary levels image processing tasks at video frame rate (i.e. 30 frames / s). The hardware algorithm consists to perform noise removal and edge detection, followed by Hough transform task to extract the segments corresponding the lanes boundaries. The rich hardware resources which are available in nowadays FPGAs (e.g. large built-in distributed RAM memories, DSP blocks, and reconfigurable PLLs) yielded for a compact and low power consumption real-time vision system. Series of tests on different roads within Abu Dhabi city were successfully conducted for different scenarios such as continues lines, discontinues lines and slightly curved lines for which the car speed reached up to 122 km/h

The background of image blurred by smoke causes the degradation of image quality, similar to adding noise to the image. In this work, we propose a method based on the structural similarity index of image determined by the image complexity for qualitative and quantitative smoke detection, and develop a DSPs system of video smoke detection based on the DM6437 EVM made by Texas Instruments. Experimental results show that the values of the structural similarity index determined by complexity of image are in good agreement with the results of the obscuration coefficient measured from optical smoke density meter.

We propose a novel scene-based non-uniformity correction method to achieve better fixed-pattern noise reduction and eliminate ghosting artifacts. Our approach is based on robust parameter updates via inter-frame registration and spatio-temporally consistent correction coefficients. We utilized a GMM-based spatio-temporal update mask to selectively refine the estimations of correction coefficients. The results of our experiments on an extensive dataset consisting of both synthetically corrupted data and real infrared videos show that the proposed algorithm achieves superior performance in PSNR and roughness metrics with notably lower ghosting artifacts when compared to other state-of-the-art methods.

To ensure the quality of especially aesthetic products (artificial or natural), the surface texture needs to be monitored and visual anomalies detected. We present a texture model-based approach for novelty detection based on features encoded by a convolutional neuronal network (CNN). We use the CNN representation to model the specific characteristic of a digital reference texture using a one-class classifier. Finally, we evaluate our inspection approach using the application case of digital print inspection. Therefore, we use a special inspection unit based on a camera array and a flashing light illumination. To compare our results with the results of manual inspection by printing specialists, we integrated our inspection unit into an industrial single-pass printing system.

In this study, we analyze color characteristics of professional videos in comparison to casual videos in the viewpoint of their aesthetic quality. It is shown that the saturation and brightness components have larger differences between amateur and professional videos than the hue component. Then, we propose aesthetic color templates that are obtained from professional videos. We provide showcases where casual videos are made closer to professional videos in terms of aesthetic appearance based on the proposed templates. The results demonstrate that employing our color templates is beneficial to enhance video aesthetics.

Stress field analysis using photoelasticity uses complex optical arrangements and spatial algorithms to recover the stress information. Such algorithms fail in time-dependent scenarios as when analyzing video. In this paper, we present a dynamic identification of stress zones in photoelasticity videos. We consider that the pixels describe motion trajectories related to the spatiotemporal stress variations in the videos. These trajectories are characterized and clustered according to its temporal dynamics. We analyze videos of two stressed analytical models, one industrial light source, and a commercial camera sensor. The proposed method describes and predicts the temporal behavior of different stress concentration zones.

Images are often corrupted by noise; therefore, it is essential to know the performance capability of a pattern recognition algorithm for images affected by it. In this work, two methodologies are analyzed when images are affected by Gaussian and salt and pepper noise. The two methods use the nonlinear signature correlation; each signature is a one-dimensional vector that represents each image and is obtained using binary masks based on the fractional Fourier transform. Both methodologies are position and rotation invariant.

This paper presents the bitmap generation for a digital-micromirror device (DMD) used in parallel two-photon lithography, an innovative micro/nanomanufacturing technology that uses femtosecond laser and two-photon absorption to polymerize a two-dimensional pattern (projected by the DMD) inside a liquid resist. Instead of toolpaths as in conventional 3D printing, here, the binary bitmap images representing the slices of 3D structures are required. In each bitmap image, every point must be assigned to 0 (empty space) or 1 (in the 3D part). The 3D structures are sliced into vectorized slices, each containing one or multiple polygons. The slices with polygons are first rendered into binary bitmap images with just the polygons’ boundary points set to 1; all other bitmap points are set to 0. A method named “extrema removing” is developed and tested so that all the bitmap points representing the 3D parts are correctly filled with 1, even in the case of multiple overlapped polygons. This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. (Document # LLNL-ABS-746081-DRAFT) Funding was available from LDRD project #16-ERD-047.

We report an approach to enlarge the field of view (FOV) and resolution of the microscope by using the fourier ptychographic microscopy (FPM) method with a laser source and Spatial Light Modulator (SLM) to generate modulated sample illumination. A ring pattern sample illumination is generated by setting the deflection of micromirrors on the DMD, and converted to Multi-angular illumination through the relay illumination system. A series of intensity sample images can be obtained by changing the size of the ring pattern and then used to reconstruct high resolution image through the ring pattern phase retrieval algorithm

While ISO WG1 recently celebrated the 25th anniversary of its most successful standard, it seems to be more than surprising that up to now, no reference implementation of this standard exists. During an ongoing activity aiming at filling this gap, several observations have been made in how far the "living standard" deviates from the ISO documents. In particular, applying the official reference testing procedure of JPEG, available as ITU Recommendation T.83 or ISO/IEC 10918-2, turned out to be more a challenge than expected. This paper sheds some light on the JPEG ISO standard, and our findings during reference testing a legacy, 25 year old standard.

The Joint Photographic Experts Group (JPEG, formally known as ISO/IEC SC29 WG1) is currently in the process of standardizing JPEG XS, a new interoperable solution for low-latency, lightweight and visually lossless video compression. The subjective assessment of the impact of visually lossless video compression poses particular challenges. This paper describes the subjective quality evaluation conducted on the JPEG XS Core Coding System. In particular, it details the test procedures and compares the results obtained by the different evaluation laboratories involved in this standardization effort.

JPEG XS is a new standard for low-latency and low-complexity coding designed by the JPEG committee. Unlike former developments, optimal rate distortion performance is only a secondary goal; the focus of JPEG XS is to enable cost-efficient, easy to parallelize implementations suitable for FPGAs or GPUs. In this article, we discuss the quantization and entropy coding modes of JPEG XS and the most recent advances during its standardization.

How to organize new data sources, content tagging, and user interactions into a coherent whole for these new “images” to ensure broad interoperability? JPEG, one of the world’s most successful standards efforts, has taken up an effort on metadata to address these issues. This paper will describe the approach under development in JPEG, including an ongoing JPEG initiative regarding the standardization of a 360 image format.

JPEG Pleno provides a standard framework to facilitate the capture, representation and exchange of point cloud, light field, and holographic imaging modalities. Such imaging modalities should be understood as light representations all inspired by the plenoptic function, regardless of which modality was used to capture or create parts of the entire content. This mind set recognizes that conversions between different modalities are possible and often useful. It defines tools for improved compression while providing advanced functionalities at system level. It also aims to support data and metadata manipulation, editing, random access and interaction, protection of privacy and ownership rights as well as other security mechanisms. JPEG Pleno targets efficient coding formats that guarantees the highest quality content representation with reasonable resource requirements in terms of data rates, computational complexity and power consumption. The associated file format will include a signalling syntax of associated metadata for the capture, creation, calibration, processing, rendering, and editing of data as well as for user interactions with such data. In this paper, an overview is provided of the current status of the JPEG Pleno framework.

The problem of assessing quality of images is central in many imaging applications. During the last decades several objective metrics have been proposed to automatically predict quality of images by machines instead of requesting human subjects to score them, as the latter requires important resources and cannot be done in real time. Machine learning has been used in the past to train an algorithm by means of example to behave as a human subject when evaluating quality or specific distortions in an image. The recent progress and increased interest in deep learning has revived this approach and a number of the approaches have recently been proposed to predict quality of pictures with machine learning based on deep neural networks. An important issue in any machine learning is that of training. In this paper, we propose a new approach to train a deep neural network by relying on an automatic training procedure instead of relying on training that would require human subjects. We show that the proposed approach will have very competitive performance when compared to the best objective metrics proposed in the literature while requiring reasonable computational resources.

The problem at solving which the project is aimed consists in the development of methods of constructing a three-dimensional combined dense map are of the accessible environment and determining a position of a robot in a relative coordinate system based on a history of camera positions and the robot's motions, symbolic (semantic) tags.

A novel enhancement method called wavelet-subband booster, which can be used to redeem face image quality. The efficient brightness detector was used to classify the color face image into three classes of dark, normal, or bright. The RGB color channels of the face image were respectively transformed to the discrete wavelet domain. Each subband coefficients were then adjusted by multiplying the singular value matrices of these frequency subband coefficient matrices with their boosting coefficients. An image denoising model was next applied. The 2D inverse discrete wavelet transform was performed to obtain the boosted color face image without the lighting effect.

The increasing of robotics equipped with machine vision sensors applied to Precision Agriculture is demanding solutions for several problems. The robot navigates and acts over a rough surface, considering specific restrictions. The information to navigate between the crops is supplied by physical sensors and mainly by some Imaging detection system to the robot. The vision system for this kind of robots has many challenges, as changes in luminosity, uncontinuous crop row, processing capacity and time, as well as terrain conditions, among others. The aim of this research is to propose a method to develop a vision system for a tractor robot based on: PCA dimensionality reduction algorithm, the Nyquist-Shannon sampling theorem and a genetic algorithm for crop row detection.

In this presentation, we deal with the design of a composite correlation filter from noisy training images for reliable recognition and localization of geometrically distorted targets embedded into cluttered blurred and noisy scenes. We consider the nonoverlapping signal model for both the reference image and input scene. The impulse response of the obtained filter is a linear combination of generalized filters optimized with respect to different criteria. The performance of the proposed composite correlation filter is compared to that of common correlation filters in terms of discrimination capability and accuracy of target location when input scenes are degraded and targets are geometrically distorted.

In this presentation, we estimate the accuracy of 3D object reconstruction using depth filtering and data from RGB-D sensor. Depth filtering algorithms carry out inpainting and upsampling for defective depth maps from RGB-D sensor. In order to improve the accuracy of 3D object reconstruction, an efficient and fast method of depth filtering is designed. Various methods of depth filtering are tested and compared with respect to the reconstruction accuracy using real data. The presented results show an improvement in the accuracy of 3D object reconstruction using depth filtering from RGB-D sensor.

In this presentation, we reconstruct 3D object shape using multiple Kinect sensors. First, we capture a RGB-G data from Kinect sensors and estimate intrinsic parameters of each Kinect sensor. Second, pre-calibration procedure is utilized to provide an initial rough estimation of the sensor poses. Next, extrinsic parameters are estimated using an initial rigid transformation matrix in Iterative Closest Point (ICP) algorithm. Finally, a fusion of calibrated data from Kinect sensors is performed. We test a system with some Kinect V2 sensors and present Experimental reconstruction results using Kinect V2 sensors are presented and analyzed in terms of the reconstruction accuracy.

In this presentation, we propose an algorithm for detection of local features in depth maps. The local features can be utilized for determination of special points for Iterative Closest Point (ICP) algorithms. The proposed algorithm employs a novel approach based on estimation of local gradient by a 3D Sobel operator. Our algorithm finds the absolute value of the gradient magnitude at each point in a depth map. Computer simulation and experimental results obtained with the proposed algorithm in real-life scenes are presented and compared with those obtained with state-of-the-art algorithms in terms of detection efficiency, accuracy, and speed of processing. The results show an improvement in the accuracy of 3D object reconstruction using the proposed algorithm followed by ICP algorithms.

This presentation deals with restoration of an image corrupted by impulsive noise using a novel cascade switching algorithm. The algorithm processes iteratively an observed degraded image changing adaptively parameters of the switching filter. With the help of computer simulation we show that the proposed algorithm is able to effectively remove impulse noise from a highly contaminated image. The performance of the proposed algorithm is compared with that of common successful algorithms in terms of image restoration metrics.

The combination of a cheap single-point laser range finder (LRF) device and cameras has become increasingly useful in recent research and industrial applications. So we focus attention on the fusion of 1-D laser rangefinder and camera.We propose a robust extrinsic calibration algorithm that is implemented easily and has small calibration error. Due to the 1-D laser rangefinder only returns a data in one dimension direction, it is difficult to build geometric constraint equations like 2-D laser rangefinder.So we are no longer constrained to build constraint equations to finish calibration.Utilizing the calibration result, we complete the calibration of camera assembly error.

In the present work new methods and algorithms for selecting features using deep learning techniques based on autoencoders will be proposed to provide high informativeness with low within-class and high between-class variance. The performance of the proposed methods in real indoor environments is presented and discussed.

The application of hyperspectral laser speckle contrast imaging (LSCI) at different conditions of ergodicity of the scattering medium has been examined with a final aim of accurate quantitative assessment of blood flow in biological tissues. Hyperspectral LSCI data resolved in depth has been experimentally verified on phantoms. The dependence of speckle contrast to the velocity of flow in vessels embedded in the scattering medium, as well as on the depth of vessel position within the medium has been revealed. Finally, a simple approach for correction of the LSCI depth dependency allowing more accurate quantitative assessment of flow has been introduced.

In this paper we propose a novel method to synthetize stereoscopic images, represented as light fields, using a multi-focus stack. This method can be applied to real and synthetized images. Experimental results are presented in an autostereoscopic screen.

A new polarization-interference system of Stokes polarimetry is proposed. The technique of variations of the polarization states of the illuminating and reference laser beams is used. A new analytical parameter is introduced: the local visibility of the polarization-interference image of an optically anisotropic layer. Algorithms for the relationship between the visibility distributions of the polarization-interference images of biological layers and the distributions of linear and circular birefringence are found. A technique for reconstructing distributions of the phase anisotropy of polycrystalline layers of biological fluids of human organs has been developed. Experimental studies of distributions of local visibility of polarization-interference images of polycrystalline blood films of healthy donors and patients with prostate cancer The sensitivity, specificity, and balanced accuracy of the method of polarization-interference mapping of polycrystalline films of blood are determined. In the framework of the statistical analysis of maps of linear and circular birefringence of polycrystalline blood films, objective criteria for the diagnosis of cancer have been found and excellent balanced accuracy has been achieved.

A new model of the correlation description of the optically anisotropic component of biological tissues is proposed. To characterize the transformation of the fibrillar network of laser radiation, a new parameter is used-the complex "two-point" Stokes vector. The interconnections between the module and the phase of the parameters of the "two-point" Stokes vector with the distributions of the optical axes and birefringence of the fibrillar network of biological tissue are found. A scheme of generalized Stokes polarimetry of microscopic images of histological sections of fibrillar biological tissues is developed. A technique for direct polarization measurement of the distributions of the real and imaginary parts of the "two-point" Stokes vector is proposed. Maps of module and phase distributions of the "two-point" Stokes vector of histological sections of the myocardium and endometrium The sensitivity, specificity and balanced accuracy of the method of polarization-correlation mapping of fibrillar networks of biological crystals are determined. Within the framework of the statistical analysis of the module maps and the phase of the "two-point" Stokes vector, histological sections of the myocardium and endometrium, objective criteria for diagnosing endometrial cancer have been found and an excellent balanced accuracy of the cause of death.

The possibility of solving the inverse problem - extraction of information about linear and circular birefringence and dichroism of light scattering biological layers is considered. An analytical model of the optical anisotropy of depolarizing biological tissues is proposed. The Muller matrix is represented as a superposition of differential matrices of the first and second orders. Interrelations between the parameters of phase and amplitude anisotropy and elements of a first-order differential matrix are obtained. The algorithms for the experimental measurement of the coordinate distributions of the elements of the polarization component of the Muller matrix of depolarizing biological tissue are found. The symmetry and features of first-order differential matrices of fibrillar (muscle) and parenchymal (liver) depolarizing biological tissues are investigated. The interrelations between the statistical moments of the first and fourth orders and the features of the morphological polycrystalline structure of the biological tissues of various human organs are found. The ways of application of the differential Mueller-matrix mapping method in clinical diagnostics of the distribution of the phase and amplitude anisotropy distributions are proposed.

It is well known that accuracy and resolution of depth data decreases with increasing the distance from the sensor, affecting the performance of 3D scene reconstruction systems based on an ICP algorithm. In this paper, to improve the accuracy of the 3D map by aligning multiple cloud points we propose: first, to split the depth plane into sub-clouds with a similar resolution; then, in each sub-cloud to select a minimum number of keypoints for aligning them separately with an ICP algorithm; finally, to merge all clouds into a dense 3D map. Computer simulation results show the performance of the proposed algorithm of the 3D scene reconstruction using real indoor environment data.

In order to design a tracking algorithm with invariance to pose, occlusion, clutter, and illumination changes of a scene, non-overlapping signal models for input scenes as well as for objects of interest and Synthetic Discriminant Function (SDF) approach are exploited. A set of optimum correlation filters with respect to peak-to-output energy is derived for different target versions at each frame. A prediction method is utilized for locate a target patch in the coming frame. The algorithm performance is tested in terms of recognition and localization errors in real scenarios and compared with that of the state-of-the-art tracking algorithms.

The energy compaction property of Discrete Cosine Transform (DCT) is highly efficient for video/image compression. By exploiting the property, DCT has been used for the compression. Since that, several transforms have been proposed. Among them, Karhunen–Loève transform (KLT) has the best performance in the energy compaction. However, KLT requires extra information to send. To achieve the performance of KLT without extra information, we propose the machine learning network, TransNet. TransNet is trained to achieve the better energy compaction than DCT and maintain the image quality simultaneously. We developed new loss function related with energy compaction and image quality.

In this presentation, an algorithm using generative adversarial networks is developed for face recognition. The proposed method consists in the interaction of two neural networks. The first neural network (generative network) generates face patterns, and the second network (discriminative network) rejects false face patterns. A neural network of feed-forward type (single-layer or multilayer perceptron) is used as the generative network. A big database of normalized to brightness changes and standardized in scale artificial images is created for the training of neural networks. The performance of the algorithm is presented and discussed.

In this presentation a new algorithm for automatic selection of face features using deep learning techniques based on autoencoders in combination with customized loss functions to provide high informativeness with low within-class and high between-class variance is proposed. The multilayer networks of feed-forward type are used. The extracted features are used for face classification. The performance of the proposed system for processing, analyzing and classifying persons from face images is compared with that of state-of-art algorithms.

One of the key steps of the algorithm is the removal a part of the points and the searching a correspondence of clouds. In the developed algorithm, the points lying on the boundaries of the planes intersections, the so-called edges of objects, are selected from the clouds. Then the intersection points of the found edges are checked to belong the main vertices of the objects. After that, additional vertices are excluded from the edges and, if necessary, new ones are added. The described approach is performed for both point clouds. In the next step the vertices found in the previous step are taken from the first cloud, with all the edges connected with them. For each such group it is necessary to find the corresponding group from the second cloud. This approach significantly reduces the number of points participating the deciding of the ICP variational subproblem.

Super resolution (SR) microscopy is a powerful technique which enhances the resolution of optical microscopes beyond the diffraction limit. Recent SR techniques achieve the resolution of 100 nm. Theoretical resolution enhancement can be mathematically defined. However, the final resolution in the real image can be influenced by technical limitations. Evaluation of resolution in a real sample is essential to assess the performance of an SR technique. Several image based resolution limit evaluation methods exist but the determination of cut-off frequency is still a challenging task. Overview of conventional methods and comparison with our approach to the resolution assessment is presented.

The key point of the ICP algorithm is the search of either an orthogonal or affine transformations, best in the sense of a quadratic metric that combines two point clouds with a given correspondence between points (the variational subproblem of the ICP algorithm). The point-to-plane metric has been shown to perform better than the point-point metric in terms of accuracy and convergence rate. A closed-form solution to the point-to-plane case for orthogonal transformations is an open problem. In this presentation, we propose an approximation of a closed-form solution to the point-to-plane problem for orthogonal transformations.

The registration of two surfaces is finding a geometrical transformation of a template surface to a target surface. To choose the most suitable transformation from all possible warps, a registration algorithm must satisfies some constraints on the deformation. The used variational functional consists of the several terms. One of them is the functional of the ICP (Iterative Closest Point algorithm) variational subproblem for the point-to-point metric for affine transformations. In proposed presentation we use variational functional based on the point-to-plane metric for affine transformations. The proposed algorithm is robust relative to bad initialization and incomplete surfaces. With the help of computer simulation, the proposed method is compared with known algorithms for the searching of optimal geometrical transformation.

Images of outdoor scenes are often degraded by the particles and water droplets in the atmosphere. Some haze removal approaches estimate a dehazed image from an observed hazed scene by solving an objective function whose parameters are adapted to local statistics of the hazed image inside a moving window. In this presentation we propose an algorithm of combining local estimates of depth maps toward a global map by regularization the total variation for piecewise-constant functions. Computer simulation results are provided to illustrate the performance of the proposed algorithm for restoration of hazed images.

This paper deals with techniques for analysis of perceived color differences in images. The performance of the methods is compared based on the results from subjective studies. Possible utilization of publicly available datasets with associated subjective scores is discussed. Dedicated database of images with common color distortions and associated subjective scores is introduced. Performance evaluation and comparison of objective image color difference measures is presented using conventional correlation performance measures and robust receiver operating characteristic (ROC) based analyses.

One of the most known techniques for signal and image denoising is based on total variation regularization (TV regularization). Many properties of the TV regularization for 1D and 2D cases are well known. In this work, we deal with TV regularization in the 3D case for the anisotropic norm. The key feature of the proposed method is to decompose the large problem into a set of smaller and independent problems, which can be solved efficiently and exactly. These small problems are can be solved in parallel. Computer simulation results are provided to illustrate the performance of the proposed algorithm for restoration of degraded data.

Eye tracker is a tool which enables analyzing visual reactions. This particular study presents measurement and numerical analysis of time variability of the eye pupil geometry. The image of the eye pupil was processed with Tobi TX-300 device (camera with 250fps rates). Every single frame from a sequence was numerically analyzed. Coherence between right and left eyes fixational movements were determined. Eventually, the website was evaluated using eye tracker in terms of neuromarketing. Neuromarketing is getting more and more popularity due to its objectiveness. It shows eyes behavior which at this rate probably can’t be controlled.

The study presents the results and analysis of subjective measurements of Purkinje effect (the different spectral sensitivity of the rod and cone systems) by using self-written algorithm. The research was treated as psychophysical experiment. The absolute thresholds of eye spectral sensitivity was measured by two methods which are the method of limits and the method of constant stimuli. To evaluate the working of algorithm, tests were carried out of 34 group of healthy people, aged from 18 to 40 years. Everyone was examined in the same conditions: for light adaptation and after 20 minutes adaptation to dark.

Complex moments are usually used to characterize, evaluate and manipulate digital information. In addition, they can be used as feature in pattern recognition. In this paper, the classification of metal-mechanical parts based on the complex moments invariant to rotation and translation is presented, where the computation of the complex moments is carried out by two methods: the zero-order approximation of the complex moments and through an arrangement of polar pixels. In the zero-order approximation, the integrals describing the complex moments are replaced by summations and the image is normalized within a unit disk. On the other hand, the complex moments calculated by means of a polar pixel scheme have the purpose of integrating the kernel of the complex moments analytically and eliminate the geometry error inherent to the computation of the complex moments. The classification method is tested with metal-mechanical parts that have intrinsically small differences between them, such as millimeter screws. Experimental results from four standard screws of differing lengths are presented.

The study presents measurement and analysis of chromatic aberration, which can be defined as the refraction difference in the conditions of disabled accommodation. For measurements, a manual refractometer was used with a specially constructed illuminator containing an RGB diode. Subjective measurements were conducted for wavelengths in the visible range from 470 to 660nm. The aberration was calculated based on the differences between refraction of red and blue light. Differences in the size of aberrations between the subjects were analyzed. The change in the amount of chromatic aberration after the insertion of a soft contact lens was also taken into account.

Nowadays, breast cancer is a condition that mainly affects women. The causes of this are numerous; changes in lifestyle and the rapid growth of industries, among others. With technological advances, it is possible to study the metabolic activity, the vascularization and celular structure of breast tissue through images of the breasts, which can determine if there is any abnormality in the functioning of the breasts. A relatively new method in the detection of breast lesions is breast thermography, which uses infrared technology. Breast thermography uses ultrasensitive infrared cameras to produce high resolution images of these temperatures and vascular changes. In the present work we propose the development of computational methods that allow the analysis and digital processing of breast thermographic images for the detection of regions related to a probable lesion. An semi-automated segmentation algorithm is presented by means of polynomial curve fitting for detection of curve inframmary. By means of the extraction of features based on image moments, information is obtained that allows the determination of a thermal symmetry between the breasts. A public database which contains information on volunteers with diagnosis made by mammography and biopsy is used.

Proton therapy is an evolution of conventional radiotherapy techniques because protons interact with living matter in a different way than traditional radiation beams. The proton beam delivers a minimum percentage of its energy at the beginning of its journey and most of it in a very precise position, depending on the characteristics of the beam and the properties of the biological tissue. In this way, it is possible both to obtain greater accuracy in the treatment, compared to conventional radiotherapy, and to increase the energy released to the tumor, saving in a consistent way the healthy tissues and organs at risk that surround the tumor tissue (target tissue). he planning of protonterapic treatment to be conferred to the patient is carried out using Monte Carlo simulation methods to calculate the effects of the interaction of the proton beam with the tissues to be irradiated and their precise location, using diagnostic images (TAC, PET and RM). The optimization of treatment plans requires a deep physical knowledge of both the proton-biological interaction and the use of advanced numerical simulation techniques that can be improved, compared to the state of the art, through the use of machine learning and high performance scientific calculation. The objective of this project is to study and to develop a system for the planning of personalized proton therapy treatments, with particular attention to the sector of paediatric oncology, which maximizes the effects of treatment, preserving organs and healthy tissues.

MPEG Omnidirectional Media Format (OMAF) specifies both a viewport-dependent video profile and a presentation profile to enable immersive media applications in current network environments. A sub-picture-based approach for viewport-dependent streaming is one of the main approaches being explored in MPEG OMAF standardization. This paper presents a sub-picture-based omnidirectional video live streaming platform with novel technologies on both server and client sides to illustrate the benefits of such viewport-dependent omnidirectional video streaming approach. The novel technologies presented include: omnidirectional video acquisition, sub-picture partitioning, real-time GPU-accelerated HEVC encoding, and DASH-based live streaming. The presented platform supports virtual reality (VR) clients including both VR head-mounted displays (HMDs) and conventional 2D displays. Viewing orientation tracking and real-time viewport extraction are also supported in our platform. As with all live streaming platforms, one of the main goals of our platform is to minimize end-to-end system latency. In order to enhance the immersive experience, we propose a new metric called comparable-quality viewport switching latency. The comparable-quality viewport switching latency is defined as the amount of time it takes for the viewport video quality to improve to a level comparable to that prior to viewport switching. The platform was demonstrated in the Joint 3GPP and VRIF workshop on VR and the 2018 Mobile World Congress as one of the first OMAF-compliant viewport-dependent streaming solutions.

A volumetric multi-planar display can facilitate the visual search performance in medical image assessment. The work aims to explore the extent to which the visual search is effective in dependence on target discriminability and describe the scrolling strategy through image stacks when stimuli are presented on many display planes. As a result, the target discriminability affected significantly the correct response rate and time, as well as the search behaviour was consistent with the physical design of stimuli set. This visual search paradigm is suggested for implementation in studies of visual perception and search behaviour in three-dimensional displays.

The key of denoising algorithm is making recovery image reserve as much as possible edge details and more when eliminating noise. Noise and image details both are part of the high frequency components of image, which are hard to saved at the same time. If the selection of the criterion and treatment for noise and marginal are inappropriate , denoising will reduce the quality of the image and increase greatly the complexity of subsequent image processing. A kind of soft threshold denoising algorithm is proposed by introducing such quantum computation methods as the quantum bit, superposition and collapse, etc. According to the edge feature in the local region of images, the algorithm can generate a template adaptively for the local region image filtering. Due to the algorithm is sensitive to the edge shapeness, a better compromise is obtained between the noise suppression and the edge preserving.

Coded aperture compressive X-ray tomography is an approach to reduce radiation without degradation of image reconstruction. This paper addresses the code optimization problem for compressive X-ray tomography based on the redundancy of sensing matrix. The computation time of the proposed optimization is much less than that of the gradient descent approach. The analysis of singular value decomposition (SVD) and the simulations of test image "phantom" show that this optimization is useful.

A 3D convolutional neural network based non-iterative compressive sensing reconstruction network is demonstrated with 3D computational imaging spectrometry data cube reconstruction. Due to non-iterative nature, this network can efficiently produces reconstruction results after the network is well trained. This network achieves preferable performance over state-of-art iterative computational imaging spectroscopy (CIS) algorithms, in terms of overall PSNR and SSIM when quantitatively compared with original data, and single point spectrum. The spectral reconstruction performance at a high compression rate (16:1), assessed by single point spectrum at sampling points, provides accurate information for object identification and thus guarantees potential applications of CIS.

A model for the transformation of laser radiation parameters by planar networks of biochemical crystals of different symmetries is developed. The algorithms for interrelating the elements of the Mueller matrix and the parameters of phase anisotropy are found. A technique of polarization reconstruction of linear and circular birefringence of polycrystalline films of liquor is proposed. The scheme of the laser polarimeter for the experimental realization of the inverse problem is described: extraction of data on the phase anisotropy of planar networks of crystals of a liquor film. he temporal transformation of distributions of linear and circular birefringence of polycrystalline films of the cerebrospinal fluid in the postmortem period is studied. The regularities of the change in the magnitude of the statistical moments of the first-fourth order, which characterize the distribution of phase anisotropy parameters, are determined. An algorithm for determining the prescription of the onset of death is developed with an accuracy of 30 minutes.

Magnetic Resonance Imaging (MRI) is a very common diagnostic methodology. Also, Diffusion tensor imaging (DTI) is increasingly used to define novel biomarkers for neurodegenerative diseases, such as Alzheimer’s disease (AD). Complex networks can be used to model the brain and outline the presence of changes in local and global connectivity, thus revealing disease patterns. In this work, we explore the use of multiplex network to characterize both inter- and intra-subject connectivity in order to accurately detect the presence of AD. This approach describes the brain in terms of typical network measures and then makes use of this measures to learn machine learning models. In particular, we use here a sample of 200 subjects from the Alzheimer’s Disease Neuroimaging Initiative. State-of-the-art classification models such as random forests, support vector machines and neural networks are used to evaluate the emergence of distinct patterns between healthy controls and AD patients. Finally, we investigate the application of multiplex network models in subjects affected by mild cognitive impairment. The method is general and could be used for the study of other disease affecting brain connectivity.

This project presents the second stage results of an algorithm processing images with the the main objective of identify and differentiate between demyelination and ischemic brain diseases through the automatic detection, classification and identification of their features found in the magnetic resonance images. The sequences of images used were T1, T2, and FLAIR and the dataset used was of the 80 and 90 patients with and without these or other pathologies, respectively. The algorithm im these stage use Discrete Wavelet Transform (DWT), principal component analysis (PCA) and a kernel support vector machine (SVM).