A new approaches and computer codes (A&CC) for automatic analysis of images are offered. The A&CC are based on presentation of object image as a collection of pixels of various colours and consecutive painting of distinguished itself parts of the image in unusual manner. The A&CC have technical objectives centred on such direction as: 1) image processing (filtration, elimination of noise, segmentation), 2) image feature extraction, 3) image analysis, 4) recognition of image and object and some others. Additional possibilities of the A&CC dealing with usage of artificial neural networks technologies. The A&CC allows to obtain various geometrical and statistical parameters of object image and object. Among them: coordinates and maximum values of cross sizes of every individual part of object image, its square and perimeter, histogram of individual parts with respect to size as well as with respect to type, to form, etc. The opportunities of the A&CC are tested at image analysis of model fires and plumes of the sprayed fluid, ensembles of particles, at a decoding of interferometric images, for elimination of a noise of the images, for filtration of the image, at detection of objects.

This paper demonstrates, for the first time, TOMBO color imaging system that employs color-splitting filters on each lens. A red, green, or blue color filter is allocated to each microlens instead of each pixel in conventional single-sensor color imaging system. Thus the microlens array here also makes up a color filter array of Bayer geometry. For the imaging device, a CMOS image sensor with 1040 x 960 pixels whose size is 6.25 um square was used. 8 x 8 microlens array with 750 um pitch was employed as a taking lens. Excellent color images were obtained by rearrangement, interpolation, and postdigital processing.

Compound eyes are a highly successful natural solution to the issue of wide field of view and high update rate for vision systems. Applications for an electronic implementation of a compound eye sensor include high-speed object tracking and depth perception. In this paper we demonstrate the construction and operation of a prototype compound eye sensor which currently consists of up to 20 eyelets, each of which forms an image of approximately 150 pixels in diameter on a single CMOS image sensor. Post-fabrication calibration of such a sensor is discussed in detail with reference to experimental measurements of accuracy and repeatability.

An artificial apposition compound eye manufactured by micro-optics technology is demonstrated. The overall thickness of the imaging system is only 320 μm, the field of view is 21° on diagonal, the F/# is 2.6. The monolithic device consists of a microlens array on a thin silica-substrate with a pinhole array in a metal layer on the backside. The image formation can be explained by the moiré-effect or static sampling. The master structures for the microlens arrays are manufactured by lithographic patterning of photo-resist and a subsequent reflow process. These master structures are replicated by moulding into UV-curing polymer. The pitch of the pinholes differs from the lens array pitch to enable an individual viewing angle for each channel. The required precision is guaranteed by using a lithographic process also for the assembly. Thus, problems with accuracy of other attempts to develop similar systems using discrete components have been overcome. Imaging systems with different sizes of pinholes, numbers of channels and separation of the viewing direction of the channels are realized and tested. A method to generate nontransparent walls between the optical channels for prevention of crosstalk is proposed. Theoretical limitations of resolution and sensitivity are discussed. Imaged test patterns are presented and measurements of the angular sensitivity function are compared to calculations using commercial raytracing software. The resolution achievable with the fabricated artificial compound eye is analyzed.

A novel surface plasmon resonance (SPR) sensor system using CMOS image sensor array is proposed in this paper. Recently, a simple SPR system was proposed by the author, which achieved high resolution and fast response time using a bi-cell photo-detector. However it requires mechanical adjustment process to balance two signals of the bi-cell before measurement. It requires not only additional time but also additional mechanical control unit, which is a source of the noise. It also suffers from the small linear range. The proposed method chooses a pixel as the center from many pixels, which gives the most balance of bi-cell signal. Therefore no mechanical adjustment is required. The method also overcomes the small linear range problem by switching the center adaptively during the test. Furthermore, it has several advantages of CMOS image sensor such as low cost, low power, and on-chip functionality, which makes the proposed SPR sensor system be a good candidate for field applications. A prototype CMOS image sensor chip with 12bits analog to digital converter is designed and fabricated with 0.5um AMI CMOS technology.

A CMOS focal-plane-array is designed for the high-throughput analysis of enzymatic reaction in on-chip spectrophotometer system. One of potential applications of the presented prototype system is to perform enzymatic analysis of biocompounds contained in blood. This function normally requires an expensive diode-array spectrophotometer, but it is possible to perform high throughput analysis with low budget if the spectrophotometer system is scaled down to a chip. The CMOS active pixel sensor array can cover a layer of polydimethylsiloxane (PDMS) forming the microfluidic channels and the substrate solution for enzymatic reaction can be injected into the channels by capillary force. Under room light, the underneath CMOS active pixel sensor with 40 x 40 pixels detect the gray levels of the fluid’s color. Inside the image sensor chip (size: 3mm x 3mm), the pixels of the same column share the same sample and hold circuits. The analog signals from 40 columns are multiplexed into one input feeding an on-chip 8 bits dual-slope analog to digital converter. The color change can be displayed on the external monitor by using a data acquisition card and personal computer.

We are exploring the application of pulse-frequency-modulation (PFM) photosensor to retinal prosthesis for the blind because behavior of PFM photosensors is similar to retinal ganglion cells, from which visual data are transmitted from the retina toward the brain. We have developed retinal-prosthesis vision chips that reshape the output pulses of the PFM photosensor to biphasic current pulses suitable for electric stimulation of retinal cells. In this paper, we introduce image-processing functions to the pixel circuits. We have designed a 16x16-pixel retinal-prosthesis vision chip with several kinds of in-pixel digital image processing such as edge enhancement, edge detection, and low-pass filtering. This chip is a prototype demonstrator of the retinal prosthesis vision chip applicable to in-vitro experiments. By utilizing the feature of PFM photosensor, we propose a new scheme to implement the above image processing in a frequency domain by digital circuitry. Intensity of incident light is converted to a 1-bit data stream by a PFM photosensor, and then image processing is executed by a 1-bit image processor based on joint and annihilation of pulses. The retinal prosthesis vision chip is composed of four blocks: a pixels array block, a row-parallel stimulation current amplifiers array block, a decoder block, and a base current generators block. All blocks except PFM photosensors and stimulation current amplifiers are embodied as digital circuitry. This fact contributes to robustness against noises and fluctuation of power lines. With our vision chip, we can control photosensitivity and intensity and durations of stimulus biphasic currents, which are necessary for retinal prosthesis vision chip. The designed dynamic range is more than 100 dB. The amplitude of the stimulus current is given by a base current, which is common for all pixels, multiplied by a value in an amplitude memory of pixel. Base currents of the negative and positive pulses are common for the all pixels, and they are set in a linear manner. Otherwise, the value in the amplitude memory of the pixel is presented in an exponential manner to cover the wide range. The stimulus currents are put out column by column by scanning. The pixel size is 240um x 240um. Each pixel has a bonding pad on which stimulus electrode is to be formed. We will show the experimental results of the test chip.

In 1999 Kodak introduced the first two-phase, front-illuminated, full-frame CCD in which the electrodes corresponding to one phase were composed of Indium Tin Oxide (ITO), a material more transmissive than poly silicon. In an effort to further increase the sensitivity of front-illuminated image sensors, Kodak has developed an all-ITO electrode CCD. A 1280 x 1024 sensor with 16-mm pixels has been manufactured and characterized. The imaging performance of this device, including its sensitivity, dark current, and charge transfer efficiency, is described. The noise characteristics of the ITO-gated output amplifier MOSFET are also discussed.

We describe a precise alignment method of attaching imagers to a prism to produce an ultra-high definition color camera system. We have already developed a prototype camera with 4-k scanning lines using this alignment method. To increase its spatial resolution, this camera has four 8-megapixel imagers (GGBR), which are attached to a prism with a half-pixel pitch offset so that their pixel arrangement is equivalent to that of a single-chip color-imaging sensor with a Bayer-pattern color filter. The precision of their positioning influences the resolution of the reproduced images. The small pixels in the latest imager make it more difficult to maintain precise imager positions. A precise alignment method for attaching imagers to prism is therefore essential for developing a camera system with high resolution. We propose a method with high detectivity using a sinusoidal pattern chart that easily reproduced by one imager, and a signal process. Images from this camera can attain a limiting resolution of more than 3200 TV lines.

We have developed a precision, 4-side buttable CCD package for 2kx2k and 2kx4k format devices with minimal mechanical stress on the CCD, excellent thermal properties, reliable electrical connectivity, and shim-free mounting. We report on the package design, assembly and quality assurance procedures, measurements of packaged device flatness and flatness excursions when cooled from room temperature to 140 K, package performance and plans for future development.

The paper describes several important characterization results obtained from the recently developed high performance front side illuminated 1k x 1k Frame Transfer CCD image sensor that employs charge multiplication concept to multiply photo generated charge directly in charge domain before its conversion into a voltage. The description includes the key device design features that were instrumental in obtaining the high performance and then focuses on the key characterization parameters such as excess noise and carrier distribution of the charge multiplication process. The remaining focus of the article is on the characterization methodology and on the obtained results such as the high clocking frequency (35 MHz), low serial register clock voltage operation, high sensitivity at low light levels, high QE, high blue response, no image lag, overload performance of lateral anti blooming drain structure, etc. In conclusion, several imaging comparisons between the conventional state of the art CCD image sensor and the developed charge-multiplying image sensor are also shown.

The creation of curved CCD’s and the mosaicing of contoured CCD’s mounted within the curved focal planes of telescopes and stereo panoramic imaging cameras introduces a revolution in optical design that greatly enhances the scientific potential of such instruments. In the alteration of the primary detection surface within the instrument’s optical system from flat to curved, and precisely matching the applied CCD’s shape to the contour of the curved focal plane, a major increase in the amount of transmittable light at various wavelengths through the system is achieved, thereby enabling multi-spectral ultra-sensitive imaging for a variety of experiments simultaneously, including autostereoscopic image acquisition. For earth-based and space-borne optical telescopes, the advent of curved CCD’s as the principle detectors provides a simplification of the telescope’s adjoining optics, reducing the number of optical elements and the occurrence of optical aberrations associated with large corrective optics used to conform to flat detectors. New astronomical experiments may be devised in the presence of curved CCD applications, including 3 dimensional imaging spectroscopy conducted over multiple wavelengths simultaneously, wide field real-time stereoscopic tracking of remote objects within the solar system at high resolution, and deep field mapping of distant objects such as galaxies with much greater precision and over larger sky regions. Stereo panoramic cameras equipped with arrays of curved CCD’s will require less optical glass and no mechanically moving parts to maintain proper stereo convergence over wider perspective viewing fields than their flat CCD counterparts, making the cameras lighter and faster in their ability to scan and record 3 dimensional objects moving within an industrial or terrain environment. Preliminary experiments conducted at the Sarnoff Corporation indicate the feasibility of curved CCD imagers with acceptable electro-optic integrity. Currently, we are in the process of evaluatingthe electro-optic performance of a curved wafer scale CCD imager. Detailed ray trace modeling and experimental electro-optical data performance obtained from the curved imager will be presented at the conference.

We developed a new method of pixel-level Analog-to-Digital (A-D) conversion for vision chips, removing Fixed Pattern Noise (FPN) at the same time. Vision chips are CMOS image sensors integrating a processing element (PE) and a photodetector (PD) in each pixel. The chip can handle high frame rate images in real time because its processing speed is high due to the parallel processing and also because it does not need high-bandwidth communication. Pixel-level A-D conversion is an essential technology for vision chips because digital operations must be performed in each pixel. The vision chip, which we have developed, contains a programmable PE in each pixel, and it directly controls the behavior of the PD with the use of the software. In our developed method, the chip controls the reference voltage to cancel the FPN by using this feature. We applied this method to our vision chip, and confirmed that the FPN was reduced and the sensitivity improved. We made a test chip including only PDs to solve the problem on the existing vision chip. As a result of applying this method to the test chip, the detectable minimum illuminance improved about 40 times in comparison with applying our existing method.

A new pixel structure for high dynamic range imaging is proposed. The internal pixel circuit resets the pixel each time the well capacity nears saturation and a counter, implemented in the pixel itself (or in an external memory), records the number of resets per integration time. The increase in the dynamic range is given by a factor of (m + 1) where m is the number of internal resets per integration time. The peak signal-to-noise ratio will be increased due to the effective increase in the well capacity. A multiple sampling technique can be used in the combination with this proposed structure to achieve a further increase in dynamic range.

The design and implementation of a smart image sensor to provide high dynamic range and pixel level digital image processing is described. In this article we investigate important issues in the development of digital intelligent pixel CMOS image sensors. We have developed a high density imager with pixel level stochastic arithmetic and a high dynamic range exceeding 90 db. The ASIC prototype named PIKASSO includes a 96×64 pixel array, each pixel has a fill factor of 15% in an area of 29×29 μm². The average power consumption per pixel at a frequency of 150 kHz is 78 μW.

We will present a 3044 x 4556 pixels CMOS image sensor with a pixel array of 36 x 24 mm², equal to the size of 35 mm film. Though primarily developed for digital photography, the compatibility of the device with standard optics for film cameras makes the device also attractive for machine vision applications as well as many scientific and highresolution applications. The sensor makes use of a standard rolling shutter 3-transistor active pixel in standard 0.35 μm CMOS technology. On-chip double sampling is used to reduce fixed pattern noise. The pixel is 8 μm large, has 60,000 electrons full well charge and a conversion gain of 18.5 μV/electron. The product of quantum efficiency and fill factor of the monochrome device is 40%. Temporal noise is 35 electrons, offering a dynamic range of 65.4 dB. Dark current is 4.2 mV/s at 30 degrees C. Fixed pattern noise is less than 1.5 mV RMS over the entire focal plane and less than 1 mV RMS in local windows of 32 x 32 pixels. The sensor is read out over 4 parallel outputs at 15 MHz each, offering 3.2 images/second. The device runs at 3.3 V and consumes 200 mW.

We describe our programme to develop science-grade CMOS active pixel sensors for future space science missions, and in particular an extreme ultra-violet spectrograph for solar physics studies on the ESA Solar Orbiter. Our goal is the development of a large format 4k x 4k pixel CMOS sensor with useful sensitivity in the extreme ultra-violet (EUV) for solar physics spectroscopy and imaging. Our route to EUV sensitivity relies primarily in adapting the back-thinning and rear-illumination techniques first developed for CCD sensors; however we are also exploring the alternative approach of using a front-etch to expose the CMOS photodiodes. We have successfully back-thinned several 525 x 525 prototype CMOS sensors and proved that the devices survived the process both structurally and functionally. We have also been successful in removing the oxide from the front side of a small array of pixels, using focused ion beam etching. Preliminary results from these pixels show they are sensitive in the Ultra Violet. We have also designed a working large format 4k x 3k prototype on a 0.25 micron CMOS imager process.

Recently introduced CMOS sensors using three layers of photodiodes for color separation1 can also function well in the near ultraviolet and infrared bands. Ultraviolet sensitivity results from the close proximity of the top (blue) photodiode junctions to the surface of the silicon and the lack of any significant UV-absorbing materials above them. Infrared sensitivity extending nearly to the silicon band-gap cutoff results from depletion of the bottom (red) phodiodes into the substrate. Preliminary measurements indicate that the layered structure has high quantum efficiency over most of the 200-1100nm band covered by silicon photodiodes. Uniquely, these devices can be switched rapidly between narrowband monochrome imaging and full-color imaging in the visible band by the introduction of a visible pass filter. The response of the three photodiode layers is broad enough to permit stable false color encoding using two or three channels in conjunction with a redefinable 3 x 3 color transform matrix. Images have been acquired in the 300-400 nm UV band and for broad and narrow infrared bands out to 1064 nm. Thermal images of objects in the range of 600C have also been acquired demonstrating color-encoding of various UV, visible and IR bands and applications for particular encoding schemes.

We demonstrate a non-orthogonal architecture for a CMOS active pixel image sensor, called here pyramid architecture, for improved two-dimensional spatial sampling. In the pyramid architecture 2D sampling using concentric rings replaces the 1D row sampling in the classical imager architecture, and diagonal output busses replace the conventional vertical column busses. Moreover, we propose a scanning scheme in which, instead of rolling over to the first ring (or row) at the end of image capture, the scan returns from the outer ring towards the first inner ring at the centre of the sensor. This leads to two scenes of differing integration times that, after being fused, results in a foveated increase in intra-scene dynamic range. Results from a sensor fabricated in 0.18μm CMOS technology are presented and discussed. We will also present a multi-resolution architecture which is based on the pixel structures as building block to control the acquired image resolution.

We present the performance characteristics of a CMOS image sensor, manufactured on wafers with a specially designed multiple epitaxial layer. At the homo-junction between two consecutive epitaxial layers a small potential drop or electric field represents a barrier for electrons diffusing towards the back of the wafer. The multiple epitaxial layer stack results thus in a net drive or confinement of photo-charges towards the surface. As a result there is anisotropical diffusion of charge that are generated deep in the Silicon, e.g. by near infrared (NIR) or X-ray radiation. The spectral response is an order of magnitude higher for than for the same image sensor on "regular" wafers. The anisotropical diffusion results in a limited MTF degradation compared to wafers with a single thick epitaxial layer.

A 1-D CMOS digital pixel image sensor system architecture is presented. Each pixel contains a photodiode, a low-power charge-sensitive amplifier, low noise sample/hold circuit, an 8-bit single-slope ADC, a 12-bit shift register and timing & control logic. The pixel is laid out on a 4µm pitch to enable a cost efficient implementation of high-resolution pixel arrays. Fixed pattern noise (FPN) is reduced by a charge-sensitive feedback amplifier, and the reset noise is cancelled by correlated double sampling read out. A prototype chip containing 512 pixels has been fabricated in the TSMC .25um logic process. A 40μV/e^- conversion gain is measured with 100 e- rms read noise.

This paper describes a 2048x1 linear image sensor implemented in a 0.35 μm 4M1P CMOS process that uses a low fixed pattern noise capacitive transimpedance amplifier (LFPN CTIA) pixel architecture. The pixel also includes circuitry for reducing 1/f noise, correlated double sampling, electronic shuttering, and a horizontal anti-blooming drain. High speed non-destructive readout of the sensor is achieved by using a hierarchical readout structure with two output ports. Using a JTAG interface the sensor can be programmed to operate in multiple readout modes. In the fastest readout mode, ROI, the sensor achieves 90Mpixel/sec (43.4Klines/sec) with 14e- RMS read noise. In the lowest noise mode, MRDI, with 13x oversampling of each pixel the sensor achieves 2.7Klines/sec with 1.2e- RMS read noise.

There is an urgent need to replace film and CCD cameras as recording instruments for transmission electron microscopy (TEM). Film is too cumbersome to process and CCD cameras have low resolution, marginal to poor signal-to-noise ratio for single electron detection and high spatial distortion. To find a replacement device, we have tested a high sensitivity active pixel sensor (APS) array currently being developed for nuclear physics. The tests were done at 120 keV in a JEOL 1200 electron microscope. At this energy, each electron produced on average a signal-tonoise ratio about 20/1. The spatial resolution was also excellent with the full width at half maximum (FWHM) about 20 microns. Since it is very radiation tolerant and has almost no spatial distortion, the above tests showed that a high sensitivity CMOS APS array holds great promise as a direct detection device for electron microscopy.

This paper presents a high-speed CMOS image sensor of whose frame rate exceeds 2000 frames/s. The pixel includes a photodiode, a charge-transfer amplifier, and circuitry for correlated double sampling (CDS) and global electronic shuttering. Reset noise, which is a major random noise factor, is greatly reduced by the CDS combined with the charge-transfer amplifier. The total number of devices in the pixel is 11 transistors and 2 MOS capacitors. Test circuits were fabricated using a 0.25μm CMOS process. The sensitivity of the 20 x 20μm² pixel using the floating diffusion capacitor of 6.2fF and the photodiode area of 15 x 12.7μm² is 34V/lux-sec. At 1000frames/sec, noise level is 2.43mV_rms (dark). The noise level and the sensitivity are greatly improved compared with a 3Tr. type APS implemented with the same technology and a previous version of the APS with in-pixel CDS.

Forensic photography, which was systematically established in the late 19th century by Alphonse Bertillon of France, has developed a lot for about 100 years. The development will be more accelerated with the development of high technologies, in particular the digital technology. This paper reviews three studies to answer the question: Can the SLR digital camera replace the traditional silver halide type ultraviolet photography and infrared photography? 1. Comparison of relative ultraviolet and infrared sensitivity of SLR digital camera to silver halide photography. 2. How much ultraviolet or infrared sensitivity is improved when removing the UV/IR cutoff filter built in the SLR digital camera? 3. Comparison of relative sensitivity of CCD and CMOS for ultraviolet and infrared. The test result showed that the SLR digital camera has a very low sensitivity for ultraviolet and infrared. The cause was found to be the UV/IR cutoff filter mounted in front of the image sensor. Removing the UV/IR cutoff filter significantly improved the sensitivity for ultraviolet and infrared. Particularly for infrared, the sensitivity of the SLR digital camera was better than that of the silver halide film. This shows the possibility of replacing the silver halide type ultraviolet photography and infrared photography with the SLR digital camera. Thus, the SLR digital camera seems to be useful for forensic photography, which deals with a lot of ultraviolet and infrared photographs.

This paper describes software development for an Airborne Multi-Spectral Imaging System that uses digital cameras to provide high resolution images at very high rates. The software controls the camera and the GPS receiver and allows the remote manipulation of various functions, including play, stop, and rewind. The GPS co-ordinates and the corresponding time are simultaneously recorded. The system is viable for many applications that require reasonably good resolution at low cost. Such applications include vegetation detection, oceanography, marine biology, geographical information systems, and environmental coastal science analysis. The paper presents results of two successful flight tests.

In these days, the CMOS image sensors are commonly used in many low resolution applications because the CMOS imaging system has several advantages against the conventional CCD imaging system. However, there are still several problems for the realization of the single-chip CMOS imaging system. One main problem is the substrate coupling noise, which is caused by the digital switching noise. Because the CMOS image sensors share the same substrate with surrounding digital circuit, it is difficult for the CMOS image sensor to get a good performance. In order to investigate the substrate coupling noise effect of the CMOS image sensor, the conventional CMOS logic, C-CBL (Complementary-Current balanced logic) and proposed low switching noise logic are simulated and compared. Consequently, the proposed logic compensates not only the large digital switching noise of conventional CMOS logic ,but also the huge power consumption of the C-CBL. Both the total instantaneous current behaviors on the power supply and the peak-to-peak voltages of the substrate voltage variation (di/dt noise) are investigated. The simulation is performed by AMI 0.5μm CMOS technology.

The range of color spaces for possible use with digital photography presents challenges and opportunities for engineers, developers and users of digital cameras. This paper will provide an overview and comparison of a sub-set of these color spaces, with specific consideration for digital photography. Primarily the RGB or red, green and blue color spaces will be compared and although other spaces, such as CIELAB and Yu’v’, will be mentioned. Background will be provided with respect to key considerations for design and use of color spaces for digital photography. Data will be presented on color space uniformity using a radial sampling of the OSA Uniform Color Scales and tritanopic confusion lines. In addition, color difference statistics across a fixed sampling will be used to assess the similarity of color spaces and to quantify the resulting error when color spaces are incorrectly assigned. Finally, the gamut shapes will be compared using procrustes analysis of a set of gamut landmark colors. The resulting dendrogram provides a means of visualizing relative similarities of the gamut shapes.

Six different methods for white-balancing digital images were compared in terms of their ability to produce white-balanced colors close to those viewed under a specific viewing illuminant. The six methods were: native camera RGB, XYZ, CAM02, ITU Rec BT.709 RGB, sharpened camera RGB, and illuminant-dependent. 4096 different sets of camera sensitivities were synthesized; 170 objects were evaluated under a canonical viewing illuminant (D65) and six additional taking illuminants (A, D50, D75, F2, F7, and F11). Each white balancing method was exercised in turn, and the mean and 90th percentile ΔE*_ab were determined. We found that illuminant-dependent characterization produced the best results, sharpened camera RGB and native camera RGB were next best, XYZ and CAM02 were often not far behind, and balancing in the -709 primaries was significantly worse. We recommend that, whenever the illuminant is identified, the illuminant-dependent technique be employed because of its superior performance.

As an optical low-pass filter, a doubly refractive crystal device is used and performs separation of its incident light into the normal light and the abnormal light shifted to the slightly different direction to the normal light. The actual optical low-pass filter is formed by combining two types of doubly refractive crystal device; the one separates its incident light into two traveling directions horizontally spaced each other by one pixel, and the other does vertically spaced by one pixel. The filter cannot sharply cut off high frequency components and it reduces the frequency components near the Nyquist frequency. Thus, images projected on the imaging surface are blurred. This paper presents a demosaicking method that can simultaneously remove image blurs caused by the optical low-pass filter. Most of the existing demosaicking methods do not try to remove the image blurs, whereas our sharpening approach to the demosaicking employs the Landweber-type iterative algorithm. For our sharpening-demosaicking approach, the Bayer’s pattern of the RGB primary color filter array is not necessarily proper, and hence we study another color-filter pattern, namely the WRB filter array that is preferable to the RGB filter array, where the W-filtering means that all the visible light passes through it. Our mathematical formulation of the sharpening-demosaicking has the form of the least square problem, but there exist multiple different least square solutions. To avoid its ambiguity, in the spatial frequency domain we introduce the pass-band limitation corresponding to the sub-sampling pattern of the mosaicking of color filters, into the iterative algorithm.

This paper proposes demosaicing methods for video clip of digital still cameras (DSCs). In video clips of DSCs, pixel mixture is performed for saving readout time, which mixes together two pixels on CCD. Pixel mixture reduces the number of pixels to be read out from CCD, and it enables video rate read out from DSCs of more than a million pixels. In this paper, an iterative method is presented in order to recover the image. Our approach achieves the results of smaller mean square errors than conventional interpolation methods.

Color aliasing, in its most visible and objectionable form, colored moire, persists as a drawback to digital imaging of periodic objects, particularly fabrics. Sensors with mosaic color filters typically measure only 1/3 of the color information for each pixel. With an optical modification, a single subject point can be measured by laterally displaced red, green, and blue pixels. This is done by a shift of the color planes laterally using a dispersive optical element. The element creates an optical effect similar to chromatic aberration but uniform in magnitude and direction over the image. Light from a subject point passes through the optical system but is not focused to a point in the image plane. Instead the red component is focused to a different point than the green and blue components and all are arranged in a line. This element, or filter, causes one subject point to be measured by three sensor pixels, one red, one green and one blue. This in fact may result in an image having more useful information: fewer pixels of color-registered, error free data, rather than a larger number of pixels with color errors.

When taking pictures, professional photographers apply photographic composition rules, e.g. rule of thirds. The rule of thirds says to place the main subject's center at one of four places: at 1/3 or 2/3 of the picture width from left edge, and 1/3 or 2/3 of the picture height from the top edge. This paper develops low-complexity unsupervised methods for digital still cameras to (1) segment the main subject and (2) realize the rule-of-thirds. The main subject segmentation method uses the auto-focus filter, opens the shutter aperture fully, and segments the resulting image. These camera settings place the main subject in focus and blur the rest of the image by diffused light. The segmentation utilizes the difference in frequency content between the main subject and blurred background. The segmentation does not depend on prior knowledge of the indoor/outdoor setting or scene content. The rule-of-thirds method moves the centroid of the main subject to the closest of the four rule-of-thirds locations. We first define an objective function that measures how close the main subject placement obeys the rule-of-thirds, and then reposition the main subject in order to optimize the objective function. For multiple main subjects, the proposed algorithm could be extended to use rule-of-triangles by adding an appropriate constraint.

In this paper, we present a super-resolution method to approximately double image resolution in both dimensions from a set of four low resolution, aliased images. The camera is shifted and rotated by small amounts between the different image captures. Only the low frequency, aliasing-free part of the images is used to find the shift and rotation parameters. When the images are registered, it is possible to reconstruct a higher resolution, aliasing-free image from the four low resolution images using cubic interpolation. We applied our algorithm in a simulation, where all parameters are known and controlled, as well as in a practical experiment using images taken with a real digital camera. The results obtained in both tests prove the validity of our method.

Most fine art reproduction workflows to date have been based on hyperspectral devices. These devices capture, process and print more than three channels of spectral data to produce spectrally accurate reproductions. While these workflows have unique advantages over standard three-channel workflows, such as the ability to produce reproductions that are colorimetrically accurate across many illuminants, they usually require custom hardware. Such hardware can be expensive, time-consuming to setup, and may require a full-time trained operator. We describe the challenges and issues in constructing a colorimetrically accurate fine art reproduction work- flow based on standard three-channel hardware. The workflow was designed to be as automated as possible, simple to use, and device-independent. The heart of the workflow is a software application that takes as input camera characterization data, reflectance statistics of the artwork, an image of the artwork, and an image of a reference card, and it outputs a properly exposed, uniformly illuminated and colorimetrically accurate reproduction. We describe the methods used to compute the exposure level, to compensate for illumination non-uniformities, and to generate a per-image color correction matrix. Finally, we present reproduction results and error statistics obtained using a workflow comprising a 4x5” Sinar camera, a Betterlight digital back, and an HP DesignJet 5500 printer.

This work characterizes three different types of sensor defects, and investigates the applicability of the Contrast Threshold Function (CTF) of the human visual system to the manufacturing test criteria for CMOS image sensors. The sensor defect types characterized are resist streaking, dye color spots, and orange-peel photodiode sensitivity noise. Algorithms are presented to objectively identify and rate the severity of each. Visual evaluations determined the subjective level of detectability and objectionability of each. The spatial frequency and modulation of the defects were measured, and compared with an appropriate CTF. The result is the minimum defect levels noticeable in test images can be almost order-of-magnitude higher than the known CTFs determined for the limits of human visual system sensitivity.

We present a virtual digital camera sensor, whose aim is to simulate a real (physical) image capturing sensor. To accomplish this task, the virtual sensor operates in two steps. First, it accepts a physical description of a given scene and simulates the entire process of photon sensing and charge generation in the sensor device. This process is affected by noise, mostly photon noise. Second, it adds to the image the noise that results from the electronic circuitry. We present a model for the different sources of noise relative to each sensor-based image formation step, and use measurements of real digital camera images to validate the model.

We describe the design of a very high resolution, low-cost scan camera for use in image-based modeling and rendering, cultural heritage projects, and professional digital photography. Our camera can acquire black&white, color, and nearinfrared images with a resolution of over 122 million pixels and can be readily built from off-the-shelf components for less than $1200. We discuss the construction of the system as well as color calibration and noise removal. Finally, we obtain quantitative measurements of the light sensitivity and the optical resolution of our camera and compare the image quality to a commercial digital SLR camera.

A new technique able to improve the performance of the standard DCT compression algorithm in terms of compression size, maintaining almost constant the perceived quality is presented. The measured improvement is obtained profiling the relative DCT error inside typical image pipeline of data acquired by digital sensors. Experimental results show the effectiveness of the methodology proposed, validated also by using two perceptual quality metrics.

A CMOS active pixel sensor array using column-level active reset has been fabricated and tested. Column-level active reset requires one additional transistor per pixel, bringing the total to 4, and a per-column op-amp. The added transistor per pixel controls the gate of the reset transistor. There are two important feedback mechanisms in active reset. The first is the amplification by the Miller effect of the effective capacitance on the photodiode during reset, hence reducing kT/C noise. The second is the control of the resetting current via modulation of the transconductance of the reset switch. The level of noise reduction is comparable to, or may exceed, what true correlated double sampling can achieve. Readout noise of 44 microvolts and a dynamic range of 90 dB (15 bits), rms, has been measured. Room temperature noise as low as 5.1 electrons, rms, referred back to the photodiode node, has been measured on small photodiode pixels (5.8 fF capacitance). This compares to 38.3 electrons when measured with a standard “hard” reset, for a factor of 7.6 improvement. Row and column fixed pattern noise were also improved by up to a factor of 21, going from 1% for both to 0.048% and 0.27%, respectively.

A new type of imaging sensor suitable for digital SLR cameras has been developed. Each pixel in the sensor comprises a buried photodiode, a junction field-effect transistor (JFET), a transfer gate, two reset gates, and a reset drain. The JFET receives signal charge directly from the buried photodiode through the transfer gate and outputs the converted voltage signal, while the reset gate together with the reset drain controls both resetting and selecting operations of the JFET. The pixel does not have a floating diffusion nor a row select gate, resulting in a simplified structure, an increased fill factor and a better production yield. Another feature of the sensor is the parallel readout scheme. Signals from green pixels arrayed in a checker pattern are read out from one output terminal, while those from red and blue pixels are read out, row-by-row, from the other output terminal. This enables high-speed readout without introducing fixed pattern noise. Total and effective pixel numbers are 4.26 and 4.08 mega respectively. A pixel size is 9.4um square with a fill factor of 36%. Power consumption is 600mW under consecutive operation mode of 9.3 frames/sec. Saturation output voltage is 700mV with a noise floor of 0.20mVrms, having a dynamic range of about 70dB including the camera system.

This work shows the progress and demonstrates the measurements performed via a unique submicron scanning system developed at the VLSI systems center in Ben-Gurion University. The system enables the combination of near-field optical and atomic force microscopy measurements with the standard electronic analysis. The obtained signal, i.e., the electrical outcome at each point as a function of the spot position provides a 2D signal map of the pixel response, representing the full 3D charge distribution in the device. This work present the results obtained by thorough scanning of several various pixel topologies of CMOS APS chips fabricated in two different CMOS technologies (the standard 0.5μm and 0.35μm CMOS technologies). We demonstrate that our system use enables a detailed, point by point, quantitative determination of the contributions to the total output signal from each particular region of the pixel. It makes possible to understand the influence of the each component composing the pixel (e.g., logic transistors, metal lines, etc.) which is extremely important for CMOS APS where the pixel structure defines a fill factor of less then 100%.

We found that accurate estimation of the actual resist patterns and impurity profiles is the key point in the case of image sensors below 2.5 um square cell size. We apply a resist patterning process model to our process/device simulation. In the photolithography process simulation, each patterned resist layer exhibits own resist corner rounding regarding as differences such as resist thickness and wavelength of stepper. For the ion implant processes and thermal processes, channeling and doped impurity diffusion models are newly applied. We introduced two dimensional Monte Carlo simulation in order to estimate channelings affected by impurity species, accelerating voltage of implanter and crystallographic orientation. This enables to get impurity profiles of implant processes with mega order accelerating energy. Three dimensional impurity diffusion profiles can be obtained by using the optimized ratio of lateral diffusion to perpendicular diffusion. We have confirmed the advantage of the new simulation method by evaluation of device characteristics in small size CCDs.

Analysis of dynamic-range (DR) and signal-to-noise-ratio (SNR) for high fidelity, high-dynamic-range (HDR) image sensor architectures is presented. Four architectures are considered: (i) time-to-saturation, (ii) multiple-capture, (iii) asynchronous self-reset with multiple capture, and (iv) synchronous self-reset with residue readout. The analysis takes into account circuit nonidealities such as quantization noise and the effects of limited pixel area on building block and reference signal performance and accuracy. Examples that demonstrate the behavior of SNR in the extended DR and implementation and power consumption issues for each scheme are presented.

Nowadays, digital still cameras are becoming as popular as conventional film cameras. As over 3 mega-pixel cameras become the main stream of digital still cameras, they are accepted from a point of view of pixel numbers. However, from a point of view of "scene toughness," digital still cameras need further improvement. Current digital cameras have difficulties reproducing high-contrast images containing both dark and bright areas, with shadows tending to lose details and whites washing out. In order to improve "scene toughness," we have developed a new concept camera system, which can capture wider dynamic range images. Applying miniaturization technology of CCD device, we developed the new structure CCD, the Super CCD SR. One photodiode of the new type CCD is divided into two parts, and each of the two photodiodes has a different size and a different sensitivity. When it outputs an image by optimally combining the images from a high sensitivity part and a low sensitivity part, it has wider dynamic range, that means the camera equipped with the new CCD has the advantage of getting wider dynamic range photographic images by one device and by one exposure. To make the best use of this CCD device, new technique in signal processing and new automatic camera control are important, then we have developed the automatic system that controls the camera corresponding to the scene variation. In this paper we discuss the concept of the signal processing and the automatic camera control for the camera equipped with the Super CCD SR. In the front-light condition the camera exposure and tone control are operated not to lose highlight details. In backlight containing both dark and bright areas, these are operated to reduce flat shadows of main subjects and washing out of highlights against strong sunlight in the background. The camera we developed is capable of obtaining wider dynamic range images and achieves richer and smoother tonality for better reproduction of fine detail, and gets over the above problem and improves "scene toughness."

Recently the digital still camera (DSC) has achieved remarkable progress especially in terms of pixel numbers, and 3 mega-pixel cameras have become the main stream. This is owing to the progress of semi-conductor processing technology and DSC market demand. On the other hand, there are some problems accompanying increase of pixel numbers, for example degradation of S/N and increase of shooting interval. For the product described here we have applied miniaturization processing technology to incresing the dynamic range and have developed a Super Dynamic Range Image Processing System using a new structure CCD, in which a pair of different-type photodiodes is arranged under a micro lens that is located on the top surface. This makes it possible to capture a high contrast scene in which dynamic range is about 400%, and we have achieved distinct images of difficult scenes for a DSC to reproduce such as backlight scenes, short range stroke light photography, etc. Here, we explain the new CCD and LSIs which realize the super dynamic image processing system, and its algorithm.

We compared the Spatial Frequency Response (SFR) of image sensors that use the Bayer color filter pattern and Foveon X3 technology for color image capture. Sensors for both consumer and professional cameras were tested. The results show that the SFR for Foveon X3 sensors is up to 2.4x better. In addition to the standard SFR method, we also applied the SFR method using a red/blue edge. In this case, the X3 SFR was 3-5x higher than that for Bayer filter pattern devices.