The future of digital imaging looks bright. Not only because of its own great success but due to the huge amount of new innovations. New methods and approaches in camera sensors and camera modules force engineers to implement new image processing algorithms that can comprehensively utilize new features.
This section defines some of the trends, which can change the way images are captured and video is recorded.
2.3.1 Sensor innovations
In the sensor area alone, there are tens of different new methods which challenge current Bayer type sensors. Aptina and Sony have introduced their clear pixel sensors which have even replaced green pixels with white pixels, like Aptina or added extra white pixels to existing filters, like Sony (Business Wire, Sony). The
use of white i.e. unfiltered pixels increases the sensitivity of sensors. Sony has also published a patent which defines triangular and hexagonal pixels with seven different pixel types (Patent US 20130153748 A1).
Another approach is a quantum film invented by InVisage. The quantum film is a photosensitive layer which may replace the silicon from traditional sensors (Invisage). InVisage published a quantum film sensor with 13 mega pixels in late 2015. The sensor should provide a dynamic range which is three f-stops better than conventional CMOS sensors. Moreover, Foveon has published its X3 sensor with stacked photodiodes. The sensor is based on the fact that light with longer wavelengths penetrates silicon more deeply than light with shorter wavelengths.
Using this phenomenon, Foveon has implemented a layered pixel, where blue light is detected on the top of the pixel, green in the middle and red wavelengths at the base of the pixel (Foveon). Obviously this kind of sensor does not need a color filter array at all and should be more sensitive than sensors which are using one.
On the other hand, Xerox PARC and IMEC develop multispectral and hyperspectral sensors mainly for industrial use, but they could also add interesting features to mobile phone cameras (GlobeNewswire, IMEC). Finally, very recently Panasonic published an organic CMOS sensor which dynamic range should be significantly better than any other conventional sensor (Panasonic).
2.3.2 New steps in lens systems
Sensors are not the only area, where innovations of new techniques occurs. In optics, LensVector has released a liquid lens. A single lens component contains electrically controlled liquid crystals and the focus can be adjusted not by moving the lens but controlling the crystals which makes the focus adjustments very fast (LensVector). On the other hand, the micro electro-mechanical system technology (MEMS) has superior performance features over current voice coil motor (VCM) methods, but manufacturing problems still prevent the approach from reaching greater success (DigitalOptics). Rambus has a technology called lensless smart sensor, which includes a spiral grating of diffractive optics and sophisticated algorithms to capture an image without lenses (Rambus<). Finally, Sony has released a market ready product with an optical variable low pass filter, where a user may control the filter to find the balance between resolution and aliasing artefacts like Moiré (Sony Optics).
2.3.3 From one sensor to sixteen
Multiple cameras and array imaging are related to three dimensional (3D) imaging but there are also other features, which can be made using multiple sensors. Pelican Imaging has introduced a compact sensor matrix containing sixteen sensors mainly targeting 3D imaging. However the technique offers also high resolution imaging by combining the information from the multiple sensors and post-capture refocus for still images and videos (Pelican Imaging). Altek has made a system containing two 13 mega pixel sensors where one is chromatic, the other is monochromatic.
Altek advertise this as instant auto focus, high resolution, good low light performance with low noise and high dynamic range (Altek).
Light co. released in year 2015 perhaps the largest array imaging product: Their L16 camera with sixteen 13 mega pixel camera modules using three different focal length (Light). The product may challenge traditional digital single-lens reflex (DSLR) cameras. Also several time-of-flight (TOF) solutions are developed for 3D and depth imaging (Lytro, Heptagon).
Finally, what is the role of presence capture cameras? These cameras will record the whole environment, capturing 360 degrees or even 720 degrees in 3D including surround sound. The end user will be able to re-experience the original moment in a very new and comprehensive way. However, the solution requires new infrastructures for cameras, data transfer and displays.
All in all, digital imaging is rapidly changing. New products with astonishing features are already available or just around the corner and the markets and end users will decide the next successful trend. The rate of digital camera evolution challenges image quality metrics and measurements, too. When new techniques are taken into use, they will generate new features to validate and new artefacts to measure.
3 IMAGE QUALITY, DISTORTIONS AND
ARTEFACTS OF MODERN DIGITAL CAMERA
In a perfect world, a digital camera would reproduce exactly the photographed real world scene. The image would present all the smallest details, reproduce exact colors, without any noise and other artefacts, and in the case of consumer cameras, use the whole spectrum of the human eye. Also the dynamic range of the camera would be at least as good as the human eye and the image processing pipeline would mimic the brain’s visual processing in a perfect way.
A quick glance at the endless image galleries of the Internet reveals that obviously we do not live in a perfect world. Limitations of cameras’ hardware and software, manufacturing issues with camera sensors and lenses and problems in image processing pipelines cause different issues in images. The issues can be content destroying problems like out of focus adjustments and wrong exposure values or very small and even artistic faults like an unnatural bokeh or a slightly wrong color tint. Also nature itself creates final boundaries on image quality by limiting the performance of lenses and defining the smallest objects that can be observed using the wavelengths of the human vision system, for example.
When the concept of image quality is considered more closely, it can be seen problematic or even controversial. Though the image quality can be measured very comprehensively, in case of consumer products, images are ultimately judged by the human eye and by the human vision system. Although perceptual image quality and measured image quality correlates well, they definitely are not the same thing.
Strictly considering image quality as a measurable entity, image quality can be defined as an overall performance of the camera in reproducing the captured scene in an image. A quality distortion can be specified as a lack of performance and image artefacts are explicit errors in the images. However, image quality is not only an objective and measurable number, it is also a perceptual view of the image.
There have been several attempts to bind objective and perceptual quality metrics together. For example Keelan defined a specific method and function, the integrated hyperbolic increment function (IIHF) to transform any objective metric into a perceptual one (Keelan 2002). Also many current image quality standards have been updated to measure the perceptual image quality, too.
This chapter defines the problematic concept of image quality in general. The content is not limited to mobile phone cameras, because the quality entities are generic to most of digital cameras. The chapter specifies the image quality entities, distortions of image quality and common artefacts of digital imaging. The purpose
of the chapter is not just to describe or list the quality issues and artefacts but highlight the diversity and number of quality problems in digital imaging and the challenges, which different issues present in quality measurement and benchmarking.