Camera performance, meaning the functional speed of the camera in general or quickness of a certain camera functionality is a very novel measurement area. As late as 2013, the first ISO standard for camera speed was published: ISO 15781. A year before, CIPA DGC-002 standard was translated from Japanese including also camera performance measurement guidelines (ISO15781 2013; CIPA DCG-002 2012). Before then, the speed of camera features were perceived as a generic usability of the camera or smoothness of the user interface.
It can be argued, if camera performance is part of image quality at all. The consideration can be started from exposure time which is one of the most critical
entity of photographing in general. Under or over exposure can destroy the captured image entirely. The correctness and accuracy of exposure timing are certainly quality features of digital imaging. On the other hand, auto-exposure feature is a standard feature in mobile phone cameras. The accuracy and speed of auto-exposure algorithm enables both correct illumination of an image and smooth capturing functionality.
ISO 15781 standard highlights the situation where pictures are taken from moving targets. Too long delay between pressing the exposure button and real image capturing may ruin to preserve the moment (ISO 15781 2013). Furthermore, different features like auto-focus, image post processing, image stabilization and video recording may generate own delays to an image capturing process. The delays do not prevent image capturing nor reduce traditional image quality but they still can prevent to capture the required moment. The usage of an image captured in wrong moment is quite same than an image with poor image quality, the captured image is deleted from mobile phones memory.
Bucher et al. describes an interesting performance feature of mobile phone cameras.
In the research several cameras had a negative shutter lag. This means that a camera system is capable of storing frames during the whole capturing process and selecting required frame afterwards. (Bucher et al. 2014) Wrong functionality of the feature may generate a strange and unwanted phenomenon where a camera captures images too early.
Masson et al. notes other performance features which affect the functionality and quality of digital imaging. Speed of a rolling shutter affects significantly the rolling shutter artefacts. If the rolling shutter speed is slow, i.e. the delay between the first row exposure and last row exposure in an image sensor is long, it may cause distortion to moving objects or exposure errors to an image. The research includes also performance measurements of image stabilization. The research revealed how much exposure time can be extended when the image stabilization is active.
(Masson et al. 2014)
It can be assumed, that the significant growth of video recording will highlight the performance of auto focus and auto exposure speed and the smoothness of these features because they are no longer pre-processing steps in image capture but they affect the real recording result. Moreover, auto white balance will have same kind of convergence delay and it should be investigated, too.
Until now, camera performance factors have been more like usability features than quality factors, because they did not affect traditional image quality. However, fast functionality of the camera is a feature which allows a user to capture an instant
moment. Conversely slow functionality could prevent this capture. According to the measurements taken during this study, a camera can generate delays of several seconds when an image is captured. It can be discussed, if camera performance is an image quality feature, but in the case of camera usability, the role of camera performance is incontrovertible.
4 IMAGE QUALITY MEASUREMENT METHODS AND METRICS OF MOBILE PHONE CAMERAS
If all the image quality standards and de facto standards are listed, numerous quality metrics can be found and the metrics can be classified in different ways. Keelan defines in his book, Handbook of Image Quality, following division into separate objective metrics: (Keelan 2002)
- Quality metric, a single number value correlating to a perceptual image quality
- Objective measurement, a function of at least one variable, for example modulation transfer function (MTF) of the slanted edge test chart
- Engineering parameter, a single number value describing a property of a camera system, for example the pixel count
- Benchmark metric, a single number variable combining usually several objective metrics to compare features of the cameras
On the other hand, Wang and Bovik define methods for image quality measurement as follows: (Wang and Bovik 2006)
- Full-reference, no-reference and reduced-reference image quality measurement. The division is used frequently when image quality measurements are defined. Obviously, the methods of the image quality measurements are very different depending on the availability of the reference data.
- General purpose and application specific image quality measurement. The application specific measurement concentrates on some specific quality feature or artefact of the image, for example lens distortion or video artefacts. On the other hand, the general purpose measurements give a generic score or result of the image quality.
- Bottom-up and top-down image quality measurement. When the image quality methods are defined, they have to simulate or mimic the human vision system (HVS). There are two ways to build up the simulation. The bottom-up method divides the HVS simulation into its relevant components and psychophysical features and builds the simulation by combining features together. The top-down procedure creates an overall model of the entire HVS and defines the simulation as a black box model.
Traditional image quality standards are mostly based on objective measurements according to the classification of Keelan and the reduced reference image quality measurements by Wang and Bovik. The combination is quite practical because the full reference method requires an exact digital reference which is not always
available. On the other hand, the no-reference method has not reached the level of reliability required to measure image quality as well as required.
This chapter defines different image quality metrics and methods starting with standardization in general, describing the division of color, noise, dynamic range, and resolution metrics, clarifying the needs of artefact measurements, new algorithms and perceptual quality metrics and ending with video and performance metrics.