Master Erasmus Mundus in
Color in Informatics and Media Technology (CIMET)
ASSESSMENT AND ANALYSIS OF RESULTS FROM COSCH ROUND ROBIN TEST
Master Thesis Report
Presented by Adela Shah
At
University College London 5
thAugust 2016
Academic Supervisor: Prof. Jon Yngve Hardeberg Host Supervisor: Prof. Lindsay MacDonald
Jury Committee:
Prof. Alain Tremeau
Asst Prof. Eric Dinet
Abstract
In the field of art and cultural heritage, it is necessary to conserve the legacy of historic artworks that are inherited from the past, maintained in the present, and bestowed on future generations to come. As a result, there has been a number of high-end digital imaging devices developed in order to accurately document such objects endeavouring to provide spectral and colorimetric characterization across the entire surface of an object so that the colour materials can be identified and colour changes can be measured.
And so, in spite of increasing technological advancement, various difficulties exist, and depending on the system used, data accuracy and information reliability can vary. This is due to two variables: device parameters and imaging conditions. Device parameters include: sensor characteristics, spectral sensitivity, spectral range, noise, dynamic range, optics, data formats. In terms of imaging conditions: illuminant spectral power distribution, imaging geometry and illuminant non- uniformities.
In the absence of best practice guidance, to standardize methodologies and instruments for the analysis of various types of coloured artworks, COSCH Working Group 1 (WG1) has carried out a Round Robin Test (RRT) exercise. The primary aim of such exercise is to standardize methodologies, clarify problems, share solution and skills and to promote research, development and application of optical measurement techniques, adapted to the needs of heritage documentation.
The main objective of this thesis is to be able to analyse spectral and colorimetric accuracy of eight multispectral imaging systems developed by different groups in Europe who are participating in the Round Robin Test exercise. In this work, the analysis was based on two RRT objects: X-Rite® ColorChecker Classic and X-Rite®
White balance. X-Rite® ColorChecker Classic defines the characteristic of each imaging device in color rendering whereas X-Rite® White balance defines the level of homogeneity in illumination during image acquisition and in detector elements. The data obtained from different spectral imaging devices were compared with the ground truth reference data acquired from Perkin Elma Spectrophotometer by University of Eastern Finland. This comparison allowed assessing the accuracy of spectral and colour reproduction processes performed by eight multispectral imaging systems.
The results obtained are satisfactory in terms of spectral accuracy but for colorimetric accuracy most of the devices showed error for one or some colours.
Acknowledgements
I would first like to thank my thesis host-supervisor Professor Lindsay MacDonald of the Civil Environmental and Geomatic Engineering Department at the University College London. He was always readily available whenever I ran into difficulties and steered me in the right direction as and when required.
I would also like to acknowledge and thank Professor Alain Tremeau of the Faculty of Sciences at the University Jean Monnet; Doctor Mona Hess of the Civil Environmental and Geomatic Engineering Department at the University College London; Professor Jon Yngve Hardeberg of the Faculty of Computer Science and Media Technology at the Gjøvik University College for all of your support and guidance as well as the University College London (UCL) and Colour And Space in Cultural Heritage (COSCH) for this opportunity.
Finally I must express my very profound gratitude to my parents and close friends for providing me with unfailing support and endless encouragement throughout my years of study and through the process of researching and writing this thesis. This accomplishment would not have been possible without you.
Thank you.
Adela Shah
Contents
Abstract……….….……….I Acknowledgement……….………II Contents………..……….……… III List of Figures……….……….…..…..V List of Tables ……….….……VIII
1. Introduction………..………1
References………..11
2. Literature review 2.1 COSCH: Color and space in cultural heritage………..………..…..5
2.1.1 COSCH RRT………..………..7
References………..…………17
2.2 Multispectral Imaging ………..………13
2.2.1 Narrow and Wide band image capture………...………14
2.2.2 Calibration and sources of error………..………16
2.2.3 Image and signal processing………...…..…………17
2.2.3.1 Transformation to colour space……….………20
2.2.3.2 Evaluation of system performance………22
References………31
3. Experimental Design 3.1 Targets 3.1.1 X-rite ColorChecker chart………..27
3.1.2 X-rite white balance card………27
3.2 Sources of data………..28
3.3 Data Acquisition systems 3.3.1 University of Eastern Finland (UEF)Spectral measurement system……….……..28
3.3.2 University College London (U.K) MSI systems……….….……29
3.3.3 University Milano (Italy) & University Jean Monnet (UJM, France) MSI systems………..……33
3.3.4 University of Basel (Switzerland) MSI systems……….…….34
3.3.5 STARC ………..…36
3.3.6 Technical University of Catalonia (UPC) MSI systems………...…37
3.4 Datasets Information 3.4.1 Datasets specifications……….39
3.4.2 Problems with datasets……… 3.5 Methodology 3.5.1 Dataset correction for Nikon D200………41
3.5.2 Dataset correction for U I monochrome……….………44
3.5.3 Image datasets from external laboratories………47
3.5.4 Extraction of spectral reflectance data from X-rite ColorChecker……50
3.5.5 Interpolation method for spectral reconstruction……….50
3.5.6 Computing tristimulus values………..…51
3.5.7 Computing CIELAB coordinates……….………51
3.5.8 Computing Color, Lightness, Chroma and Hue difference………..……52
3.5.9 Computing Spectral differences………..…52
References………..…61
4. Results 4.1 Root mean square error……….54
4.2 CIELAB metrics under D65 Illuminant……….66
5. Discussion of results………82
6. Conclusion………88
7. Appendix: Appendix A: Information acquired from each group………i
Appendix B: Some of the matlab functions used………..……….x
Appendix C: Additional results of CIELAB metrics………..……….xii
Appendix D: Additional RRT performed by 3 laboratories...xiv
List of Figures
Figure 1: COSCH project overview………5
Figure 2: RRT test objects 1 to 5………7
Figure 3: Illustration of Multispectral images………13
Figure 4: Example of narrow and wide band filters spectral transmittances………14
Figure 5: Interpretation of CIELAB colour space……….……21
Figure 6: X-rite ColorChecker Classic……….…………27
Figure 7: Representation of 24 colour patches………..…… 27
Figure 8: X-rite White balance………27
Figure 9: Spectral reflectance curve of 24 colour patches measured with Perkin Elma Lambda 18 Spectrophotometer………29
Figure 10: DSLR Nikon D200 camera………30
Figure 11: UI 5480CP-M-GL monochrome camera………30
Figure 12: Transmittance factors of 21 optical bandpass filters in the visible and NIR spectrum………31
Figure 13: Acquisition setup for Nikon D200 camera………31
Figure 14: Spectral reflectance data of color patches measured from X-rite Spectrophotometer………32
Figure 15: Spectral reflectance curve of white card measured from Eye one X-rite Spectrophotometer ………32
Figure 16: DMK41AU02.AS monochrome camera………33
Figure 17: DSLR Canon 1000D camera ……….………33
Figure 18: Acquisition setup for Nikon D3 ………..34
Figure 19: Transmittance factors of 13 interference filters in the visible spectrum……34
Figure 20: DSLR Nikon D3 camera………...……35
Figure 21: Study of artwork using MuSIS system as Imaging Colorimeter or
Imaging Spectrometer……….……..…..36
Figure 22: Hardware setup for MuSIS HS system……….……….……36
Figure 23: MuSIS software for acquisition control……….….…..…36
Figure 24: QICAM fast 1394 camera……….…….……37
Figure 25: Spectral transmittance of the IR cutoff filter and LCTF spectral transmittances……….…38
Figure 26: RGB images of ColorChecker and White card acquired from Nikon D200 using interference filters……….……41
Figure 27: Spectral reflectance curve of X-rite white card measured using spectrophotometer ………42
Figure 28: Monochrome X-rite white card images from NIKON D200………43
Figure 29: Corrected monochrome X-rite white card images from NIKON D200…….43
Figure 30: Monochrome X-rite ColorChecker images from NIKON D200………44
Figure 31: Flatfield corrected X-rite ColorChecker images from NIKON D200……..…44
Figure 32: X-rite ColorChecker images from U I monochrome……….……45
Figure 33: X-rite ColorChecker dark images from U I monochrome……….………45
Figure 34: X-rite White card images from U I monochrome……….….………46
Figure 35: X-rite White card dark images from U I monochrome………..……46
Figure 36: Flatfield corrected X-rite ColorChecker images from U I monochrome……46
Figure 37: X-rite ColorChecker images from Canon 1000D………47
Figure 38: X-rite ColorChecker images from DMK monochrome………47
Figure 39: X-rite ColorChecker images from Nikon D3………48
Figure 40: Four sets of X-rite ColorChecker images from MuSIS system………48
Figure 41: X-rite ColorChecker images from LED based MSI system……….…..……49 Figure 42: X-rite ColorChecker images from LCTF based MSI system……….………49 Figure 43: Patch size equivalent pixel calculated
using imtool function from Matlab………50 Figure 44: Spectral reflectance of ground truth dataset for each colour patch…….……51 Figure 45- Figure 55: Spectral reflectance curve exacted from three different patch sizes of each system plotted against ground truth data……….……54-64 Figure 56: Comparison of RMSE of all systems in the visible range……….65 Figure 57: Comparison of RMSE of three systems in the NIR range……….………65 Figure 58 – Figure 73: CIELAB Colorimetric (Colour, lightness, Chroma and Hue) differences for each system considering three different patch sizes………66-81 Figure 74: Spectral reflectance comparison between ground truth and datasets from all devices in the wavelength range 420-660 nm with 1 nm interval ……….………83 Figure 75: Comparison of RMSE results from different systems in the
visible range for six colour patches………..……84 Figure 76: Comparison of RMSE results from three systems in the NIR range for six colour patches………..…84 Figure 77: Comparison of Colour difference of all imaging systems for six colour patches……….…85 Figure 78: Comparison of Chroma difference of all imaging systems for six colour patches……….…85 Figure 79: Comparison of Lightness difference of all imaging systems for six colour patches……….……86 Figure 80: Comparison of Hue difference of all imaging systems
for six colour patches………86
List of Tables
Table 1: Rule for the practical interpretation of ΔE* ab measuring the colour
difference………22
Table 2: Measurement geometry and data specifications of ground truth……….………29
Table 3: Information of spectral filtering used and spectral range for each system ……….………..39
Table 4: Information of data processing for each system………..….………39
Table 5: Information of data type and file size………40
Table 6: Data file format and no of bits………..…40
Table 7: Nikon D200 Mean intensity values for ColorChecker and White card…………41
Table 8: Nikon D200 Mean spectral reflectance values of X-rite white card measured using spectrophotometer……….……42
Table 9: List of all systems with different patch sizes 1mm, 8mm and 35mm equivalent pixel for all the systems………..……50
Table 10 – Table 19: RMSE results for all systems………54-64 Table 20- Table 35: Colorimetric differences results for all systems for patch size area 1mm………66-81 Table 36: Summary of the quality specifications of all imaging systems……….82
Table 37: Comparison of Mean color difference value for all systems………87
Table 38: Comparison of Mean spectral difference value in the visible range for all systems………..…………88
Table 39: Comparison of Mean spectral difference value in the NIR range for all systems………..……88
1. Introduction
The study and documentation of the artworks that are part of our cultural heritage is important for long-term preservation and ease of access for art historians, curators, conservators, and all those interested on the understanding of cultural heritage [1].
Whatever ones purpose, accuracy and high quality are very important features concerning the data that is acquired and saved [2]. These records have to be true representations that accurately document the required information, without anything adding to or taking away from the original artwork [3]. This is particularly important when coloured materials are concerned.
Colour is an essential language of cultural heritage that often needs more decoding than what was originally intended by the artist. Coloured materials are generally prone to changes, leading to alterations in the artworks visual appearance and, consequently the objects are interpreted differently from the artists’ intention. Often conservators need to go beyond what is seen, and to successfully conserve the artworks, they need to identify and document colours with as much accuracy as possible.
The use of non-contact high-resolution optical techniques such as Multi- and hyper- spectral imaging systems has led to improved colorimetric accuracy as they avoid colour measurement to a limited number of points on the surface of the object [4,5].
These techniques acquire images in different wavebands, enabling the reflectance spectrum to be estimated at each pixel of the image and thereby increasing the representativeness of the data and enabling a high-quality colorimetric reproduction [6,7,8,9]. Multispectral imaging generally refers to the number of spectral bands greater than 3 and less than 20, for which the bandwidths are between 10 and 50 nanometres. In turn, systems with hundreds of contiguous bands with bandwidths between 1 and 10 nanometres are considered hyperspectral [9]. All these devices generate a dataset with three dimensions (two spatial and one spectral) over which data is collected [10]. The acquired spectral reflectance of each pixel has several advantages such as: the possibility to reconstruct it in the CIE colour space with any choice of illuminant and of the colour matching functions; the possibility to monitor the conservation state of the object since a change in the spectral reflectance evidences the alterations of the material; and the possibility to reliably identify and discriminate the coloured materials used by the artists [11,12,13,14,15]
Therefore, to promote research, development and application of spectral imaging techniques towards the study and documentation of cultural heritage, European Cooperation in Science and Technology (COST) Action of Colour & Space in Cultural Heritage (COSCH) Working Group 1 (WG1) aims to identify and explore important characteristics of different spectral imaging systems and understand how they influence data accuracy and information reliability with respect to the various types of studied artworks[16]. As a matter of fact, several different types of devices have been developed to implement spectral imaging techniques for applications in cultural heritage [17]. The acquired information from different devices for the same object can be significantly influenced by the method of data collection, the system used, and any
other parameters influencing the general experimental setup such as illumination, camera, filters or dispersive gratings, calibration, file format, and metadata [18].
Therefore, WG1 initiated a Round Robin Test (RRT) of imaging five objects: X-rite Color Checker, X-rite White Balance Chart, Reference Panel SpectralonTM, Painted Test Panel – Medieval Tuscan Technique and Antique Russian Icon on Copper Plate.
Each of these RRT objects was imaged using different imaging devices developed by participating research groups as a coordinated research effort that aims to identify the impact of each instrumentation on the results obtained, and ensure the usefulness, accuracy and comparability of the data [19,20].
Colour being such an important issue, one of the objects that integrates the RRT is an X-Rite® ColorChecker Classic reference chart used as a colour reference. Constituted by 24 standard coloured patches, these charts allow to assess the accuracy of the colour reproduction processes of the systems used, to guarantee that the information obtained is valuable and represents the true colours of the object that has to be studied and documented.
In this thesis, the research objective was to contribute in the analysis of the COSCH RRT object: X-Rite® ColorChecker chart and X-Rite® White balance chart by evaluating spectral and colorimetric accuracy of eight multispectral imaging systems that has been developed by the research groups participating within RRT. The datasets were compared against the ground-truth reflectance dataset measured at University of Eastern Finland with Perkin-Elmer Lambda 18 Spectrophotometer based on the CIELAB metric – to account for the colour, lightness, chroma and hue differences and Root Mean Square Error – to account for the spectral difference.
References
[1] Bianchi, C.: Making online monuments more accessible through interface design.
In: MacDonald, L. (ed.) Digital Heritage – Applying Digital Imaging to Cultural Heritage, pp. 445–466. Butterworth-Heinemann (2006)
[2] Saunders, D.: High-quality Imaging at the National Gallery: Origins,
Implementation and Applications. Computers and the Humanities 31, 153–167 (1998) [3] Beckett, N. et al.: Imaging historical architectural sites for conservation. In:
MacDonald,L. (ed.) Digital Heritage – Applying Digital Imaging to Cultural Heritage, pp. 377–410. Butterworth-Heinemann (2006)
[4] Cucci, C. et al.: A hyper-spectral scanner for high quality image spectroscopy:
Digital documentation and spectroscopic characterization of polychrome surfaces. In:
ART 2011 -10th International Conference on Non-destructive Investigations and Microanalysis for theDiagnostics and Conservation of Cultural and Environmental Heritage (2011)
[5] Martinez, K. et al.: Ten years of art imaging research. In: Proceedings of the IEEE, vol.90(1), pp. 28–41 (2002)
[6] Saunders, D.: High-quality Imaging at the National Gallery: Origins,
Implementation and Applications. Computers and the Humanities, 31, 153-167 (1998) [7] Cucci, C. et al.: Extending Hyper-Spectral Imaging from Vis to NIR spectral regions: a novel scanner for the in-depth analysis of polychrome surfaces. In: Pezzati, L., Targowski, P. (eds.) Proc. of SPIE, Vol. 8790, O3A: Optics for Arts, Architecture, and Archaeology IV, pp. 879009-1- 879009-9 (2013)
[8] Liang, H.: Advances in Multispectral and Hyperspectral Imaging for Archaeology and Art Conservation. Applied Physics A, 106, 309-323 (2012)
[9] Martinez, K. et al.: Ten Years of Art Imaging Research. In: Proceedings of the IEEE, 90(1), 28-41 (2002)
[10] Hagen, N. et al.: Snapshot Advantage: a Review of the Light Collection Improvement for Parallel High-dimensional Measurement Systems. Optical Engineering, 51(11), 111702.1-111702.7 (2012)
[11] Antonioli, G. et al.: Spectrophotometric scanner for imaging of paintings and other works of art. In: Proceedings of CGIV 2004, pp. 219–224 (2004)
[12] Liang, H.: Advances in Multispectral and Hyperspectral Imaging for Archaeology and Art Conservation. Applied Physics A 106, 309–323 (2012)
[13] Colantoni, P.: Analysis of multispectral images of paintings. In: 14th European Signal Processing Conference (EUSIPCO 2006), Florence, Italy (2006)
[14] Burns, P.D., et al.: Error Propagation Analysis in Color Measurement and Imaging. Color Research and Application 22(4), 280–289 (1997)
[15] Kubik, M.: Hyperspectral imaging: A new technique for the non-invasive study of artworks. In: Creagh, D., Bradley, D. (eds.) Physical Techniques in the Study of Art, Archaeology and Cultural Heritage, pp. 199–259. Elsevier (2007)
[16] Boochs, F. et al.: Towards optimal spectral and spatial documentation of cultural heritage. COSCH – An interdisciplinary action in the cost framework. In: International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. XL-5/W2, pp. 109–113 (2013)
[17] Fischer, C., Kakoulli, I.: Multispectral and hyperspectral imaging technologies in conservation: current research and potential applications. Reviews in Conservation 7, 3–16(2006)
[18] Berns, R. S., Frey, F. S.: Direct Digital Capture of Cultural Heritage –
Benchmarking American Museum Practices and Defining Future Needs, Final Report.
Rochester Institute of Technology (2005)
[19] Saunders, D.: High-quality Imaging at the National Gallery: Origins,
Implementation and Applications. Computers and the Humanities, 31, 153-167 (1998) [20] Boochs, F. et al.: Towards Optimal Spectral and Spatial Documentation of
Cultural Heritage. COSCH – An Interdisciplinary Action in the Cost Framework. In:
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-5/W2, pp. 109-113 (2013)
2. Literature Review 2.1 COSCH
Colour and Space in Cultural Heritage (COSCH) is a network of experts in the latest optical measurement techniques and imaging systems used for the study and documentation of cultural heritage artefacts. As specified in the COSCH TD1201 COST Action Memorandum of Understanding, true, precise and complete documentation of artefacts is essential for conservation and preservation of cultural heritage. And so, based on an interdisciplinary European cooperation, COSCH aims to provide a novel, reliable, independent and global knowledge base, which facilitates the use of present and future optical measuring techniques to support the documentation of European heritage for its long-term preservation [21]. A brief overview of COSCH project can be seen in Fig. 1 below [22].
COST ACTION
Figure 1: COSCH project overview [22]
The COSCH activities have been organised among five working groups around five main subjects areas as given below:
• Working group 1: Spectral object documentation;
• Working group 2: Spatial object documentation;
• Working group 3: Algorithms and procedures;
• Working group 4: Analysis and restoration of cultural heritage surfaces and objects;
• Working group 5:Visualization of cultural heritage objects and its dissemination
Among these groups, the activities of COSCH Working Group 1 (WG1) have been focussed for the purpose of this thesis. The group’s primary aim is two-fold [23,24]:
• To identify, characterise and test 2-D spectral imaging techniques in the visible and near infrared (IR) field (task st1.1).
• To Identify, characterise and test imaging techniques beyond the visible and short wave radiation (task st1.2).
The above tasks have mostly been dedicated for the use of spectral imaging techniques in the investigation of mainly 2-D artworks, such as paintings. As a non-invasive technique, it offers an ideal approach for an in-depth examination of artworks. The state of the art techniques range from well-established methodologies such as: visible light photography/imaging techniques (normal, raking, specular, transmitted);
ultraviolet-induced fluorescence and reflected ultraviolet imaging; trans-illumination and trans-irradiation imaging techniques; infrared photography; infrared reflectography; infrared thermography; X-ray radiography; digital and computed X- ray radiography and tomography; neutron activation autoradiography; and photogrammetry [25-33] to cutting-edge applications like ultraviolet-visible; near- infrared and luminescence imaging spectroscopy; terahertz imaging; reflectance transformation imaging; and synchrotron imaging techniques [34-56]. These techniques could be grouped as multi-band, multi-spectral and hyperspectral methodologies depending on the number of bands selected over a given spectral interval and on their bandwidths.
For the documentation and study of artworks aimed at digital archives, accurate colour reproduction is one of the key requirements for imaging techniques. The analysis of chromatic and spectral characteristics is important, not only to make a reliable reproduction of the artwork that can be available at any time, but also to provide quantitative and objective information on the conservation state of the artwork. However, there are still no well-established and commonly accepted standards for a precise, non-contact study and documentation of artworks that could implement and combine the techniques mentioned above. Therefore, one of the COSCH Action tasks has been focused on creating and proposing recommendations for colour and spectroscopic measurements through the use of imaging systems that would provide the art conservation community with guidelines for the most common applications. Hence, the main objective of WG1 has been focused on the standardisation of the acquired imaging data and the calibration procedures to be followed with the diverse imaging systems. In particular, great emphasis has been placed on creating a well-defined and structured Round Robin Test (RRT) in order to compare colour and spectroscopic measurements, as well as information on calibrated standards and laboratory mock-ups obtained through the use of diverse imaging devices developed by the different research groups that have participated in the COSCH Action[23].
2.1.1 COSCH RRT
Spectral imaging techniques have been applied as an in situ method for the study and accurate digital documentation of artworks. An increasing number of devices have been developed for such purpose but depending on the system used, data accuracy and reliability may vary. For this reason, 2D object of different typologies were carefully selected as test samples to be used on a Round Robin Test (RRT) as shown in Fig.2.
This RRT exercise has been carried out by WG1, within a coordinated research effort that aims to identify the characteristics and performances of different spectral imaging instruments, and to standardise methodologies for the analysis of various types of artworks with those instruments, thereby ensuring the usefulness, accuracy and comparability of the data acquired. Different laboratories measured the same RRT 2D test samples and the results obtained by each group were compared. In each case, the working conditions and technical parameters were not predetermined, since the point was that each group uses the setup commonly applied in its laboratories. In addition, the individual setups were carefully documented by each group [23].
Round Robin Test 1 Round Robin Test 2 Round Robin Test3
Round Robin Test 4 Round Robin Test 5 Figure 2: RRT test objects 1 to 5
• Round Robin Test 1: The test chart X-Rite® ColorChecker Classic was provided by M.
Hauta-Kasari. The test target is a matt chart with a dimension of 280 mm x 216 mm and has been used in colour rendering to define the characteristics of each imaging device. It consists of twenty-four standard coloured square patches, each with 40 mm of side, that represents true colours of natural matter (such as skin, foliage and sky), additive and subtractive primary colours, various steps of grey, and black and white (Fig. 2). Different colours are arranged with a dimensional scale in a 4 by 6 array.
Hence, the chart has been used as a standard for colour reference to evaluate colour reproduction processes of different imaging systems. The information obtained would
guarantee valuable information for the representation of true colours of the object that has to be studied and documented.
• Round Robin Test 2: The test chart X-Rite® White Balance was provided by M.
Hauta-Kasari. It is a full-size version of the white reference square from the X-Rite®
ColorChecker Classic and presents a spectrally neutral and uniform surface under any lighting condition. It is used to define the level of homogeneity in illumination during the image acquisition and in detector elements.
• Round Robin Test 3: Labsphere Spectralon® Multi-Component Wavelength Calibration Standard was provided by A. Jung. It is impregnated with a combination of three rare earth oxides (holmium oxide, erbium oxide, dysprosium oxide). It is a stable and chemically inert reflectance standard, with ca. 90 mm of diameter and ca.
15 mm of height, and an opaque, homogenous and smooth cylindrical shaped surface.
Displaying sharp absorptions at specific wavelengths covering the UV-VIS-NIR region of the electromagnetic spectrum, it is commonly used for establishing the accuracy of the wavelength scale of reflectance spectrophotometers.
• Round Robin Test 4: Antique Russian icon (1899) printed on copper plate glued to a wooden support,(dimension: 26.5 cm x 22.0 cm) was provided by L. MacDonald. It represents the Virgin of Kazan and is marked “Moscow 1899”. From unknown provenance, the icon was purchased from Estonia. Its surface presents specular reflectance from pseudo- golden areas and the composition of the coloured materials is unknown. This object was chosen to stress the behaviour of the different imaging devices when highly reflective surfaces, such as the gold background, are investigated.
The icon was also selected to assess the systems’ spectral resolution for the identification and characterisation of the unknown coloured materials, and to establish the systems’ performance while documenting the object with high accuracy.
• Round Robin Test 5: A test panel painted with medieval Tuscan technique was provided by M. Picollo). It was produced to highlight the performances of the different imaging devices in penetrating the paint layers and detecting the under-drawings. This test object was also useful to stress the importance of having reference materials with known preparation, that help in the identification of artists’ materials through comparison with their spectral reflectance. The painted panel was approx. 12 cm x 29 cm dimensions. It was reconstructed by Elena Prandi and Marina Ginanni (Restoration Laboratories of the Soprintendenza SPSAE e per il Polo Museale della città di Firenze, Italy), in order to reproduce the medieval Tuscan technique as described by Cennino Cennini.
The datasets of X-rite ColorChecker chart and X-rite White chart were considered for this thesis. Previously for COSCH project, X-Rite® ColorChecker Classic acquired by two push-broom hyper-spectral systems (IFAC-CNR & IP-UEF) were analysed. These systems are in the same spectral range 400-1000 nm wavelength and collects data with a 2D array detector at all wavelengths simultaneously one for spatial line of the
object so that only one spatial dimension needs to be scanned to fill out the file-cube [58]. Although both systems were based on the same working principle, they were designed and optimized in different ways depending on the purpose of analysis of each group if they were seeking for high spatial or spectral resolution, or high colour accuracy). For the treatment of Vis-NIR range file-cube obtained, the reflectance spectrum of each patch was extracted from centered squares with 35 mm of side (400pixels x 400 pixels) to represent the average of each patch. In addition, reflectance curves were also extracted from five different small areas of squares with 1 mm of side (15pixels x 15pixels) for each patch in the middle and in the four corners.
This method was adopted to see uniformity of colour within each coloured patch.
Reflectance data were also extracted from centered square with 8 mm of side (90pixels x 90pixels) in order to resemble the common area of analysis of a contact colorimeter.
The spectra extracted from each coloured patch were then used to calculate the colorimetric values of the same areas using CIELAB system with the CIE illuminant D65 (natural daylight) and CIE 1964 100 standard observer [57]. The results obtained were satisfactory in terms of spectral and colorimetric accuracy for some colours, but showed differences at both ends of the visible range. This comparison allowed assessing the accuracy of colour reproduction processes performed by the two systems [59].
In this thesis, the spectral datasets were collected from eight multispectral imaging devices developed by different groups participating within RRT. Hence, this thesis focuses on the multispectral imaging that will be dealt in the next chapter.
References
[21] COSCH.: http://www.cost.eu/about_cost/mission,
[22] Frank Boochs.: Report on COSCH, A New COST Action Starts, EuroMed 2012, LNCS 7616, pp. 865–873, 2012.
[23] Marcello Picollo, Sérgio Nascimento.,2015 “Working Group 1: Spectral object documentation report”
http://cosch.info/documents/10179/145770/DRAFT_COSCH_interim+report_WG1.
pdf/587b2768-7ad0-4548-906b-8c152f948593
[24] COSCH website.: http://cosch.info/project
[25] Mairinger, F., 2004. UV-, IR- and X-ray- imaging. In: K. H. A. Janssens and R.
Grieken eds. Non-destructive microanalysis of cultural heritage materials. Antwerp:
Wilson & Wilson Elsevier, pp.15-73.
[26] van Asperen de Boer, J. R. J., 1968. Infrared Reflectography: a Method for the Examination of Paintings. Applied Optics, 7(9), pp. 1711-1714.
[27] Aldrovandi, A., Bertani, D., Cetica, M. and Matteini, M., 1988. Multispectral image processing of paintings. Studies in Conservation, 33, pp. 154-159.
[28] Schreiner, M., Frühmann, B., Jembrih-Simbürger, D. and Linke, R., 2004. X-rays in art and archaeology: An overview. Powder Diffr., 19(1), pp. 3-11.
[29] Saunders, D., Billinge, R., Cupitt, J., Atkinson, N. and Liang, H., 2006. A new camera for high-resolution infrared imaging of works of art. Studies in Conservation, 51, pp. 277-290.
[30] Cucci, C., Picollo M. and Vervat, M., 2012. Trans-illumination and trans-
irradiation with digital cameras: potentials and limits of two imaging techniques used for the diagnostic investigation of paintings. Journal of Cultural Heritage, 13, pp. 83–
88.
[31] Sayre, E. V. and Lechtman, H. N., 1968. Neutron Activation Autoradiography of Oil Paintings. Studies in Conservation, 13(4), pp. 161-185.
[32] Gavrilov, D., Maeva, E., Grube, O., Vodyanoy, I. and Maev, R., 2013.
Experimental comparative study of the applicability of infrared techniques for non- destructive evaluation of paintings. Journal of the American Institute for
Conservation, 52(1), pp. 48-60.
[33] Hackney, S. and Townsend, J., 1999. Methods of Examination and Analysis. In: S.
Hackney, R. Jones and J. Townsend, eds. Paint and Purpose – A study of technique in British Art. London: Tate Gallery Publishing, pp. 17-24.
[34] Fischer, C. and Kakoulli, I., 2006. Multispectral and hyperspectral imaging technologies in conservation: current research and potential applications. Reviews in Conservation, 7, pp.3-16.
[35] Kubik, M., 2007. Hyperspectral Imaging: A New Technique for the Non-Invasive Study of Art-works. In: D. Creagh and D. Bradley, eds. Physical Techniques in the Study of Art, Archaeology and Cultural Heritage. Amsterdam: Elsevier, pp.199-259.
[36] Picollo, M., Bacci, M., Casini, A., Lotti, F., Poggesi, M. and Stefani, L., 2007.
Hyperspectral image spectroscopy: a 2D approach to the investigation of polychrome surfaces. In: J. Townsend, L. Toniolo and F. Capitelli, eds. Proceedings of
Conservation Science 2007. London: Archetype Publications, pp. 162-168.
[37] Daffara, C., Pampaloni, E., Pezzati, L., Barucci, M. and Fontana, R., 2010.
Scanning Multispectral IR Reflectography SMIRR: An Advanced Tool for Art Diagnostics. Accounts of Chemical Research, 43(6), pp. 847-856.
[38] Delaney, J.K., Zeibel, J.G., Thoury, M., Littleton, R., Palmer, M., Morales, K.M., de la Rie, E.R. and Hoenigswald, A., 2010. Visible and Infrared Imaging Spectroscopy of Picasso’s Harlequin Musician: Mapping and Identification of Artist Materials in Situ. Applied Spectroscopy, 64(6), pp.584-594.
[39] Cucci, C., Casini, A., Picollo, M., Poggesi, M. and Stefani, L., 2011. Open issues in hyperspectral imaging for diagnostics on paintings: when high spectral and spatial resolution turns into data redundancy. In: R. Salimbeni and L. Pezzati, eds.
Proceedings of SPIE. O3A: Optics for Arts, Architecture, and Archaeology III. Vol.
8084, pp. 80848.1-10.
[40] Liang, H., 2012. Advances in Multispectral and Hyperspectral Imaging for Archaeology and Art Conservation. Applied Physics A, 106, pp.309-323.
[41] Ricciardi, P., Delaney, J. K., Facini, M., Zeibel, J. G., Picollo, M., Lomax, S. and Loew, M., 2012. Near Infrared Reflectance Imaging Spectroscopy to Map Paint Binders In Situ on Illuminated Manuscripts. Angew. Chem. Int. Ed., 51, pp. 5607- 5610.
[42] Köhler, W., Panzer, M., Klotzach, U., Winner, S., Helm, M., Rutz, F., Jördens, C., Koch, M. and Leitner, H., 2006. Non-destructive investigation of paintings with THz- radiation. Proc. European Conference of Non-Destructive Testing, pp. 181.
[43] Fukunaga, K., Ogawa, Y., Hayashi, S. and Hosako, I., 2007. Terahertz
Spectroscopy for Art Conservation, IEICE Electronics Express, 4 (8), pp. 258-263.
[44] Hosako, I., Sekine, N., Patrashin, M., Saito, S., Fukunaga, K., Kasai, Y., Baron, P., Seta, T., Mendrok, J., Ochiai, S. and Yasuda, H., 2007. At the Dawn of a New Era in Terahertz Technology. Proc. IEEE, 95, pp. 1611-1623.
[45] Jackson, J. B., Mourou, M., Whitaker, J. F., Duling III, I. N., Williamson, S. L., Menu, M. and Mourou, G. A., 2008. Terahertz imaging for non-destructive evaluation of mural paintings. Optics Communications, 281, pp. 527-532.
[46] Adam, A. J. L., Planken, P. C. M., Meloni, S., Dik, J., 2009. Terahertz imaging of hidden paint layers on canvas. Optics Express, 17, pp. 3407-3416.
[47] Labaune, J., Mourou, G., Duling, I., Lavier, C. and Menu, M., 2009. Terahertz metrology of tree rings for dendrochronology and cultural heritage applications, In: R.
Salimbeni and L. Pezzati, eds. Proceedings of SPIE. O3A: Optics for Arts, Architecture, and Archaeology II. Vol. 7391. DOI: 10.1117/12.827508.
[48] Jackson, J. B., Mourou, M., Labaune, J., Whitaker, J. F., Duling III, I. N., Williamson, S. L., Lavier, C., Menu, M. and Mourou, G. A., 2009. Terahertz pulse imaging for tree-ring analysis: a preliminary study for dendrochronology applications.
Meas. Sci. Technol., 20, pp. 866-868.
[49] Abraham, E., Younus, A., Delagnes, J. C. and Mounaix, P., 2010. Non-invasive investigation of art paintings by terahertz imaging. Appl. Phys. A, 100, pp. 585-590.
[50] Labaune, J., Jackson, J. B., Pagès-Camagna, S., Duling, I. N., Menu, M. and Mourou, G. A., 2010. Papyrus imaging with terahertz time domain spectroscopy. Appl.
Phys. A, 100, pp. 607-612.
[51] Fukunaga, K. and Picollo, M., 2010. Terahertz spectroscopy applied to the analysis of artists’ materials. Appl. Phys. A, 100, pp. 591-597.
[52] Fukunaga, K. and Picollo, M., 2012. Characterisation of Works of Art. In: K. E.
Peiponen, A. Zeitler and M. Kuwata-Gonokami eds. Terahertz Spectroscopy and Imaging, Springer Series in Optical Sciences, Vol. 171. Berlin Heidelberg: Springer- Verlag, pp. 521- 538.
[53] Picollo M., Fukunaga K. and Labaune J., 2015. Obtaining noninvasive stratigraphic details of panel paintings using terahertz time domain spectroscopy imaging system. Journal of Cultural Heritage, 16, pp. 73-80. DOI:
10.1016/j.culher.2014.01.006.
[54] Delaney, J. K., Ricciardi, P., Glinsman, L. D., Facini, M., Thoury, M., Palmer, M.
and de la Rie, E. R., 2014. Use of imaging spectroscopy, fiber optic reflectance spectroscopy, and X-ray fluorescence to map and identify pigments in illuminated manuscripts. Studies in Conservation, 59(2), pp. 91-101.
[55] Bertrand, L., Robinet, L., Thoury, M., Janssens, K., Cohen, S. X. and Schöder, S., 2012. Cultural heritage and archaeology materials studied by synchrotron
spectroscopy and imaging. Applied Physics A. 106, pp. 377-396.
[56] Reflectance Transformation Imaging (RTI)
http://www.si.edu/MCIImagingStudio/RTI (19 April 2015)
[57] Sharma, G., et al.: The CIEDE2000 Color-Difference Formula: Implementation Notes,
Supplementary Test Data, and Mathematical Observations. Color Research and Application
30(1), 21–30 (2005)
[58] 13. Hagen, N. et al.: Snapshot Advantage: A Review of the Light Collection Improvement for Parallel High-dimensional Measurement Systems. Optical Engineering 51(11), 111702.
1–111702.7 (2012)
[59] Tatiana Vitorino, Andrea Casini , Costanza Cucci , Ana Gebejes, Jouni Hiltunen , Markku Hauta-Kasari , Marcello Picollo , Lorenzo Stefani.: “Accuracy in Colour Reproduction: Using a ColorChecker Chart to Assess the Usefulness and
Comparability of Data Acquired with Two Hyper-Spectral Systems”, CCIW 2015, LNCS 9016, pp. 225–235 (2015)
2.2 Multispectral Imaging
A multispectral image is a set of monochrome images collected by measuring a series of wavelengths over a broad range of the light spectrum. Images are normally acquired at visible wavelengths (400–700 nm), and may also include regions of ultraviolet (<400 nm) and infrared (>700 nm)[Fig.3] [60]. Each image records the intensity of radiation from the scene in the corresponding waveband [61]. This technique was originally developed to identify geographic resources that did not require high spatial image resolution [62].
Figure 3. Multispectral images are captured in a similar process as colour images. A colour image is a combination of three separate images captured at selected ranges of the visible spectrum representing blue, red and green tones. A multispectral image is captured is a combination of a series of images captured at discrete narrow ranges of the light spectrum.
[60]
From the early 1990s, multi-spectral imaging was applied in the field of cultural heritage. It was used for qualitative band-to-band comparison that identified areas of different material composition, natural degradation of material, past conservation intervention, preparatory sketches and quantitatively for improved precision in colour measurement. [63]
In essence, a multispectral image provides information about the physical characteristics of the sample, independent from the environment and observer that can be targeted to any desired description specific for a given observer and viewing conditions. Being device independent, a multispectral image is invariant across different acquisition devices, allowing comparison of sample whose images are taken from different devices. However in general the reflective properties of a surface is dependent from the geometry of the illumination and it is assumed that the illumination and acquisition geometry are controlled, as it would happen if the sample surface were measured by a spectrophotometer. The color information captured with RGB device cannot generate a fully accurate colorimetric representation due to the fact that the sensitivities of the sensor employed do not correspond to those of the standard colorimetric observer [64]. If a multispectral image is available, precise colorimetric coordinates can be computed for each pixel in the image. For an accurate color measurement, the acquisition and reproduction of color are mainly two fundamental processes[65]
2.2.1 Narrow and wide band image capture:
Two different approaches exist for multispectral imaging: narrow-band and wideband image capture [66]. They differentiate in the way they sample the wavelengths of the visible spectrum.
Figure 4: An example set of 4-band filters for narrow and wide band-pass filters’
spectral transmittances.
In the narrow-band approach, the acquisition of radiance information is obtained by a set of narrow-band filters, centered in principle one for each wavelength sample.
There are several different technologies available to produce spectrally narrow filters.
One way is to adapt a filter wheel with narrow bandpass glass filters in front of a camera. But these filters are normally costly custom-made and a filter wheel comes with several electro-mechanical drawbacks such as slow band switching, small number of filters, sequential access to color bands, cumbersome design, and limited versatility.
Therefore, narrow-band systems using a tunable filter is realised to be more convenient to use due to the fact that spectral transmission of this device can be electronically controlled through the application of voltage or acoustic signal. Tunable filters can provide finer spectral sampling, and rapid and random switching between color bands because there are no moving parts and no discontinuity in the spectral transmission range [67].
In particular a solid state Liquid Crystal Tunable Filter (LCTF) has been widely used [68-71]. The peak wavelengths of the LCTF can be controlled and records a fine spectral sampling and producing usually thirty-one peaks in the range from 400 to 720 nm [72]. One of the most important advantages of this system is its robustness to arbitrary spectral shapes. In fact a sampling rate of 10 nm (in the ideal case of infinitesimally wide band filters) permits one to reconstruct spectral features that are at least as wide as twice the sampling rate [66]. It has been accepted that LCTF is free from spectral non-uniformities introduced from the angular sensitivities, which are associated with alternative technologies such as interference filters [73]. The LCTF has the advantage of being solid state and reliably repeatable, and can be easily controlled by a computer for an efficient, automated, and relatively fast imaging. On the other hand, a large storage space is required for each acquired target, and registration of the thirty-one images is a serious issue. Moreover this system has severe drawbacks in terms of size, costs, and unwieldiness.
Unlike LCTF systems, the LED (Light emitting diodes) is based on active illumination that captures images by illuminating the scene with a set of different narrow band color LED light sources. The whole process can be controlled electronically, and speed can be increased further by three times using an RGB camera (RGB-LEDMSI) [74]
instead of a monochrome camera (Mono-LEDMSI) [75,76]. Therefore, it is considered a faster approach for multispectral imaging. However, the system can be considered a constrained case if it is limited with the number of LEDs.
In the wide band approach, the visible spectrum is sampled at a wide step and each adopted sensor is sensitive to light energy in a sufficiently large wavelength interval.
With respect to narrow band systems, they are more easily deployed, manageable, flexible in their use, and comparatively cheaper. Different works have demonstrated that five to eight basis vectors are sufficient enough to obtain an accurate spectral reconstruction [74, 77-81]. Therefore, this approach has the advantage of reducing the number of filters (from thirty-one adopted in the narrow-band approach) and still recovering accurate spectral reflectances of the target. However, data obtained from wide band approach are not a direct measurement of reflectance and therefore these data needs to be further processed in order to attain the true multispectral image [82].
In general, a multispectral acquisition system consists of a multispectral camera, a processing module to derive reflectance from the acquired radiance images, and a transformation module for the conversion into a colorimetric space. A number of different multispectral acquisition systems have been developed and tested. They differ among themselves on the basis of number of sensors employed. Normally, a multispectral camera is a standard monochrome digital camera and a set of coloured filters. In a typical wide-band system, optical filters are used to simulate sensors of different sensitivity either by traditional filters like those used in standard photography or a tunable filter can be adopted. Burns and Berns [83] used a monochrome digital camera placed with seven interference filters while Imai et al.
[84] combined a monochrome camera with a filter wheel containing six absorption filters. Methods have also been proposed that use commonly available optical filters and trichromatic digital still cameras [85]. Imai et al. [86] adopted a conventional trichromatic digital camera combined with a series of absorption filters. Seven filter combinations were placed in front of the digital RGB color CCD camera. One was no filter at all, while the remaining six were combinations of Wratten filters. A system based on a commercial color-filter array (CFA) digital camera coupled with a two- position filter slider containing absorption filters has been adopted to facilitate multispectral imaging for imaging cultural heritage [87-90]. For each target, two RGB images are taken through each filter, so there are in total six channels for this camera system. The acquired images are usually affected by some form of hardware noise, such noise must be estimated and modeled, so that a noise correction procedure can be established and applied. If the illumination is not evenly distributed across the scene, its uneven effect must then be discounted; this can be done by acquiring a reference image to estimate the effect and correct it.
2.2.2. Calibration procedure:
The method of calibrating multispectral devices involves the normal procedure of calibrating CCD images (e.g. [91]) as well as the calibration of the spectral response of the imaging system. The acquisition procedure needs to be carried out in a fixed environment that includes using the same illuminant source and illumination geometry. The information retrieved from the acquisition of a representative target may be extended to other acquisitions performed under the same conditions [63].
Generally, thermal radiators and electric discharge lamps are used as an illuminant source. Widely known thermal radiators are Tungsten and Tungsten-Halogen lamps.
Examples of electric discharge lamps are Mercury Vapour lamps, Xenon bulbs, Fluorescent lamps or Flashtubes. In the case of discharge lamps, the spectral power distribution (SPD) of illuminant source is not continuous and so this can cause problems during the spectral reconstruction process. Whereas, halogen lamps have continuous shaped SPD, which is one of the reasons why it is the most commonly used light sources [92].
Thermal noise associated with CCD detectors includes dark noise and readout noise [91]. Using a cooled detector can reduce the dark current and the noise associated with it. The readout noise is also determined by the speed of the readout. The shot noise or photon noise is determined by the Poisson statistics of the quantum arrival of photons and is given by the square root of the number of photoelectrons detected. While CCD detectors are highly linear, non-linearity is still observed close to saturation and at very low exposure times. Optimum exposure time for maximum signal-to-noise is determined by the maximum counts over which the CCD is linear. For best quality images and spectral reflectance measurements, exposure times should be adjusted per channel for maximum signal-to-noise ratio images. It is best to capture images after the light has been switched on for 10-20 minutes for a stable intensity of illumination and to avoid temperature gradients along the imaging pathlength, which can degrade the spatial resolution of the system which is similar to the ’seeing’ effect in astronomy [63].
The measured spectral reflectance of an object depends on the geometry of the illumination/collection setup, which determines the relative amount of surface reflection to that of the diffuse reflection. For comparison, it is best to image the objects in the same setup as the reference [63]. This is particularly important for systems with moderate spectral resolution. The measurement geometry recommended by CIE (Commission Internationale de l’Eclairage) for reflective measurements within the graphic arts industry is 45°/0° denoting 45° angle of incident illumination with the detector normal to the surface. The geometry is intended to reduce the effect of specular reflection and to represent typical viewing conditions. The disadvantage is that the result is dependent on the structure of the surface topology because of the directed illumination. For the colour measurement within paper industry, d/0°
geometry is recommended for diffuse illumination and measurement from surface
normal. An integrating sphere illuminates the sample diffusely. This angle is also known and noted as (d/α), e.g. (d/8) for 8o angle. The sample is placed against an opening in the sphere, and the illumination is arranged as such that neither the sample nor the detector is directly illuminated so that only diffuse illumination strikes the sample, and no light from the illuminant reaches the detector directly [93,94].
Normally calibration method involves taking the following calibration images [63]:
• Dark images of the same integration time as the object frames but with the lens cap on (or illumination off) to correct for the accumulated thermal dark current;
• Flatfield images per channel of a uniform white target to correct for the spatial inhomogeneity of illumination and pixel-to-pixel gain variation of the CCD;
• Images of a spectral standard through each wavelength channel to correct for the spectral response of the imaging system.
The reflectance at a pixel (the ith pixel) captured with a spectral channel of central wavelength λ is given by [63]:
Ri(λ) = RW (λ)g (Ii(λ) − Di(λ))fi(λ) (1) Σn[(Wi(λ) − DWi(λ))fi(λ)]/n
where,
• RW is the true spectral reflectance of a spectral standard,
• Ii is the counts for the light reflected from the object,
• Di is the dark counts for the same integration time as the object frame,
• fi is the flatfield correction factor for that pixel,
• Wi is the counts for the spectral standard and
• DWi is the corresponding dark counts for the same integration time as the standard,
• g is the scale factor to adjust the integration time of the object to that of the spectral standard n is the number of pixel over which to average the response of the spectral standard.
2.2.3. Imaging and Signal Processing:
Generally, the acquisition performed by a given i-th sensor at a single 2-D point x will return camera response value, ai(x) in the form [95]:
(2.1)
This value is obtained by integrating contributions from the spectral radiance of the illuminant E that reaches the physical sample observed, the spectral reflectance R of the sample, and the spectral sensitivity Si of the i-th sensor. The integration is performed in the wavelength range λ1 (minimum) to λ2 (maximum) of the sensor’s sensitivity. If this range exceeds than the visible light spectrum, appropriate steps is taken to cut off unwanted radiation. Currently, narrow-band and wide-band approaches are used in multispectral imaging in order to obtain the spectral reflectance estimation.
“Narrow-Band” Multispectral Imaging:
In narrow-band systems, the sensors of the device are sensitive to a very narrow wavelength interval or the light sources used shows a very narrow spectrum emission.
In both cases, the selective property of the system can be modeled as a delta function.
The camera response value ai(x) at a single point x can be interpreted as the value of function E(λ, x)R(λ, x)S(λ) at the specific wavelength λi. Thus, by changing sensors or light sources, different values of this function can be estimated on the whole visible light spectrum. For a given wavelength λi, Equation 2.1 then becomes:
(2.2)
If the values of illuminant E(λi) and sensor(s) S(λi, x) are known then reflectance value R(λi, x) of the sample can be computed. Alternatively, the values ai(x) can be compared with the corresponding values previously obtained from the acquisition of a reference physical sample whose reflectance is known [44]. If the result of this previous acquisition is indicated with ai (x), then it is:
(2.3)
where, R(λi, x) is the known reflectance of the reference sample. The value of R(λi, x) can then be computed using the following equation:
(2.4)
“Wide-Band” Multispectral Imaging:
In wide-band systems, the device is based on wide-band sensors. Each sensor is sensitive to a large wavelength interval of light energy. The emission of light source is considered to have sufficient broad spectrum. In this case, the output values a(x) obtained do not associate to a specific wavelengths as they do not allow direct measurement of reflectance [96]. This type of system requires a correlation based method learned from a suitable training set to relate the output values obtained at each pixel with the spectral reflectance of the corresponding surface point in the scene.
The output values may be indicated as:
(2.5)
where, index i varies with the filter used and x represents a 2-D vector containing identified point considered within the acquired scene. The value a(x) is an M- dimensional vector if M filters are used. The reflectance of the target at point x is a function of the wavelength λ, and can be denoted as r (λ, x). However, in practice to give an analytical form to r, sampling its value at a discrete number of λ, is considered mandatory. The reflectance is expressed as:
(2.6)
where, index j varies with the sample wavelengths. The value r(λ, x) is an N- dimensional vector if N sample values of λ are considered. In order to correlate output values with the corresponding reflectance, the characterization function of the system requires to be estimated:
(2.7) The discrete form of Equation 7.1 for a given wavelength λi, becomes:
(2.8)
To form a linear system, if M number of sensors (filters) are used then the system can be denoted as:
a = Dr , (2.9) with:
(2.10)
If matrix D is known, then Equation 2.9 could be solved with respect to r by means of an inversion technique that belongs to a direct reconstruction approach. This method simply inverts Equation 2.9 by using a pseudoinverse approach or ordinary least squares regression. However, this method adopted by Tominaga [97] is not well applied in practice because this solution is sensitive to noise [98]. Herzog et al. [99]
has proposed a weighted least squares regression based on a weighting matrix to invert the system characteristics under a smoothness constraint. Hardeberg [98] has proposed a method based on a priori knowledge of a spectral reflectance database, without consideration of camera noise.
A direct reconstruction approach is not widely used, as it requires spectral characterization of the entire imaging system. In addition estimation of illuminant E and sensitivity Si is not easy, and complex illumination geometry would require additional computational cost. There are other approaches such as empirical linear model based on dimension reduction techniques that analyses reflectance spectra in the frequency domain using Principal Component Analysis (PCA) [100] and Independent Component Analysis (ICA) [101] to estimate the relationship between the acquisition output and the sampled reflectance function. Learning-based approach do not require prior knowledge of the spectral characteristics of the imaging system but is greatly affected by the choice of a calibration target such as the Macbeth ColorChecker and ColorChecker DC [102] are available.
Interpolation method for spectral reconstruction simply assumes that a multispectral system is sampling spectral reflectance curves. Instead of using delta Dirac functions for the sampling as in the classical framework, the spectral transmittance functions of filters are consider to be the sampling functions. It just requires the camera response
itself. But before representing camera response to spectral curve, two problems should be considered:
• The position of the samples in the spectral range. For narrow filters, this uncertainty can be low but for wide filters this uncertainty increases and therefore, interpolation methods are used only with multispectral devices using narrow band pass filters.
• The camera response values must be normalized [0,1] for all channels. This normalization matrix is obtained taking into account the responses of the system to the spectral curve of a “perfect white”. Normally, a standard white target is imaged as a reference for normalisation as part of a calibration procedure. Interpolation without normalisation would provide bad results.
Therefore, a good calibration of the system is necessary to apply this technique.
Existing interpolation techniques include cubic splines for the representation and reconstruction of spectral reflectance curves because they are piecewise polynomials of degree three, cubic polynomials, with pieces smoothly connected together to generate smooth curves [92].
2.2.4. Transformation to color space
The CIE 1931 XYZ colour coordinate system is related to the spectral sensitivity of human vision by the use of color matching functions, which match to the CIE 1931 Standard Observer. The X, Y, and Z tristimulus values are calculated by integrating the product of the spectral reflectance r(λ), the illuminant l(λ), and the corresponding 𝑥, 𝑦, 𝑧 colour matching functions for the standard 2 degree observer [103] normally from 380( λmin ) to 760nm (λmax):
(3.1)
Even though tristimulus values define absolute color of an object, but they are considered to be perceptually non-uniform colour space. They do not define eye’s response to the color that depends on the environment and adaptation of the eye. The color spaces XYZ (or RGB) are linearly related to the spectrum of the coloured light whereas, sensitivity of human eye is not linear. Therefore, changing the tristimulus values of XYZ (or RGB) for a color stimulus, the observer will perceive a difference in color for differences greater than the Just Noticeable Difference (JND). In both spaces the JND depends on the location in the color space. Hence, CIELAB space was proposed by the CIE in 1976 that aimed at making JND constant and led to a uniform color space where the JND do not depend on the location. But in practice this condition can only be fulfilled approximately. Thus the term pseudo-uniform for CIELAB is used [104]. Since the idea of JND is observer-dependent and is a resultant from psychophysical experiments, therefore this makes CIELAB a psychometric color space. The CIELAB pseudo-uniform color space is defined by the quantities L*, a* and
b*. They are calculated from X, Y, Z using:
(3.2)
where, the tristimulus values Xn, Yn and Zn are those of the white stimulus.
For a given illuminant l(λ), they are defined as:
(3.3)
In the CIELAB space, L* represents the lightness of a color and it is known as the CIE
1976 psychometric lightness. The scale of L* is 0 to 100, 0 represents the ideal black, and 100 represents the reference white. The chromaticity of a color can be represented in a two-dimensional (a*, b*) diagram, a* representing the degree of green versus red, and b* the degree of blue versus yellow.
Figure 5: Interpretation of CIELAB colour space [105].
C*ab denotes chroma and h*ab denotes hue. (3.4)
(3.5) The hue angle is measured in degrees starting with h*ab=0 in the +a* axis direction
and increasing counterclockwise.
That is, (3.6)
