3.2 Experiments
3.2.1 Blind denoising
We further employ the estimated PSD for blind denoising of correlated noise.
As the denoiser, we use BM3D with the ’refilter’ profile identical to the experi-ments of Publication I. Due to the simplicity of the estimation procedure, and in particular the way the patch-matching uses the crude prior image estimate, we do not expect the algorithm to be directly deployable to real denoising.
Nevertheless, we compare our algorithm against two blind denoisers for corre-lated noise: blind BLS-GSM (Portilla 2004a; Portilla 2004b), and Noise Clinic (Buades et al. 2011).
PSNR values for the experiment results are reported in Table 3.1 and vi-sualized in Figure 3.3. Examples of estimated noise PSDs and the denoising results for the tested methods are shown in Figure 3.4, Figure 3.5, Figure 3.6, and Figure 3.7. We note that although not shown, we also computed SSIM (Wang et al. 2004) values for each experiment; ordering of the methods
ob-0.0010 0.01 0.02
Figure 3.2 Average error of PSD estimates by the proposed method as well as the patch-based PSD estimation over 10 noise realizations for each of Barbara, Lena, andMan(each512×512px) corrupted by noise defined by kernels shown in Figure 2.1, as well as the four extended kernels g5, g6, g7, g8defined in Eq. 4 of Publication I. Each line reports the average mean squared error normalized by the corresponding noise variance var{η}and the squared image size; the shaded areas visualize the corresponding standard deviations of the errors.
The proposed method enables more adaptive estimation of the PSD, but may also contain more noise in the PSD estimate due to the larger amount of utilized sample coefficients and hence also smaller individual sample size.
tained by these values is very similar to that by the PSNR.
Of the compared methods, BLS-GSM offers very good feature preservation with the strongly structured kernelsg1 andg3, consistent with the strengths of the non-blind BLS-GSM in Publication I. It also performs well with low noise variance on most kernels, but overall tends to leave some visible noise in the denoised images, as can be noted in Figure 3.6 and Figure 3.7. Noise Clinic, although prone to high-frequency artifacts, offers competitive performance with the pink noise g4 as well as low levels of white noise; however, it fails almost completely in removing the more structured noise types, particularly with high noise variance. BM3D denoising with the patch-based PSD yields competitive PSNR values with several noise kernels, but the visual quality tends to be worse due to visible artifacts. Examples of such cases can be seen in Figure 3.4 and Figure 3.6.
In comparison, the proposed noise estimation procedure achieves consistent results across all tested noise cases. In particular, BM3D with the collabo-ratively estimated PSD manages to remove almost all noise with moderate smoothing, and does not generate extra noise artifacts with any tested kernel.
As such, it is suitable for filtering even within a fully blind denoising setup. We further demonstrate the denoising performance of the proposed procedure with various image sizes in Figure 3.8, showing that although the denoising quality can be improved somewhat by increasing image size, there are no drastic dif-ferences even when using slightly smaller images than those used in Table 3.1.
Although the estimation tends to under-estimate the lowest noise frequen-cies due to both limited support of the groups, and due to the aggressive weighting to avoid signal components, it does not significantly impact BM3D denoising due to the similarly small processing neighborhood of BM3D. This can be seen particularly in Figure 3.6, where, although the PSD is missing the highest peak values around the DC, the denoising quality is very similar to that with the exact PSD. More thorough estimation and denoising of the low noise frequencies could then be achieved with multiscale application of the estima-tion and BM3D denoising. BM3D also operates with a downscaled version of the PSD, meaning that apart from possible refiltering of a very detailed PSD, there is very little quality reduction from the limited pixel size of the estimated PSD.
Although BM3D utilizes group variances similar to those that are estimated through (3.5), direct use of the variances to BM3D denoising is not feasible.
The estimation covers only a limited number of group configurations, whereas the BM3D groups can have a different shape for each reference patch, leading to unique variances for each reference location. As such, intermediate estimation of the PSD is both necessary and an efficient way to compute the required group variances.
Since the denoised image of the based estimate is utilized for patch-matching in the collaborative estimator, significant misestimation in this step can also cause inaccuracies in the collaborative estimate. Hence, eliminating this step through other strategies for improved patch-matching and and allow-ing for more sophisticated and selective group buildallow-ing for computation of the sample variance would be likely to significantly improve estimation quality. It may also be possible to further improve the estimation by performing the de-noising in a fully iterative fashion, repeating the collaborative estimator based on patch positions computed from the previous estimate. Some observable misestimations of the PSDs (e.g. oscillations across the PSD) can further be assumed to follow from the simple minimum-norm estimate and the simplified resizing through zero-padding, and could be potentially negated through better modeling of these steps.
g ∥g∥22 noisy
non-blind blind
(collaborative) (patch)
new BM3D new BM3D new BM3D BLS-GSM Noise (refilter) (refilter) (refilter) Clinic gw
0.001 30.00 35.80 33.56 33.24 33.74 35.37
0.01 20.00 30.55 29.68 29.25 28.86 28.68
0.02 16.99 28.97 28.29 28.00 27.22 26.22
g1
0.001 29.99 37.17 34.90 34.32 34.92 32.95
0.01 19.99 29.39 28.72 29.19 29.28 22.98
0.02 16.98 26.82 26.27 26.83 27.44 19.77
g2
0.001 30.00 36.96 33.94 33.46 33.43 34.02
0.01 20.00 31.62 29.07 28.84 28.87 26.30
0.02 16.99 30.06 27.54 27.45 27.76 23.66
g3
0.001 30.12 41.92 35.26 33.07 36.29 31.26
0.01 20.12 40.71 31.89 28.58 33.37 21.04
0.02 17.11 39.86 30.29 27.38 32.60 17.92
g4
0.001 29.99 34.33 32.57 32.37 33.05 34.27
0.01 19.99 27.73 27.26 27.11 26.81 27.51
0.02 16.98 25.66 25.26 25.40 24.85 25.34
g5
0.001 30.00 35.88 33.99 33.92 34.47 33.16
0.01 20.00 28.79 28.27 27.87 28.40 23.57
0.02 16.99 26.42 26.15 25.26 26.52 20.40
g6
0.001 30.00 35.29 33.31 33.13 33.48 34.13
0.01 20.00 29.45 28.53 28.22 28.03 27.13
0.02 16.99 27.89 27.16 26.83 26.68 24.79
g7
0.001 30.09 37.45 34.60 33.11 34.89 31.85
0.01 20.09 31.98 30.80 27.94 30.35 21.87
0.02 17.08 30.39 29.70 26.81 29.45 18.79
g8
0.001 29.99 34.57 32.80 32.47 33.03 34.47
0.01 19.99 28.14 27.62 27.45 27.12 27.79
0.02 16.98 26.14 25.72 25.78 25.15 25.61
Table 3.1 Average PSNR of denoising for the set of noise kernels, images, and noise vari-ance described in Figure 3.2 for BM3D, ”collaborative” blind BM3D, ”patch”
blind BM3D, blind BLS-GSM, and blind Noise Clinic. The ”non-blind” appli-cation of BM3D is supplied with the corresponding Ψ; all blind applications are supplied only the noisy imagez. The ”blind (patch)” estimates the PSD through local patch spectra, and ”blind (collaborative)” estimates the PSD as described in Chapter 3.
blind BM3D (collaborative) blind BM3D (patch)
blind BLS-GSM Noise Clinic (blind)
−2
−1 0
var{η},gw
differenceinPSNR(dB)
−8
−6
−4
−2 0
var{η},g1
−4
−2 0
var{η},g2
−15
−10
−5 0
var{η},g3
differenceinPSNR(dB)
−2
−1 0
var{η},g4
−6
−4
−2 0
var{η},g5
0.001 0.01 0.02
−2
−1 0
var{η},g6
differenceinPSNR(dB)
0.001 0.01 0.02
−10
−5 0
var{η},g7
0.001 0.01 0.02
−2
−1 0
var{η},g8
Figure 3.3 Visualization of the average PSNR values for each noise kernel g, variance var{η}, and blind denoising method. On y-axis, relative PSNR to the best-performing method, i.e., the best method of the particular kernel and vari-ance combination is at zero. The shaded areas visualize the corresponding standard deviation. On x-axis, the three tested noise variances for each ker-nel. Although the absolute differences are much larger for var{η}= 0.001, the visual difference is in most cases minimal due to the overall high PSNR values.
PSD ofg1
Estimated PSD (patch)
Estimated PSD (collaborative)
Man noisy
(19.97dB)
new BM3D (refilter) non-blind (28.81dB)
new BM3D (refilter) blind, collaborative
(28.12dB)
new BM3D (refilter) blind, patch
(28.40dB)
BLS-GSM blind (28.48dB)
Noise Clinic blind (22.39dB)
Figure 3.4 PSD estimation and denoising of Man corrupted by g1 with ∥g∥22 = 0.01.
On top row, ground-truth PSD and visualizations of the patch PSD estima-tion and the collaborative estimate. On middle row, views of the noise-free image, noisy image, and denoising of Man with the exact PSD, and the
”collaborative” estimation described in Chapter 3. On bottom, views of de-noising results for BM3D with the ”patch” estimate, blind BLS-GSM, and Noise Clinic. All BM3D results are computed with new BM3D ”refilter”
profile. The PSDs are estimated from the full image. Both blind BM3D estimates perform well on this kernel, although the patch-based estimate contains faint horizontal artifacts despite its high PSNR. BLS-GSM offers similar performance, with better attenuation of low-frequency noise, but also some residual noise visible on the hair; Noise Clinic fails to remove the noise.
PSD ofg3 Estimated PSD (patch)
Estimated PSD (collaborative)
Lena noisy
(20.12dB) new BM3D (refilter) non-blind (40.68dB)
new BM3D (refilter) blind, collaborative
(33.09dB)
new BM3D (refilter) blind, patch
(29.01dB)
BLS-GSM blind (34.74dB)
Noise Clinic blind (20.77dB)
Figure 3.5 PSD estimation and denoising of Lena corrupted by g3 with ∥g∥22 = 0.01. The patch-based estimator cannot separate the diagonal and antidiagonal components within the simple 2-D patch DCT, leading to errors in estimation.
The collaborative estimate manages to capture the peaks fairly accurately, although some overestimation occurs on the flat part of the original PSD, leading to loss of the finest details. BLS-GSM performs well on this noise;
Noise Clinic does not attenuate any significant noise.
PSD ofg4 Estimated PSD (patch)
Estimated PSD (collaborative)
Barbara noisy
(16.99dB) new BM3D (refilter) non-blind (25.77dB)
new BM3D (refilter) blind, collaborative
(25.54dB)
new BM3D (refilter) blind, patch
(25.33dB)
BLS-GSM blind (24.16dB)
Noise Clinic blind (25.45dB)
Figure 3.6 PSD estimation and denoising ofBarbaracorrupted byg4with∥g∥22= 0.02.
Although the lowest frequencies are severely underestimated, the collabora-tive PSD yields results very similar to those of the exact power spectrum.
The patch-based estimate achieves very similar PSNR, but noise artifacts are consistently generated through its use in denoising. BLS-GSM leaves considerable amounts of noise; Noise Clinic estimate offers good PSNR, but contains many high-frequency noise artifacts.
PSD ofg6 Estimated PSD (patch)
Estimated PSD (collaborative)
Lena noisy
(29.97dB)
new BM3D (refilter) non-blind (35.67dB)
new BM3D (refilter) blind, collaborative
(34.55dB)
new BM3D (refilter) blind, patch
(34.34dB)
BLS-GSM blind (34.62dB)
Noise Clinic blind (34.62dB)
Figure 3.7 PSD estimation and denoising of Lenacorrupted by g6 with∥g∥22 = 0.001.
Note that although small in comparison to the peak value, the flat part of the g6 PSD is non-zero. Despite the higher PSNR values obtained by BLS-GSM and Noise Clinic, their estimates contain a significant amount of residual noise.
100 300 550 800 1000 31
32 33 34 35 36 37
√︁|X|
PSNR(dB)
var{η}= 0.001
100 300 550 800 1000 26
28 30 32 34
√︁|X| var{η}= 0.01
100 300 550 800 1000 24
26 28 30 32
√︁|X| var{η}= 0.02
gw
g1
g2
g3
g4
Figure 3.8 Average PSNR values of the proposed blind denoising as a function of image size over a set of 20 images for different noise kernels and noise variance.
The full images are1000×1000px in size; for other sizes, a random square segment of each image is obtained for each experiment. While larger images yield on average better results in all experiments, particularly when noise is weak, i.e., var{η}= 0.001, the estimation quality is in most cases similar between300×300px and1000×1000px images. Algorithm parameters are used identical for all experiments; size-dependent adjustments could be ex-pected to further improve the estimation quality particularly on larger images.