# Fileset

[Final draft_Applied Microscopy.docx](https://mdr.nims.go.jp/filesets/b4ab97d4-a533-4893-ada9-e205d237ca64/download)

## Creator

Sujin Lee, Yoshihiro Midoh, Yuto Tomita, Takehiro Tamaoka, Mitsunari Auchi, [Taisuke Sasaki](https://orcid.org/0000-0002-5952-7638), Yasukazu Murakami

## Rights

[Creative Commons BY Attribution 4.0 International](https://creativecommons.org/licenses/by/4.0/)

## Other metadata

[Noise reduction of electron holography observations for a thin-foiled Nd-Fe-B specimen using the wavelet hidden Markov model](https://mdr.nims.go.jp/datasets/8d4e70a1-8ba5-4045-9cbc-5b298bdcaaac)

## Fulltext

1 1234 256 378 4 (5)91011 61213 71415 81617 9 (10)1819 (11)202122 122324 132526 142728 15 (16)2930 (17)313233 183435 193637 203839 21 (22)404142 234344 244546 254748 264950 2751 (28)525354 295556 30575859606162636465Noise reduction of electron holography observations for a thin-foiled Nd-Fe-B specimen using the wavelet hidden Markov modelSujin Lee1,2*, Yoshihiro Midoh3, Yuto Tomita4, Takehiro Tamaoka1, Mitsunari Auchi4, Taisuke Sasaki5, Yasukazu Murakami1,4,*1. Department of Applied Quantum Physics and Nuclear Engineering, Kyushu University, Fukuoka 819-0395, Japan2. presently with Korea Institute of Materials Science, Changwon 51508, Korea3. Graduate School of Information Science and Technology, Osaka University, Osaka 565-0871, Japan4. The Ultramicroscopy Research center, Kyushu University, Fukuoka 819-0395, Japan5. National Institute for Materials Science, Tsukuba 305-0047, Japan*Corresponding author:Sujin Lee: sujin4524@kims.re.krYasukazu Murakami: murakami.yasukazu.227@m.kyushu-u.ac.jpAbstractIn this study, we investigate the effectiveness of noise reduction in electron holography, based on the wavelet hidden Markov model (WHMM), which allows for the reasonable separation of weak signals from noise. Electron holography observations from a Nd2Fe14B thin foil showed that the noise reduction method suppressed artificial phase discontinuities generated by phase retrieval. From the peak signal-to-noise ratio, it was seen that the impact of denoising was significant for observations with a narrow spacing of interference fringes, which is a key parameter for the spatial resolution of electron holography. These results provide essential information for improving the precision of electron holography studies.Keywords: noise reduction, wavelet hidden Markov model, electron holography, Nd-Fe- B magnet11 1234 2 (3)567 489 51011 61213 7 (8)1415 (9)161718 101920 112122 122324 13 (14)2526 (15)272829 163031 17323334 183536 19 (20)373839 214041 224243 234445 24 (25)4647 (26)484950 275152 285354 295556 305758596061626364651. IntroductionOff-axis electron holography, a method related to transmission electron microscopy (TEM), determines the phase shift of an electron wave traversing a thin-foil specimen [1,2,3]. As described in detail later, the phase of the object wave can be retrieved from the electron hologram produced by interference with the reference wave. Since the phase shift in the object wave originates from the electromagnetic field of the specimen, the reconstructed phase image approximates the map of the electrostatic potential and/or magnetic induction (that is, the in- plane component of the magnetic flux density, B). Because of this functionality, electron holography has been applied to the mapping of electromagnetic fields in p-n junctions of semiconductor devices [4,5], permanent magnets [6-8], magnetic nanoparticles [9-14], magnetic skyrmions [15-17], magnetic fluxons in superconductors [18,19], and several other materials. As shown by Lichte and Lehman [20], the precision of the phase analysis of electron holography depends on the image quality of the electron holograms, which comprise interference fringes of the object and reference electron waves. Indeed, the phase-detection limit can be explained by factors such as the number of electrons per resolved pixel, detection quantum efficiency of the camera, and fringe contrast of the electron hologram (that is, the visibility, 𝑉𝑜𝑏𝑠) [21,22].A noise reduction method based on image processing that can be applied to either an electron hologram or reconstructed phase image is effective for improving the accuracy of the phase analysis. As a noise-reduction strategy applicable electron holograms, Anada et al. [23,24] demonstrated the usefulness of sparse coding, which clearly visualized the electrostatic potentials of p-n junctions in GaAs-based semiconductor devices. Nomura et al. [25] employed tensor decomposition to derive the phase information from electron holograms acquired under low-electron-dose conditions. Another effective noise-reduction method is the wavelet hidden Markov model (WHMM) developed by Midoh and Nakamae [26,27]. The WHMM performs noise reduction using a wavelet transform by depressing the wavelet coefficients statistically. In the conventional thresholding method of the wavelet coefficients, the application of a threshold eliminates not only noise, but also weak signals lower than the threshold limit [28]. To solve this problem, Midoh and Nakamae expressed a process of wavelet transform using Markov parameters [26]. To briefly explain WHMM, we suppose two hidden states L and S,21 123 245 3 (4)678 5910 61112 71314 81516 9 (10)1718 (11)192021 122223 132425 1426 (15)2728 (16)293031 173233 183435 193637 20 (21)3839 (22)404142 234344 244546 254748 26 (27)495051 285253 295455 305657 315859606162636465which are related to the production of signal and noise, respectively. Regarding the two hidden states, we define the parameters 𝜎𝐿 and 𝜎𝑆 for each image pixel in all frequency levels of the wavelet transform. With reference to the hidden state L, the parameter 𝜎𝐿 represents that an image pixel produces the wavelet coefficient w (determined for all the image pixels by using observations) with probability 𝜎𝐿. Similarly, the parameter 𝜎𝑆 represents the probability to produce w with reference to the hidden states S. The transition probability 𝜀 is used to explain the feature of hidden states (as to which state is dominant in one image pixel) can be inherited to the other pixel at the lower/upper frequency level during the forward/inverse wavelet transform. The 𝜀 should be defined for all the image-pixel pairs (for corresponding pixels between upper and lower frequency levels). These parameters are optimized by the Baum- Welch algorithm, so that the observations of w can be most reasonably explained by the model. In other words, the WHMM offers optimized noise reduction for individual image pixels depending on the probability of representing the signal or noise. We refer the reader to the original paper [26] for more details on WHMM.Midoh and Nakamae [26] discussed the usefulness of the WHMM using artificially calculated electron holograms. Regarding holograms obtained experimentally, Tamaoka et al. [29] applied WHMM to observations from a non-magnetic, multiple-layered film of LaFeO3/SrTiO3. Noise reduction improved the quality of the reconstructed phase image, representing a gap in the electrostatic potential at the LaFeO3/SrTiO3 interface. From these results, it is expected that WHMM can be widely applied to electron holography observations. One of the essential targets is magnetic domain observation from a permanent magnet, including an Nd2Fe14B crystal. Due to the large magnetocrystalline anisotropy in Nd2Fe14B crystals, Nd-Fe-B-based magnets are the strongest commercial magnet widely used for traction motors in electric vehicles and several other applications [30]. Magnetic domain analysis is vital for improving the performance of Nd-Fe-B-based magnets. However, in studies using TEM, the Nd2Fe14B crystal provides poor image contrast due to the significant electron absorption by the heavy element Nd. Indeed, it appears that the visibility of electron holograms from Nd2Fe14B crystals is poor compared to observations from other magnetic alloys and compounds. For example, an Fe70Al30 alloy allowed for magnetic flux mapping over a wide range of the foil thickness up to 230 nm with an acceleration voltage of 300 kV [31]. However, in the previous electron holography study of Nd2Fe14B magnet, the magnetic flux density was determined using a thin- 31 123 2 (3)4567 489 51011 61213 7 (8)1415 (9)161718 101920 112122 122324 13 (14)2526 (15)27282930 163132 173334 18 (19)35363738 203940 21414243 224445 23 (24)464748 254950 265152 275354 285556 29 (30)575859606162636465foiled specimen the thickness of which was approximately 100 nm in the acceleration voltage of 300 kV [8]. Thus, noise reduction is essential for precision improvement in magnetic domain analysis.The spacing of the interference fringes (s) in the hologram is generally of the order of 1 nm, as shown in the subsequent section. Narrow fringe spacing is essential for high spatial resolution in electron holography [1]. However, as a feature of noise reduction using a wavelet transform (including the application of the WHMM), the high-frequency component related to the narrow fringe spacing is more depressed than the low-frequency component related to wide- spacing fringes. To address this issue, an alternative method of noise reduction involves applying the WHMM to the real (r) and imaginary parts (i) of the complex image produced via a fast Fourier transform (FFT) using the electron hologram [Figs. 1(c) and (d)]. Indeed, in the previous studies using WHMM [26,29], the noise reduction was applied to the electron holograms which were made of interference fringes of electron waves. In contrast, in this study, the noise reduction is applied to the complex images. The phase-retrieval process shown in Fig. 1 is discussed in further detail later.Thus, the purpose of this study is to evaluate the usefulness of noise reduction by WHMM, for which the noise reduction was applied to the real and imaginary parts of the complex image in the real space. Electron holograms were acquired from a thin-foil specimen comprising Nd2Fe14B crystals.2. Experimental methodA thin-foil Nd2Fe14B specimen was used to evaluate the denoising effect using a WHMM. A focused ion beam/scanning electron microscope (FIB/SEM; Helios G4 UX, FEI Co.) was used to obtain a polycrystalline rectangular Nd-Fe-B block, which was polished into a thin foil using another FIB instrument (MI4000L, Hitachi Ltd.). To reduce surface damage, the foil was polished at low acceleration voltages (5 kV and 2 kV) using a Ga-ion beam during the final stage of specimen preparation. Figure 2(a) shows the TEM image of the Nd2Fe14B thin-foil specimen. The specimen contained grain boundaries and a triple junction comprising an Nd- rich phase, as shown schematically in Fig. 2(b). The grey circles in Fig. 2(b) indicate thepositions of the artificially produced dimples that were referred to in the thinning process; the41 123 2 (3)45 (4)678 5910 61112 71314 8 (9)1516 (10)171819 112021 122223 132425 142627 15 (16)2829 (17)303132 183334 193536 203738 21 (22)3940 (23)414243 244445 254647 264849 27 (28)5051 (29)525354 305556 31575859606162636465grey dotted lines offer a guide to visualize the positions of the dimples. Since the specimen was an isotropic magnet, the directions of the c-axis in the three Nd2Fe14B grains were approximately parallel to those indicated by the red arrow. With reference to the x-y-z coordinate system in Fig. 2(a), the left edge of the thin-foil specimen was parallel to the y-axis, while the electron was incident along the −z direction. Electron holograms were acquired with various values of fringe spacing, s, using a 300-kV TEM (HF-3300X, Hitachi Hightech) with double-biprism electron interferometry [32]. The double-biprism system enables changing the fringe spacing s and interference width W (that is, fringe spacing multiplied by the number of fringes) independently. In this study, the applied voltage of the upper biprism VBP1 was varied (−50 V, −70 V, −90V, −110 V, −130 V, and −150 V) to obtain several values of s, while the voltage of the lower biprism VBP2 was fixed at -100 V. As described later in detail, these conditions attained the values of s at 5.2 nm, 3.7 nm, 2.9 nm, 2.4 nm, 2.1 nm, and 1.7 nm, although W remained unchanged at 1344 nm. For all holograms, the electron exposure time (ta) was 3.0 s. The electron-dose rate was approximately 1.5 e−/(Å ∙ s) (that is, beam current density of 24 A/m2). The variation in s affects the visibility of the electron holograms, as shown in Fig. 2(c), which is discussed in greater detail in the next section. Holograms were recorded using a high-sensitivity camera (K3 IS camera, Gatan Inc.) [33].As the mother wavelet in the two-dimensional wavelet transform, the “Farras wavelet” was used. Figure 1 shows the process of phase retrieval using FFT and the real and imaginary parts of the reconstructed images (that is, complex images) produced by the inverse Fourier transform (FFT-1). An electron hologram was subjected to FFT to obtain digital diffractograms [Figs. 1(a) and (b)]. A frequency-selection mask was applied to one of the sidebands that contained the phase information due to the electromagnetic field, as shown in Fig. 1(b), followed by moving the selected sideband to the position of the center band. FFT-1 generates a complex image (reconstructed image) that can be decomposed into real (𝑟) and imaginary (𝑖) parts, as shown in Figs. 1(c) and (d). In this study, denoising by the WHMM was applied to the real and imaginary parts of the images [Figs. 1(e) and (f)] instead of to the electron hologram [Fig. 1(a)]. A phase image revealing the electromagnetic field can be reconstructed by considering the arctangent tan−1 𝑖/𝑟, as shown in Fig. 1(g).The phase shift ( 𝜙 ) in the object electron wave, which can be determined by electron holography, can be expressed as follows:51 123 2456 378 4910 5 (6)1112 (7)131415 81617 91819 102021 11 (12)2223 (13)242526 142728 152930 163132 173334 18 (19)3536 (20)373839 214041 224243 234445 24 (25)4647 (26)484950 275152 2853𝜙 = 𝜎 ∫ 𝑉(𝑥, 𝑦, 𝑧) 𝑑𝑧 − 𝑒 ∫ 𝐴𝑧(𝑥, 𝑦, 𝑧) 𝑑𝑧,   (1) (ℏ)where 𝜎, 𝑒, and ℏ are the interaction constant based on the acceleration voltage of incident electrons (along z direction), elementary charge, and Planck’s constant divided by 2 𝜋 , respectively. 𝑉(𝑥, 𝑦, 𝑧) represents the electrostatic scalar potentials. If the electric charge on the specimen is negligible, this term approximates the mean inner potential (Vo) of the specimen, when the additional phase shift due to electron scattering in the crystal is disregarded. Indeed, to depress the undesired phase shift due to electron scattering, the specimen was tilted off a crystal zone axis to suppress the Bragg reflections. To separate the contribution of 𝑉(𝑥, 𝑦, 𝑧) [the first term in Eq. (1)] from that of 𝐴𝑧(𝑥, 𝑦, 𝑧) [the second term in Eq. (1)], we employed the method referred to as time-reversal operation using electron waves [34]. The time reversal can be attained by flipping the specimen upside down with respect to the incident electrons. These operation remains the electrostatic contribution to the phase shift (𝜙) unchanged but makes the magnetic field contribution opposite sign. Finally, by subtracting the two holograms for the time reversal the phase shift only due to 𝐴𝑧(𝑥, 𝑦, 𝑧) can be obtained from Eq. (1).𝐴𝑧(𝑥, 𝑦, 𝑧) is the z-component of the vector potential (A), which is related to the magnetic fluxdensity (B) by the equation 𝑩 = 𝑟𝑜𝑡 𝑨. Because of this relationship, the second term in Eq. (1) determines the in-plane (x-y plane) component of the magnetic flux density using electron holography observations.3. Results and DiscussionFigure 2(c) shows the visibility 𝑉𝑜𝑏𝑠 of electron holograms as functions of the applied voltage to the upper biprism VBP1 (that is, lower horizontal axis) and fringe spacing s (that is, upper horizontal axis). As mentioned earlier, when the double-biprism system is employed, the value of s can be tuned by 𝑉𝑜𝑏𝑠, while the interference width W remains unchanged [32].𝑉𝑜𝑏𝑠 was evaluated by using Eq. (2), where 𝐼𝑚𝑎𝑥 and 𝐼𝑚𝑖𝑛 are the maximum and minimum intensities of the electron hologram [2]. (29)54555657 305859606162636465𝑉𝑜𝑏𝑠= 𝐼𝑚𝑎𝑥 − 𝐼𝑚𝑖𝑛𝐼𝑚𝑎𝑥 + 𝐼𝑚𝑖𝑛6(2)1 123 245 3 (4)67 (5)8910 61112 71314 81516 9 (10)1718 (11)192021 122223 132425 142627 15 (16)282930 173132 183334 193536 203738 21 (22)3940 (23)414243 244445 254647 264849 275051 28 (29)5253 (30)545556 31575859606162636465Figure 2(c) shows the average value of 𝑉𝑜𝑏𝑠 determined for the rectangular area (framed by yellow lines) in Fig. 2(a). For additional information about 𝑉𝑜𝑏𝑠, Fig. 2(d) provides a series of electron holograms with s values of 5.2 nm, 2.4 nm, and 1.7 nm. As the fringe spacing decreases,𝑉𝑜𝑏𝑠 gradually decreases. Following the discussion by Chang et al. [35], the observed visibility is related to several factors, including the time-dependent part of visibility related to instrument instabilities (slower than exposure time), spatial coherence envelope of the wave field which includes instabilities faster than exposure time, and modulation transfer function (MTF) of the camera at the fringe spatial frequency 𝑘0. When the electron holograms are collected at a fixed value of VBP2 (that is, at a constant value of W, which represents the spatial coherency), the decrease in visibility can be explained by the frequency dependence of the MTF [32,33]. Eventually, the visibility of the electron hologram degrades when the fringe spacing is reduced, as shown in Figs. 2(c) and 2(d). Note that, according to the study by Chang et al. [33], a direct detection camera (similar to the product used in this study) shows only a gradual change in MTF in a wide range of the Fourier space. The impact of the decrease in MTF, which is observed in the neighborhood of the side band, on the phase retrieval remains to be an open question.The deterioration of 𝑉𝑜𝑏𝑠 affects the quality of the phase image reconstructed using the FFT process, as shown in the schematic in Fig. 1. Figures 3(b)-(d) provide phase images reconstructed from the holograms acquired at fringe spacings of (b) 5.2 nm (VBP1 = −50 V),(c) 2.4 nm (VBP1 = −110 V), and (d) 1.7 nm (VBP1 = −150 V), respectively. The change in phase is represented by colors as per the color bar in Fig. 3. Phase images were obtained from the field of view shown in Fig. 3(a) [identical to that in Fig. 2(a)]. In the reconstructed phase images, the position of the specimen border is indicated by a white dotted line.As mentioned earlier, the visibility 𝑉𝑜𝑏𝑠 of the electron holograms was reduced by decreasing the fringe spacing, as shown in Figs. 2(c) and 2(d). Importantly, when the fringe contrast of an electron hologram is poor, a phase-unwrapping algorithm [1] that was used introduce an artificial discontinuity in phase: that is, an unwanted phase difference of 2, which is represented by patches in the colored phase maps. The appearance of the phase jump has been briefly explained in Refs. [1] and [36]. As explained earlier, a phase shift can be determined by calculating tan−1 𝑖/𝑟 using the real and imaginary parts of a complex image (Fig. 1). Due to the calculation using an arctangent function, the reconstructed phase is71 123 2 (3)45 (4)678 5910 61112 71314 81516 9 (10)171819 112021 122223 132425 14 (15)2627 (16)282930 173132 183334 193536 20 (21)3738 (22)394041 234243 244445 254647 26 (27)4849 (28)505152 295354 305556 31575859606162636465presented in a “wrapped” form, plotted in a limited range from -𝜋 to 𝜋. A phase-unwrapping algorithm is employed to “unwrap” the phase image such that the phase can be plotted in an extended range [that is, a continuous change in phase can be revealed over the field of view, as shown in Fig. 3(b)]. For phase unwrapping, this study used a code incorporated in the commercial software HoloWorks v.5.0 (Gatan). However, the phase unwrapping was unsuccessful when the visibility 𝑉𝑜𝑏𝑠 of the electron hologram was poor. Indeed, as shown in Figs. 3(b)-(d), the population of patches (that is, artificial phase jumps) increased with a decrease in 𝑉𝑜𝑏𝑠 . Since the foil thickness was reduced on the right side of the specimen (showing a wedge-shaped cross-section), the fringe contrast was better than that observed on the left side. The number of patches on the right side of the specimen was negligible. However, owing to the increase in foil thickness, the fringe contrast on the left side was poor. Therefore, more patches are observed on the left side of the specimen.We discuss the impact of WHMM on phase retrieval, in which denoising is applied to the real and imaginary parts of a complex image, as shown in Fig. 1. To discuss the usefulness of the WHMM, we focus on the population of artificial phase jumps (that is, patches in colored phase images), which appear to be reduced by denoising. Figures 4(a)–(i) summarize the phase images collected under three conditions of fringe spacing s: (a)–(c) 2.4 nm, (d)–(f) 2.1 nm, and (g)–(i) 1.7 nm. The field of view corresponds to the rectangular area shown in Fig. 3(a). For each s series of images, the “reference measurement” represents the phase images showing only negligible patches as they were reconstructed from holograms (at the three conditions of s) with a sufficiently long exposure time ta = 15 s; see Figs. 4(a), (d), and (g). As revealed by the phase plots measured in the R-S line crossing the specimen [Figs. 4(j), (k), and (l)], the three reference-measurement images were almost identical in terms of the magnitude and smoothness in the phase map (black dots in (j), (k), and (l)), although they are mostly overlapped by red dots representing the denoised (denoised) image, as explained later in detail. Note that the steep changes in the phase, which were observed at the specimen borders (𝑥 = 170 nm and 𝑥 = 825 nm) were due to the contribution from the mean inner potential of the crystal. The phase shift owing to the magnetic flux density in the Nd2Fe14B crystal [that is, the second term in Eq. (1)] was superimposed on the phase shift due to the mean inner potential [that is, the first term in Eq. (1)] for electron holography observations. Nevertheless, because of the significant contribution from the magnetic flux density, the phase plot continued to81 123 245 3 (4)678 5910 61112 71314 8 (9)1516 (10)171819 112021 122223 132425 142627 15 (16)282930 173132 183334 193536 203738 21 (22)3940 (23)414243 244445 254647 264849 27 (10𝑖=1𝑗=1)50decrease over the range of 170 nm < 𝑥 < 825 nm, in which the specimen was magnetized along the −y direction. Reference-measurement images were used to quantify the denoising effect.The images labeled “noised” represent the original phase images to which the noise reduction by WHMM was not applied; see Figs. 4(b), 4(e), and 4(h). Since the exposure time was only 3 s (that is, the visibility of the interference fringe was poor), many patches were produced during the phase retrieval process, particularly in Figs. 4(e) and 4(h). With respect to the original images labeled “noised,” the phase shift was measured along the R-S line, where the position was identical to that of the reference-measurement images. The results are indicated by the blue dots in Fig. 4(j) for s = 2.4 nm, Fig. 4(k) for s = 2.1 nm, and Fig. 4(l) for s = 1.7 nm. In the presence of several patches in the original images, the blue dots deviated from the reference-measurement curve. The deviation was especially large in the observation at s = 1.7 nm since the visibility of electron hologram was the lowest. Additionally, as mentioned earlier, the deviation from the reference-measurement curve was significant on the left side of the specimen compared to that on the right side, because the visibility was reduced in the thick portion of the specimen.Figures 4(c), 4(f), and 4(i), labeled “denoised,” show the phase images to which denoising by WHMM was applied. As a result of the noise reduction applied to the real and imaginary parts of the complex image (Fig. 1), the patches in the phase images were mostly eliminated in all observations. Regarding the noise-reduced (denoised) images, the plots of the phase shift measured along the R-S line are shown in red in Fig. 4(j) for s = 2.4 nm, in Fig. 4(k) for s =2.1 nm, and in Fig. 4(l) for s = 1.7 nm. The plots from the denoised images agree well with those from the reference-measurement images. The results explicitly indicate the usefulness of noise reduction by WHMM.For further examination of the noise reduction, the peak signal-to-noise ratio (PSNR) defined by Eq. (3) was calculated for the denoised images [37]: (28)51525354 2955𝑃𝑆𝑁𝑅 = 20 ∙ log (255⁄√1/𝑀𝑁 ∑𝑀  ∑𝑁  (𝐼(𝑖, 𝑗) − 𝐺(𝑖, 𝑗))2), (3)56 30575859606162636465where 𝐼(𝑖, 𝑗) represents the intensity of either the original (without denoising) or denoised91 123 245 3 (4)67 (5)8910 61112 71314 81516 9 (10)171819 112021 122223 132425 142627 15 (16)2829 (17)303132 183334 193536 203738 21 (22)3940 (23)414243 244445 254647 264849 27 (28)5051 (29)525354 305556 31575859606162636465images measured at pixel positions (𝑖, 𝑗). M and N represent the pixel size; in this study, M = 536 for 𝑖 (in x direction) and N = 536 for 𝑗 (in y direction). 𝐺(𝑖, 𝑗) represents the intensity of the reference-measurement image measured at pixel position (𝑖, 𝑗). The PSNR was calculated for the area enclosed by the yellow lines in Fig. 5(a), which is identical to Fig. 4(a). Figure 5(b) shows the PSNR as a function of the voltage applied to the upper biprism VBP1 (that is, lower horizontal axis) and fringe spacing s (that is, upper horizontal axis). The open squares represent the PSNR determined for the original phase images without noise reduction, whereas the closed squares represent values for the denoised images. Regarding the open squares representing noisy images (without WHMM denoising), the PSNR continued to decrease with decreasing fringe spacing, because the fringe contrast decreased with decreasing fringe spacing. Note that the plot at s = 1.7 nm (for open square) takes a negative value at PSNR = −1.9. This result indicates that the signal is much weaker than the noise under these conditions. When noise reduction by WHMM was applied, the PSNR increased under all fringe spacing conditions, as shown in the closed square in Fig. 5(b). Importantly, the improvement in PSNR [that is, ΔPSNR, defined by the difference between the PSNR for the denoised image (closed square) and noised images (open square)] was significant for the observations with small fringe spacing. As demonstrated in Fig. 5(c), ΔPSNR continues to increase as the fringe spacing is reduced. It is interesting that the PSNR at s = 1.7 nm (for denoised images) becomes comparable to the value at s = 2.9 nm (for noisy images) as a result of noise reduction by WHMM. Although the visibility of the fringes (responsible for the sensitivity of phase detection) deteriorates when the fringe spacing (responsible for the spatial resolution) is reduced, noise reduction helps improve the fringe contrast. Thus, noise reduction using the WHMM is advantageous for electron holography with a high spatial resolution. To see the effectiveness of noise reduction on the magnetic domain analysis, the left panel of Fig. 5(d) shows the original reconstructed phase image (labelled by “noised”) which was obtained at the condition of s = 2.4 nm. The field of view is identical to the area closed by the dotted in Fig. 5(a). Although this area represents a single magnetic domain that was magnetized in the y direction, the original image shows several phase discontinuities (i.e., patches in the phase images) due to the imperfect phase retrieval using a noised electron hologram. However, when the noise reduction was applied, the phase discontinuities were removed as shown in the right panel of Fig. 5(d) (labelled by “denoised”). The residual patch in the right panel (denoised101 123 2 (3)45 (4)678 5910 61112 71314 8 (9)1516 (10)171819 112021 122223 132425 14 (15)2627 (16)282930 17313233 183435 19 (20)3637 (21)383940 224142 234344 244546 25 (26)474849 275051 285253 295455 305657 315859606162636465image), which is present at the center/left position in the field of view, represents the area the thickness of which was reduced in the process of sample preparation. Thus, the noise reduction helps us examine the magnetic domain structure and/or magnetic flux density using electron holography. From the viewpoint using Nd-Fe-B permanent magnets, which generally provide poor contrast in TEM images and/or electron holograms because of the significant absorption of electrons, noise reduction by WHMM is also helpful for magnetic domain observations that are executed by electron holography.There are several methods of transmission electron microscopy that allow for magnetic domain observations: i.e., Lorentz microscopy, differential phase contrast (DPC) with scanning transmission electron microscopy (STEM), electron holography, etc. Regarding the advantage of electron holography, this method is useful for the magnetic flux density measurement from a grain boundary (GB) region [8,38], while tailoring of the grain boundary region is essential for the coercivity improvement. Thus, the noise reduction using WHMM, which is applied to electron holography, can be a powerful tool in the research and development of Nd-Fe-B permanent magnets.4. ConclusionWe demonstrated the effectiveness of noise reduction when the WHMM is applied to phase retrieval by electron holography. Electron holograms were acquired from thin-foiled Nd2Fe14B crystals with various fringe spacings. The narrow fringe spacing, which can provide a high spatial resolution of the phase analysis, reduces the visibility of the holograms, resulting in artificial jumps of the phase shift in the reconstructed phase image. When WHMM denoising was applied to both real and imaginary parts of the complex images obtained in the process of phase retrieval, the undesired patches (due to the artificial phase jump) could be suppressed in all the observations acquired in the range of fringe spacing from 1.7 nm to 5.2 nm. Particularly, the denoising effect is significant for observations with narrow fringe spacing. The visibility of the interference fringes in holograms deteriorates when the fringe spacing is reduced, whereas a narrow spacing is required for observations at a high spatial resolution. Denoising holograms are a promising approach to solve this technical problem.List of Abbreviations111 123 2 (3)45 (4)678 5910 61112 71314 8 (9)1516 (10)171819 112021 122223 132425 14 (15)2627 (16)282930 173132 183334 193536 20 (21)373839 224041 234243 244445 25 (26)4647 (27)48495051 28525354 295556 30575859606162636465TEM Transmission electron microscopy WHMM Wavelet hidden Markov model FFT Fast Fourier transformFIB Focused ion beamSEM Scanning electron microscope FFT-1 Inverse Fourier transform MTF Modulation transfer function PSNR Peak signal-to-noise ratio DPC Differential phase contrastSTEM Scanning transmission electron microscopyDeclarations:Availability of data and materialsNot applicableCompeting interestsThe authors declare that they have no competing interestsFundingNot applicableAuthors’ contributionsS. Lee carried out experiments and analysis of the data, and wrote the manuscript. Y. Midoh contributed to denoising parts. Y. Tomita and T. Tamaoka collected and analyzed the electron holography data. M. Auchi and T. Sasaki prepared the specimen. Y. Murakami wrote the manuscript. All authors approved the final version of the manuscript.AcknowledgementsThis study was supported in part by KAKENHI (21H04623 and 22K18904) from the Japan Society for the Promotion of Science.121 123 2 (3)45 (4)6789 5101112 613 (7)1415 (8)16171819 9202122 1023 (11)24252627 122829 13303132 14 (15)33343536 16373839 174041 1842 (19)4344 (20)45464748 214950 2251 (23)5253 (24)545556575859606162636465References1. E. Völkl, L. F. Allard, and D. C. Joy, Introduction to Electron Holography (Kluwer Academic/Plenum Publishers, New York, 1999).2. A. Tonomura, Electron Holography, 2nd edn. (Springer, Heidelberg, 1999).3. P. A. Midgley, Micron 32, 167 (2001).4. W. D. Rau, P. Schwander, F. H. Baumann, W. Höppner, and A. Ourmazd, Phys. Rev. Lett. 82, 2614 (1999).5. Z. Wang, T. Hirayama, K. Sasaki, H. Saka, and N. Kato, Appl. Phys. Lett. 80, 246 (2002).6. M. R. McCartney and Y. Zhu, Appl. Phys. Lett 72, 1380 (1998).7. Y. Zhu, V. V. Volkov, and M. De Graef, J. Electron Microsc. 50, 447 (2001).8. Y. Murakami, T. Tanigaki, T. T. Sasaki, Y. Takeno, H. S. Park, T. Matsuda, T. Ohkubo, K. Hono, and D. Shindo, Acta. Mater. 71, 370 (2014).9. A. Tonomura, T. Matsuda, J. Endo, T. Arii, and K. Mihama, Phys. Rev. Lett., 44, 1430 (1980).10. S. L. Tripp, R. E. Dunin‐ Borkowski, and A. Wei, Angewandte Chemie 115, 5749 (2003).11. S. Yamamoto, K. Yamamoto, D. L. Peng, T. Hirayama, and K. Sumiyama, Appl. Phys. Lett. 90, 242510 (2007).12. M. Ammar, M. LoBue, E. Snoeck, M. Hÿtch, Y. Champion, R. Barrué, and F. Mazaleyrat, J. Magn. Magn. Mater. 320, e716 (2008).13. Y. Takeno, Y. Murakami, T. Sato, T. Tanigaki, H. S. Park, D. Shindo, R. M. Ferguson, andK. M. Krishnan, Appl. Phys. Lett. 105 (2014).14. N. Biziere, D. Reyes, T. K. Wade, B. Warot-Fonrose, and C. Gatel, J. Appl. Phys. 126, 163906 (2019).131 123 24 (3)567 48910 51112 613 (7)1415 (8)16171819 92021 1022 (11)23242526 12272829 13303132 14 (15)33343536 163738 17394041 184243 1944 (20)45464748 214950 225152 2353 (24)545556 2557585960616263646515. H. S. Park, X. Yu, S. Aizawa, T. Tanigaki, T. Akashi, Y. Takahashi, T. Matsuda, N. Kanazawa, Y. Onose, D. Shindo, A. Tonomura, and Y. Tokura, Nat. Nanotech. 9, 337 (2014).16. A. Kovács, J. Caron, S. A. Savchenko, N. S. Kiselev, K. Shibata, Z. Li, N. Kanazawa, Y. Tokura, S. Blügel, and R. E. Dunin-Borkowski, Appl. Phys. Lett. 111, 192410 (2017).17. K. Shibata, A. Kovács, N. S. Kiselev, N. Kanazawa, R. E. Dunin-Borkowski, and Y. Tokura, Phys. Rev. Lett. 118, 087202 (2017).18. T. Matsuda, A. Fukuhara, T. Yoshida, S. Hasegawa, A. Tonomura, and Q. Ru., Phys. Rev. Lett. 66, 457 (1991).19. K. Harada, O. Kamimura, H. Kasai, T. Matsuda, A. Tonomura, and V. V. Moshchalkov, Science 274, 1167 (1996).20. H. Lichte and M. Lehmann, Rep. Prog. Phys. 71, 016102 (2008).21. A. Harscher and H. Lichte, Ultramicroscopy 64, 57 (1996).22. W. J. De Ruijter and J. K. Weiss, Ultramicroscopy 50, 269 (1993).23. S. Anada, Y. Nomura, T. Hirayama, and K. Yamamoto, Ultramicroscopy 206, 112818 (2019).24. S. Anada, Y. Nomura, T. Hirayama, and K. Yamamoto, Microsc. Microanal., 26, 429 (2020).25. Y. Nomura, K. Yamamoto, S. Anada, T. Hirayama, E. Igaki, and K. Saitoh, Microscopy, 70, 255 (2021).26. Y. Midoh and K. Nakamae, Microscopy 69, 123 (2020).27. Miura Lab. (Graduate School of Information Science and Technology, Osaka University), Wavelet hidden Markov model denoising (2023). http://www-ise3.ist.osaka-u.ac.jp/whmm/. Accessed 16. Oct 2023.28. M. Jansen, Noise reduction by wavelet thresholding (Springer Science & Business Media, New York, 2012).141 123 24 (3)5678 4910 5111213 6 (7)14151617 81819 9202122 1023 (11)242526 12272829 13303132 14 (15)33343536 163738 17394041 1842434445464748495051525354555657585960616263646529. T. Tamaoka, Y. Midoh, K. Yamamoto, S. Aritomi, T. Tanigaki, M. Nakamura, K. Nakamae, M. Kawasaki, and Y. Murakami, AIP Adv. 11, 025135 (2021).30. K. Hono and H. Sepehri-Amin, Scripta Mater. 67: 530-535 (2012).31. Y. Murakami, K. Niitsu, T. Tanigaki, R. Kainuma, H. S. Park, and D. Shindo, Nature comm. 5, 4133 (2014).32. K. Harada, A. Tonomura, Y. Togawa, T. Akashi, and T. Matsuda, Appl. Phys, Lett. 84, 3229 (2004).33. S. L. Chang, C. Dwyer, J. Barthel, C. B. Boothroyd, and R. E. Dunin-Borkowski, Ultramicroscopy 161, 90 (2016).34. A. Tonomura, T. Matsuda, J. Endo, T. Arii, and K. Mihama, Phys. Rev. B 34, 3397 (1986).35. S. L. Chang, C. Dwyer, C. B. Boothroyd, R. E. Dunin-Borkowski, Ultramicroscopy 151, 37 (2015)36. E. Völkl, L. F. Allard, and B. Frost, J. Microsc. 180, 39 (1995).37. S. Arora, J. Acharya, A. Verma, and P. K. Panigrahi, Pattern Recognition Letters 29, 119 (2008).38. Y. Cho, T. Sasaki, K. Harada, A. Sato, T. Tamaoka, D. Shindo, T. Ohkubo, K. Hono, and Y. Murakami, Scripta Mater. 178, 553 (2020).151 1234 25 36 4 (5)789 610 7 (8)11121314 91516 10 (11)171819 1220 13 (14)212223 1524 1625 (17)26272829 1830 19 (20)313233 213435 2236 (23)373839 2440 25 (26)414243 2744 28 (29)454647 3048 314950 32515253 3354 3455 355657 365859606162636465Figure CaptionsFig. 1 Process of phase retrieval. (a) Electron hologram. (b) Digital diffractogram obtained by fast Fourier transform (FFT). The yellow circle in (b) indicates a selected specific frequency zone for FFT-1 of the diffractogram. (c) and (d) Real and imaginary parts of a complex image reconstructed from the hologram via FFT-1. (c) and (f) Results of real and imaginary parts after applying the noise reduction to (c) and (d), respectively. (g) Reconstructed phase image obtained by calculating 𝐭𝐚𝐧−𝟏 𝒊/𝒓, where 𝒊 and 𝒓 are the imaginary part of (f) and the real part of (e), respectively.Fig. 2 Observation of a thin-foiled Nd2Fe14B specimen by electron holography. (a) TEM image of the Nd2Fe14B specimen. The yellow rectangular area in (a) indicates the field view for electron holograms in (d). (b) Schematic of electron holography process. (c) Visibility of electron holograms as a function of the voltage of upper biprism (lower horizontal axis) and fringe spacing (upper horizontal axis). (d) Series of electron holograms acquired at fringe spacings of 5.2 nm, 2.4 nm, and 1.7 nm, corresponding to VBP1 of −50 V, −110 V, and −150 V, respectively.Fig. 3 Influence of the fringe spacing on the visibility of the reconstructed phase image. (a) TEM image of the thin-foiled specimen, identical to Fig. 2(a). The blue rectangular area in (a) indicates the enlarged area for phase images in Fig. 4. (b)–(d) Reconstructed phase images at fringe spacings of 5.2 nm, 2.4 nm, and 1.7 nm, respectively.Fig. 4. Phase image series collected with three fringe spacings: (1) s = 2.4 nm, (2) s = 2.1 nm, and (3) s = 1.7 nm. (a), (d), and (g) Reference-measurement images with long exposure time ta of 15 s for (1), (2), and (3), respectively. (b), (e), and (h) Original phase images, labelled “Noised,” with ta of 3 s for (1), (2), and (3), respectively. (c), (f), and (i) Phase images, labelled “Denoised,” after the application of the noise reduction to (b), (e), and (h), respectively. (j), (k), and (l) Plots of the phase shift measured along the R–S line in (a)–(c), (d)–(f), and (g)–(i), respectively. Black, blue, and red plots in (j)–(l) indicate the results from reference- measurement image [from (a), (d), and (g)], noised image [from (b), (e), and (h)], and denoised image [from (c), (f), and (i)], respectively.Fig. 5. Evaluation of the denoising effect using peak signal-to-noise ratio (PSNR). (a) Reference-measurement phase image for the fringe spacing s = 2.4 nm. PSNR was determined for the area indicated by the yellow lines. (b) PSNR as functions of the upper biprism voltage VBP1 (lower horizontal axis) and the fringe spacing (upper horizontal axis).161 1 Open and closed squares indicate PSNR values determined from the original phase images2 2 (including noise) and the noise-reduced (denoised) phase images, respectively. (c) Difference3  (3) (4)between the PSNR for the denoised images (closed squares) and original phase images (open5 4 squares) as function of VBP1 (lower horizontal axis) and s (upper horizontal axis). (d) Noised6 5 (left) and denoised (right) images obtained at s = 2.4 nm. The black dotted lines in (a) indicate7  (6) (8)the enlarged areas in (d). (7)91011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859 17606162636465Figure1 Figure2 Figure3 Figure 4 Figure 5 image1.pngimage2.pngimage3.pngimage4.pngimage5.png