Skip to main content

Directly and instantly seeing through random diffusers by self-imaging in scattering speckles

Abstract

Imaging through scattering medium is challenging but important for different applications. Most advances rely on computational image reconstruction from scattering signals. In these conventional investigations, speckles were always treated as scrambled grainy patterns. Directly seeing through scattering diffusers has never been realized. Here, we report a new strategy to see through random diffusers directly using self-imaging of speckles. By analyzing the physics, a direct observation strategy through scattering media is reported with improved image quality. Using this method, we experimentally demonstrated reconstruction-free real-time imaging of static and moving objects with their actual orientation information under single-wavelength and white light illumination. We also proposed a modified speckle autocorrelation imaging (SAI) method inspired by the self-imaging results. Importantly, our strategy requires no pre-calibration or acquisition of point-spread-function, no active control of wavefronts or complicated equipment, nor iterations or carefully adjusted parameters, paving the way towards rapid and high-quality imaging through scattering diffusers.

Introduction

Imaging through scattering is one of the biggest challenges facing optical technologies, which plays crucial roles in many fields including biomedical, and astronomy applications [1,2,3]. In the past decades, there have been increasing interests in coherent and incoherent imaging through scattering media. To enable imaging through this challenging condition, it is an emerging topic to extract spatial information through scattering by employing the active control of light with the aid of prior knowledge of objects or media, or combined with post-imaging recovery processes, i.e., computational imaging of hidden objects [4,5,6,7,8,9,10,11,12]. Many powerful computational imaging approaches have been reported, utilizing interference (such as holography [13]) and interference-less methods [14], transmission matrix [15, 16], active wavefront shaping techniques [17, 18] for precise light control and smart computational imaging through scattering media [19,20,21,22,23,24]. In these techniques, strict stability of operation with complex optical setups are required. To overcome these technical challenges, speckle-autocorrelation imaging (SAI) [25], deconvolution [26,27,28] and cross correlation [29,30,31] approaches were developed. In particular, the SAI method received extensive attention due to their advantages of single-shot, easy-operation, low-cost, and calibration-free properties. However, although the object’s image was recovered using SAI, its orientation remained unknown due to the random-initial guess and indeterministic iterations in phase retrieval (PR) [25, 32]. Based on SAI, various recovery algorithms [30, 33] and engineering of the point-spread-function (PSF) for imaging are still under active investigation for multitargets imaging [34], depth resolved [35] and three-dimensional (3D) imaging [36], and multispectral imaging [37]. However, all aforementioned imaging technologies through scattering media require sophisticated optical setups (e.g. [4,5,6,7,8,9,10,11,12,13,14,15,16]) or time-consuming post computational reconstruction with carefully adjusted parameters for the pre-processing of the speckle and/or the autocorrelation (e.g. [15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37]). In those previously reported strategies (e.g. [25,26,27,28,29,30,31,32,33,34,35,36,37]), directly obtained random speckle patterns have never been used for imaging due to severe degradation and multiple scattering of photons. Speckles were always considered as scrambled patterns with unseen chaos.

In this work, we report a ‘counterintuitive’ strategy to directly observe self-imaging units of small objects using speckles through diffusers. By carefully examining this speckle (which was considered as noise or chaotic patterns in previously reported imaging strategies), clear self-imaging phenomena can be directly obtained from a single shot of the speckle image, releasing the needs in time-consuming image acquisition and computational reconstruction. As a result, directly seeing through random diffusers with the naked eye and real-time video imaging of multitargets are realized. The orientation of the object can be directly seen and traced in real-time. Intriguingly, this method requires no complex equipment for the active control of light, nor prior knowledge of medium or pre-calibration of the system, with no need of iterations and parameter adjustments. Our results provide new perspective of diffuser-imaging systems, and suggest novel imaging applications through scattering diffusers.

Results

Figure 1(a) shows the diagram of a simple optical system for incoherent imaging through a thin-diffuser scattering medium (similar to those used in ref. [25,26,27,28,29,30,31,32,33,34,35,36,37]). By moving the camera along the Z direction, different slices of the volume-speckles can be captured at different depths of z. In this experiment, a standard USAF resolution test object (1951USAF, Edmund,1.2 mm in length) was illuminated by a collimated LED light source (M625L3, Thorlabs, with the central wavelength of λ = 625 nm and the bandwidth of ~ 16 nm) with an object-distance u = 50 cm to the diffuser. The speckle through a glass diffuser with 220-grit (DG10-220-MD Thorlabs) was recorded by a camera (Zyla 5.5, Andor) placed at a distance of v = 10 cm to the diffuser (see supplement information (SI) Note 1 and Fig. S1 for more experimental details). Figure 1(b) and (c) show the captured original raw speckles (first column) of different isolated numbers ‘1’ (Fig. 1b) and ‘3’ (Fig. 1c), respectively. One can see typical grainy and scrambled patterns which were widely observed in previous reports (e.g. [25,26,27,28,29,30,31,32,33,34,35,36,37]). However, in those previously reported imaging strategies (e.g. [4, 25,26,27,28,29,30,31,32,33,34,35,36,37]), digital-filtering and computational reconstructions with carefully tuned parameters were applied to process their raw speckles. Surprisingly, by analyzing enlarged regions in the speckle in our experiment, we recognized that those extra computational efforts are unnecessary! Self-reproduction of the objects can be observed directly (see black-rectangle-boxes in Fig. 1(b, c)). As shown in the second and third columns of Fig. 1(b-c), the self-reproduction of the object with perfect profiles can be identified from the speckle. In particular, the right column of Fig. 1(b-c) highlights the clear lineament. Although their contrasts are not ideal, self-reproduction of the object was directly observed in the chaotic speckle, which has never been reported.

Fig. 1
figure 1

Direct observation of self-imaging in raw speckles. a The schematic of the diffuser imaging system. b-d Experimental observation of self-imaging in speckles. Camera captured original raw speckles of isolated objects of (b) digit ‘1’, (c) digit ‘3’, and (d) multi-objects ‘2’ and ‘6’

To confirm this self-imaging effect, we further recorded the speckle of multi-objects: As shown in Fig. 1(d), self-imaging patterns of two digits of ‘2’ and ‘6’ were identified. By analyzing zoomed-in regions, one can see that random and chaotic speckle patterns contain rich self-imaging units. These units were generated by different extents of degenerated, out-of-focus, and incomplete self-imaging units with random position-shifts and scaled transmission intensity. As shown in Fig. S2, S3, S4 and S5 in the supporting information, we recorded self-imaging patterns of varied objects through diffusers at different wavelengths. Intriguingly, by carefully analyzing previously reported experimental data (e.g. [25,26,27,28,29,30,31,32,33,34,35,36,37]), this type of self-imaging patterns was also recorded but mostly omitted! The key question is: what is the mechanism of this self-imaging phenomenon?

Physical mechanism of self-imaging in speckle

To reveal the physical mechanism of the self-imaging phenomenon, here we explore the model for the incoherent diffuser imaging system. As shown in Fig. 2, we compare the incoherent imaging model of lens-system and a diffuser-system. A classic lens-imaging system will result in a point-to-point mapping (Fig. 2a) or object-to-image mapping (Fig. 2b). In contrast, the output of a diffuser-imaging system is a speckle for both a point source (Fig. 2c) or an object source (Fig. 2d). In this speckle, many ‘hot-spot’-like grains are obtained due to the constructive interference of incident beams from different optical paths (see Fig. 2c). This speckle can be simplified as a superposition of shift-and-scaled object’s self-reproduction images with degeneration, aberrations, and vignetting. Similar to the conventional optical lens imaging system (Fig. 2a and b), the diffuser imaging system can also be presented as a convolution. However, due to the random phase bumps and distributions of the diffuser, the ‘hot-spot’ (Fig. 2c) and the random self-imaging speckles (Fig. 2d) were induced by the interference of multiple scattered light. Next, we develop an incoherent-imaging model to interpret the diffuser imaging system.

Fig. 2
figure 2

Operational principles of diffuser-lens imaging systems. The schematic of (a) the PSF for (b) the lens imaging system. (c) The ‘hot-spot’ effect in the diffuser imaging speckle, and (d) the imaging model and experimental setup for incoherent illumination. (e-f) Random phase screen model (e) without features of phase distribution, and (f) with lenslet array phase features. The speckle in (e) is generated by calculating the light-propagation through phase-screen [38]. The speckle in (f) is generated considering the lenslet-phase with incoherent-imaging model using Eq. 1

By illuminating a transmission-type point or object O using a spatial incoherent light, the scrambled speckle I after the diffuser can be described as [25]:

$${I}_{\lambda }={O}_{\lambda}\ast {PSF}_{\lambda }$$
(1)

where’*denotes a convolution operation. No matter how many objects are within a single optical memory effect (OME) zone, they share a common PSF [25, 29, 33,34,35,36,37]. Using the SAI model and the convolution theory in SAI method [25], we get

$${I}_{\lambda}\bigstar {I}_{\lambda }=\left({O}_{\lambda}\bigstar{O}_{\lambda}\right)\ast \left({PSF}_{\lambda}\bigstar {PSF}_{\lambda}\right)=\left({O}_{\lambda}\bigstar {O}_{\lambda}\right)+C,$$
(2)

where’‘denotes the autocorrelation operation, and the ‘C’ represents the background term. The object’s diffraction-limited image can then be reconstructed from speckle’s autocorrelation by the SAI method using an iterative phase-retrieval algorithm [25, 39, 40]. Here we introduce a modification of the SAI imaging with the self-imaging model to investigate the self-imaging formed speckle and speckle’s autocorrelation. An array of identical objects O are introduced at varied random positions (xn’, yn,’ z) to represent the pattern of the speckle I′. The self-imaging modeled speckle can then be represented as:

$${I_{\lambda}}^{\prime}\left({x}^{\prime },{y}^{\prime },z\right)=\sum_{n=1}^N{O}_{\lambda}\left({x}^{\prime },{y}^{\prime },z\right)\ast \delta \left({x}^{\prime }-{x}_n^{\prime },{y}^{\prime }-{y}_n^{\prime },{z}^{\prime }-z\right)$$
(3)

In this case, one can easily show that the self-imaging modeled speckle’s autocorrelation gives

$${I_{\lambda}}^{\prime}\bigstar {I_{\lambda}}^{\prime }={F}^{-1}{\left|F\left({I_{\lambda}}^{\prime}\left({x}^{\prime },{y}^{\prime },z\right)\right)\right|}^2={F}^{-1}{\left|F\left({O}_{\lambda}\right)F\left(\sum\limits_{n=1}^N\delta \left({x}^{\prime }-{x}_n^{\prime },{y}^{\prime }-{y}_n^{\prime },z\right)\right)\right|}^2$$
(4)

where F represents the Fourier transformation operation, and F−1 is its inverse. Using Wiener-Khinchin theorem [25, 31,32,33,34], this equation can be simplified as

$${I_{\lambda}}^{\prime}\bigstar {I_{\lambda}}^{\prime}\cong {C}^{\prime}\bullet \left({O}_{\lambda}\bigstar {O}_{\lambda}\right),$$
(5)

where, ‘C″ is a constant coefficient. By comparing Eqs. 2 and 5, one can see that the speckle’s autocorrelation is exactly the original object’s autocorrelation (Eq. 5) with intensity-stretch but no background term nor cross-talks (Eq. 2). This relation between the object and its speckle provides clues to model the diffuser as a collection of randomly distributed lenses with a random phase-mask. As a result, the speckle can be treated as an accumulated random self-reproduction (Eq. 3) of the diffuser-modulated self-imaging of the object. With these self-reproduced images, degenerated and dimmed partial self-image units, the accumulation of the overlapped units forms the final scrambled speckle. It is therefore feasible to find partial or complete images (i.e. hot spots) after the phase-modulation by the diffuser: i.e., the emergence of clear self-imaging of the object (i.e. hot spots) in the scrambled speckle background can easily be identified at different depths of the field.

Intriguingly, based on the proposed incoherent self-imaging model, the broadband illuminated speckles can be represented as an incoherent sum of speckles with self-imaging units at each single wavelength, which cannot be obtained using conventional methods [25, 29, 32,33,34,35]. According to previous studies on spectral-correlation of speckles [29, 41,42,43,44], the correlations and the contrast of broadband speckles are significantly reduced, leading to a severe dispersive blur in speckles. Based on conventional view of the diffuser scattering [25, 29], the diffuser was treated as a random complex Gaussian distribution phase-screen [38, 45]. Using this method, a fully-developed speckle is obtained as shown in Fig. 2(e) (see details in Note 6 in the supplementary material). One can see that the self-imaging is unpredicted for this single wavelength case, let alone the situation of broadband speckle with drastically increased blur. In contrast, inspired by our proposed new view of the diffuser, the self-imaging phenomenon in speckle can be predicted using Eq. 1, as shown in Fig. 2(f). Although this prediction was widely obtained in conventional imaging systems (e.g., using the system in Fig. 2(d) with the aid of pre-calibrated PSF) [14, 26, 27, 33, 35, 36], the self-imaging in speckle has never been predicted since the PSF itself was a chaotic pattern with no describable features (e.g. [14, 26, 27, 33, 35, 36]). Due to multiple-scattering and chaos in PSF’s speckle, increased blur in the final speckle can be predicated as a sum of speckle from every point of the object. But the self-imaging in broadband white-light speckle was considered infeasible (i.e., post processing is required as suggested by [26, 29, 33]). In contrast, here we treat the hot-spot in the speckle as the image of the point. As a result, calibration-free self-imaging in broadband speckle can be predicted using our proposed model, similar to the lens-imaging system for broadband illuminations. The resolution of the diffuser speckle-imaging system then is given as δx = λu/D = λu/(λv/Δx) = Δx/M, where M = v/u represents the magnification of the imaging system, and D represents the effective diameter of any limiting aperture stop placed between the scattering medium and the camera. In a nutshell, the resolution of the proposed system is limited by the resolution of the detector and the magnification of the system (see details in Note 5 in the SI). In addition, these hot-spots are strong in intensity, which can easily be resolved using regular cameras with no need of expensive scientific ones, as will be verified below.

Directly seeing through diffusers

In order to verify the theoretical and numerical predictions, here we collected speckles using a white light LED source (MCWHL7, Thorlabs) and an inexpensive color camera (CS165CU, Thorlabs, see experimental details in SI, Note 1). The raw image of a sample “S” is shown in the left column of Fig. 3(a) with no obvious information at the first glance using the naked eye. This is one of the major reasons that this phenomenon was missed in previously reported works. However, by zooming into a randomly selected region-of-interest (ROI), the self-imaging of the object can be observed! ROIs 1 and 2 are depicted in the central and right columns in Fig. 3(a), respectively. One can see many blurred self-imaging units in the speckle (see yellow boxes): i.e., the chaotic pattern of speckle is composed of random superpositions of these blurred self-imaging units and multi-scattering as the background. Intriguingly, this broadband self-imaging phenomena can also be observed directly for different objects and varied grits of diffusers (see more results in Note 2 in the SI). This experimental observation shows intuitive link between the object space and speckle pattern’s spatial information which was largely overlooked in precious studies. However, these self-imaging units are fairly blur (see details in Note 3 in SI), which is undesired for practical imaging applications. The next important question is how to enhance the contrast of these self-imaging units.

Fig. 3
figure 3

Self-imaging in white-light/single-wavelength speckles. (a) Raw speckle captured using broadband white light LED source. (b-c) Experimental results of imaging of different objects (b) digit ‘2’, and (c) digit ‘3’, through thin diffuser with narrow-band filter for high-contrast speckles. The ground truth object is shown in the bottom-left corner. Clearer view and abundant self-imaging units in speckle are directly observed by the naked eye, with no filtering pre-processing or post-processing utilizing computational reconstruction

According to incoherent imaging theory and speckle-spectral-correlation properties [25, 29, 32,33,34], high-contrast speckles can be obtained using narrow-band light source. This strategy is also applicable to our direct self-imaging system (see Note 2, and 3 in the SI). Remarkably, spectral imaging using speckles recorded at different filtered wavelengths (channels) can be efficiently obtained using self-imaging with no need of encoding-introduction and post-reconstruction (see Note 3 in the SI). Here we introduced a narrow-band LED through a narrowband filter (Andover, 633FS02-50, with the central wavelength of λ = 633 nm and the bandwidth of ~ 1 nm) in front of the camera. Two objects of digit ‘2’ and ‘3’ with the dimension of ~ 0.35 mm on the resolution test target were characterized with u = 60 cm and v = 10 cm, respectively. As shown in Fig. 3(b-c), self-imaging units with improved contrast were directly observed in the raw speckles. As shown in those zoom-in ROIs (last column in Fig. 3(b-c)), the composition of the speckle manifests itself as a random accumulation of self-imaging units. Compared with the results in Fig. 3(a), one can see more self-imaging units with clearer details due to the improved contrast of speckles. Intriguingly, this direct observation strategy with no need of post computational data process will enable real-time imaging of the targets through diffusers.

To demonstrate real-time imaging, we introduced a group of individual letters one by one (small objects with the dimension of ~ 0.2 mm, see the upper panel in Fig. 4(a)) and shoot a video by an ordinary color camera within the OME region (~ 0.84 mm, see video 1 in SI, and details for experimental setup). One self-imaging unit in the speckles is highlighted by the yellow circle in the central panel of Fig. 4(a) showing clear profile of the object. Intriguingly, for a given diffuser, the positions of self-imaging units for different objects are unchanged (see the lower panel in Fig. 4(a) and videos 1, and 2), since isoplanatic linear space-invariant PSF is maintained within the OME region (see Eq. 1). To further demonstrate real-time self-imaging of larger targets in the OME range, a letter ‘A’ with the dimension of ~ 0.6 mm was tested. To erase the salt-point-noise, denoised speckles are shown in Fig. 4(b) (using 2 × 2 pixels median-filtering of the raw-speckle). One can see self-imaging units of isolated letter ‘A’ with obviously high contrast. Furthermore, a target with three letters ‘EDU’ (with the total lateral dimension of ~ 0.75 mm) were also tested. For this relatively complicated target, incoherent superposition of isolated objects’ speckle is expected. As shown in the yellow box in Fig. 4(c), the complete self-imaging of multi-targets is observable. Importantly, since prior information of the diffuser-medium and the objects is unnecessary, one can obtain the self-imaging patterns of unknown objects in the scrambled speckle with no need of post computational reconstruction nor active wave control or precise modulation devices. Moreover, the object’s orientation information is directly seen and instantly tracked (see video 3 in SI for dynamic tracking results), which was unknown when reconstructed using phase retrieval (PR) (e.g. [19, 25, 34]). Further efficient processing of speckles to extract the image of the target can be realized using Gaussian filtering and/or homomorphic filtering, which is a robust way for image extraction (see details in Note 7 in SI). The quality of the directly observed images is even comparable to a majority of reconstruction results processed by SAI (e.g. [25, 30]), but was not better than the reconstruction with pre-calibrations or multi-frames-related techniques [28, 32,33,34, 46]. Therefore, post imaging processing is still useful to further enhance the quality of these self-imaging strategy, as will be explored below.

Fig. 4
figure 4

Real-time video imaging through a diffuser. (a) Real-time imaging of moving objects through random diffusers. (b) Imaging of isolated object of letter ‘A’, and (c) multitargets of letters ‘EDU’. Clear self-imaging images of multitargets are observed directly and instantly. The insets in the lower right inset shows the object, in which the red-dashed circles depict the OME region. The upper right inset shows the observed self-imaging unit. Object-distance and image-distance are u = v = 10 cm

To improve the quality of reconstructed images, refined structures of speckle’s autocorrelation are desired for PR processing [25, 30,31,32,33,34]. By refining the detailed structure of speckle’s autocorrelation (i.e., the detailed object’s autocorrelation, see Eq.5), refined detail information of the object in the frequency domain is therefore predicted according to the Wiener-Khinchin theorem [25, 38]. As a result, improved profiles of reconstructed images are expected using PR iteration. Here we propose a modified SAI method to reveal this potential (see details in Note 8 in the SI). In the speckle, clear units can be treated as in-focus images of the object, while other dimmed units are treated as the out-of-focus images of the object. By introducing intensity-extension with typical homomorphic filtering (HF) of the speckle, the information encoded in the high-intensity in-focus units and dimmed out-of-focus units are both considered, contributing to the refined structure of speckle’s autocorrelation. To verify this improved methodology, we tested the experimental speckles of ‘3’ obtained in Fig. 3(c) and compared the results of traditional SAI and the modified SAI method. First, the raw speckle of digit ‘3’ and its autocorrelation result are shown in Fig. 5(a) and (b), respectively. For the reconstruction process, a typical hybrid PR algorithm is used: i.e., a hybrid input-output (HIO) algorithm with a decreasing factor from 2 to 0 with a step of 0.02 and a loop of 50 iterations in each trial followed with classic error-reduction (ER) algorithm for another 50 iterations [25, 39, 40]. As a result, the image of the object is retrieved from a single-shot speckle using conventional SAI as shown in Fig. 5(c). In comparison, the imaging results processed using our modified SAI with HF are shown in Fig. 5(d-f). Specifically, Fig. 5(d) presents the speckle of Fig. 5(a) after HF. As shown in the zoomed-in ROI, besides those original high-intensity self-imaging units, more self-imaging units with lower-intensity in Fig. 5(a) are emerging in the speckle with HF. Remarkably, the autocorrelation of this new speckles (Fig. 5(e)) shows obviously more information than Fig. 5(b). Therefore, the reconstructed image in Fig. 5(f) shows sharper profiles of the object “3”. To quantitatively reveal the improved imaging quality of our modified SAI method, in Fig. 5(g, h), we directly compared the cross-line plots at the central vertical lines of the images in Fig. 5(b & e) and (c & f), respectively. One can see more details in speckle’s autocorrelation and sharper reconstruction profile, demonstrating the better performance of our modified SAI. Similarly, the comparison of the digit “2” obtained in Fig. 5(c) is shown in Note 8, Fig. S12 in SI.

Fig. 5
figure 5

Modified SAI reconstruction inspired by self-imaging. Experimental results of imaging through a thin diffuser using (a-c) classic SAI (Ref. [25]) and (d-f) improved SAI reconstruction methods. (a) Speckle for reconstruction. (b) Autocorrelations of speckle in (a). (c) The recovered image using the direct SAI method. (d) Image of speckle in (a) after homomorphic filtering processing. (e) Autocorrelations of speckle in (d). (f) The reconstructed results using (d) and (e). Scale bars in (c, f): 10 pixels. (g, h) The cross-line plots of the intensity of direct SAI (first row) and modified SAI (second row) at the central vertical line position (shown as the colored-dashed vertical line in (b-c) and (e-f)). AC: autocorrelation. HF: homomorphic filtering

Discussion

These results provide further understanding of the autocorrelation properties of speckle, and the features of diffuser-imaging system, representing a new efficient way for imaging recovery using speckles. A more detailed comparison in pros and cons with previously reported methods is listed in Table S1 in the supplementary information. In a nutshell, this proposed method shows fascinating capabilities with new potentials for imaging through scattering diffuser due to the narrowband/broadband, single-shot, real-time, calibration-free, computation-free, low-cost, and high-efficiency features using simple apparatus. In conclusion, this work reported a direct observation strategy through scattering diffusers using self-imaging effect that was largely overlooked by previous works. By reexamining inherent nature of speckles, a self-imaging speckle model was developed and validated using experiment, enabling visual understanding of the speckles and the statistic properties of speckle. Remarkably, a reconstruction-free strategy for real-time imaging of static and moving objects with their actual orientation information was demonstrated. This new technique requires no complicated or expensive devices for detection nor post computational reconstruction. The visibility of directly observed imaging patterns is equivalent to those processed with direct SAI (e.g. [25]). Furthermore, using a simply modified SAI method with efficient joint-filtering, the achieved imaging quality and resolution are comparable to the best results processed by previously reported computational reconstruction methods (e.g. [30,31,32,33,34,35,36,37, 47]). These results processed by the modified SAI show that the reconstructed images beyond the diffraction-limit can be achieved, which is obviously better than conventional SAI methods. Intriguingly, this directly observed self-imaging phenomenon indicates new potentials for emerging deep-learning-assisted imaging, design of physics-based or physics-aware intelligent technologies for computational imaging [48, 49], design of customized speckle [50], and will create new opportunities for bio-imaging (e.g., [2, 24, 25]), super-resolution imaging through thin scattering layers, and other novel imaging methodologies [51].

Methods

Experimental setup

The object in the USAF test target (1951USAF, Edmund) is used. Objects of letters and symbols are fabricated using laser etching of metallic masks. The target is illuminated from the back at varied distance u, by a collimated LED light source (Thorlabs, see details in the SI, for different sources used). The ground glass diffuser of a 220-grit (DG10-220-MD, Thorlabs) and a 600-grit (DG10-600-MD, Thorlabs) are used in the experiment. The speckle is recorded by a scientific camera (Zyla 5.5, Andor). As a comparison, a color camera (1440 × 1080 pixels, Pixel Size 3.45 μm, CS165CU CMOS Cameras, ThorLabs) is used to record the results of color speckles. Self-imaging observations in speckles are directly observed, for both narrow band and broadband illuminations. For further details of experimental setups and additional supporting results, see the Supplementary information.

Availability of data and materials

All data generated or analyzed during this study are included in this published article (and its supplementary information files). Additional data related to this paper may be requested from the authors.

Abbreviations

SAI:

Speckle autocorrelation imaging

PR:

Phase retrieval

PSF:

Point-spread-function

3D:

Three-dimensional

LED:

Light emitting diode

ROI:

Region-of-interest

OME:

Optical memory effect

HIO:

Hybrid input-output

ER:

Error-reduction

HF:

Homomorphic filtering

AC:

Autocorrelation

References

  1. Rotter S, Gigan S. Light fields in complex media: Mesoscopic scattering meets wave control. Rev Mod Phys. 2017;89:015005.

    Article  Google Scholar 

  2. Kang S, Jeong S, Choi W, Ko H, Yang TD, Joo JH, et al. Imaging deep within a scattering medium using collective accumulation of single-scattered waves. Nat Photonics. 2015;9:253–8.

    Article  Google Scholar 

  3. Lindell DB, Wetzstein G. Three-dimensional imaging through scattering media based on confocal diffuse tomography. Nat Commun. 2020;11:4517.

    Article  Google Scholar 

  4. Katz O, Small E, Silberberg Y. Looking around corners and through thin turbid layers in real time with scattered incoherent light. Nat Photonics. 2012;6:549–53.

    Article  Google Scholar 

  5. Si K, Fiolka R, Cui M. Fluorescence imaging beyond the ballistic regime by ultrasound-pulse-guided digital phase conjugation. Nat Photonics. 2012;6:657–61.

    Article  Google Scholar 

  6. Park C, Park J-H, Rodriguez C, Yu H, Kim M, Jin K, et al. Full-field subwavelength imaging using a scattering Superlens. Phys Rev Lett. 2014;113:113901.

    Article  Google Scholar 

  7. Newman JA, Webb KJ. Imaging optical fields through heavily scattering media. Phys Rev Lett. 2014;113:263903.

    Article  Google Scholar 

  8. Satat G, Heshmat B, Raviv D, Raskar R. All photons imaging through volumetric scattering. Sci Rep. 2016;6:33946.

    Article  Google Scholar 

  9. Escobet-Montalbán A, et al. Wide-field multiphoton imaging through scattering media without correction. Sci Adv. 2018;4:eaau1338.

    Article  Google Scholar 

  10. La Cavera S, Pérez-Cota F, Smith RJ, Clark M. Phonon imaging in 3D with a fibre probe. Light Sci Appl. 2021;10:91.

    Article  Google Scholar 

  11. Stellinga D, et al. Time-of-flight 3D imaging through multimode optical fibers. Science. 2021;374:1395–9.

    Article  Google Scholar 

  12. Li S, Horsley SAR, Tyc T, Čižmár T, Phillips DB. Memory effect assisted imaging through multimode optical fibres. Nat Commun. 2021;12:3751.

    Article  Google Scholar 

  13. Singh AK, Naik DN, Pedrini G, Takeda M, Osten W. Exploiting scattering media for exploring 3D objects. Light Sci Appl. 2017;6:e16219.

    Article  Google Scholar 

  14. Rai MR, Vijayakumar A, Rosen J. Extending the field of view by a scattering window in an I-COACH system. Opt Lett. 2018;43:1043–6.

    Article  Google Scholar 

  15. Popoff SM, et al. Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media. Phys Rev Lett. 2010;104:100601.

    Article  Google Scholar 

  16. Katz O, Ramaz F, Gigan S, Fink M. Controlling light in complex media beyond the acoustic diffraction-limit using the acousto-optic transmission matrix. Nat Commun. 2019;10:717.

    Article  Google Scholar 

  17. Jang M, et al. Wavefront shaping with disorder-engineered metasurfaces. Nat Photonics. 2018;12:84–90.

    Article  Google Scholar 

  18. Yeminy T, Katz O. Guidestar-free image-guided wavefront shaping. Sci Adv. 2021;7:eabf5364.

    Article  Google Scholar 

  19. Bertolotti J, et al. Non-invasive imaging through opaque scattering layers. Nature. 2012;491:232–4.

    Article  Google Scholar 

  20. Plöschner M, Tyc T, Čižmár T. Seeing through chaos in multimode fibres. Nat Photonics. 2015;9:529–35.

    Article  Google Scholar 

  21. Sun L, Shi JH, Wu XY, Sun YW, Zeng GH. Photon-limited imaging through scattering medium based on deep learning. Opt Express. 2019;27:33120–34.

    Article  Google Scholar 

  22. Li Y, Xue Y, Tian L. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica. 2018;5(10):1181–90.

    Article  Google Scholar 

  23. Seow KLC, Török P, Foreman MR. Single pixel polarimetric imaging through scattering media. Opt Lett. 2020;45:5740–3.

    Article  Google Scholar 

  24. Guo E, Zhu S, Sun Y, Bai L, Zuo C, Han J. Learning-based method to reconstruct complex targets through scattering medium beyond the memory effect. Opt Express. 2020;28:2433–46.

    Article  Google Scholar 

  25. Katz O, Heidmann P, Fink M, Gigan S. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations. Nat Photonics. 2014;8:784–90.

    Article  Google Scholar 

  26. Edrei E, Scarcelli G. Memory-effect based deconvolution microscopy for super-resolution imaging through scattering media. Sci Rep. 2016;6:33558.

    Article  Google Scholar 

  27. Antipa N, Kuo G, Heckel R, Mildenhall B, Bostan E, Ng R, et al. DiffuserCam: lensless single-exposure 3D imaging. Optica. 2018;5:1–9.

    Article  Google Scholar 

  28. Monakhova K, Yanny K, Aggarwal N, Waller L. Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter array. Optica. 2020;7:1298–307.

    Article  Google Scholar 

  29. Xu X, Xie X, He H, Zhuang H, Zhou J, Thendiyammal A, et al. Imaging objects through scattering layers and around corners by retrieval of the scattered point spread function. Opt Express. 2017;25:32829–40.

    Article  Google Scholar 

  30. Yang W, Li G, Situ G. Imaging through scattering media with the auxiliary of a known reference object. Sci Rep. 2018;8:9614.

    Article  Google Scholar 

  31. Guo C, Liu J, Wu T, Zhu L, Shao X. Tracking moving targets behind a scattering medium via speckle correlation. Appl Opt. 2018;57:905–13.

    Article  Google Scholar 

  32. Wu T, Katz O, Shao X, Gigan S. Single-shot diffraction-limited imaging through scattering layers via bispectrum analysis. Opt Lett. 2016;41:5003–6.

    Article  Google Scholar 

  33. Wu T, Dong J, Shao X, Gigan S. Imaging through a thin scattering layer and jointly retrieving the point-spread-function using phase-diversity. Opt Express. 2017;25:27182–94.

    Article  Google Scholar 

  34. Li W, Liu J, He S, Liu L, Shao X. Multitarget imaging through scattering media beyond the 3D optical memory effect. Opt Lett. 2020;45:2692–5.

    Article  Google Scholar 

  35. Xie X, Zhuang H, He H, Xu X, Liang H, Liu Y, et al. Extended depth-resolved imaging through a thin scattering medium with PSF manipulation. Sci Rep. 2018;8:4585.

    Article  Google Scholar 

  36. Okamoto Y, Horisaki R, Tanida J. Noninvasive three-dimensional imaging through scattering media by three-dimensional speckle correlation. Opt Lett. 2019;44(10):2526–9.

    Article  Google Scholar 

  37. Li X, Greenberg JA, Gehm ME. Single-shot multispectral imaging through a thin scatterer. Optica. 2019;6:864–71.

    Article  Google Scholar 

  38. Goodman JW. Speckle phenomena in optics: theory and applications: Roberts and Company (Englewood, Colorado).2007.

    Google Scholar 

  39. Fienup JR. Phase retrieval algorithms: a comparison. Appl Opt. 1982;21:2758–69.

    Article  Google Scholar 

  40. Shechtman Y, Eldar YC, Cohen O, Chapman HN, Miao J, Segev M. Phase retrieval with application to optical imaging: a contemporary overview. IEEE Signal Process Mag. 2015;32:87–109.

    Article  Google Scholar 

  41. van Beijnum F, van Putten EG, Lagendijk A, Mosk AP. Frequency bandwidth of light focused through turbid media. Opt Lett. 2011;36:373–5.

    Article  Google Scholar 

  42. Osnabrugge G, Horstmeyer R, Papadopoulos IN, Judkewitz B, Vellekoop IM. Generalized optical memory effect. Optica. 2017;4:886–92.

    Article  Google Scholar 

  43. Liu H, Liu Z, Chen M, Han S, Wang LV. Physical picture of the optical memory effect. Photonics Res. 2019;7:1323–30.

    Article  Google Scholar 

  44. Zhang R, Du J, He Y, Yuan D, Luo J, Wu D, et al. Characterization of the spectral memory effect of scattering media. Opt Express. 2021;29:26944–54.

    Article  Google Scholar 

  45. Haskel M, Stern A. Modeling optical memory effects with phase screens. Opt Express. 2018;26:29231–43.

    Article  Google Scholar 

  46. Liu Y, Chen L, Liu W, Liang X, Wan W. Resolution-enhanced imaging through scattering media by high-order correlation. Appl Opt. 2019;58:2350–7.

    Article  Google Scholar 

  47. Lu D, Xing Q, Liao M, Situ G, Peng X, He W. Single-shot noninvasive imaging through scattering medium under white-light illumination. Opt Lett. 2022;47:1754–7.

    Article  Google Scholar 

  48. Barbastathis G, Ozcan A, Situ G. On the use of deep learning for computational imaging. Optica. 2019;6:921–43.

    Article  Google Scholar 

  49. Luo Y, Zhao Y, Li J, Çetintaş E, Rivenson Y, Jarrahi M, et al. Computational imaging without a computer: seeing through random diffusers at the speed of light. eLight. 2022;2(1):4.

    Article  Google Scholar 

  50. Yılmaz H, Kühmayer M, Hsu CW, Rotter S, Cao H. Customizing the angular memory effect for scattering media. Phys Rev X. 2021;11:031010.

    Google Scholar 

  51. Sahu SP, Mahigir A, Chidester B, Veronis G, Gartia MR. Ultrasensitive three-dimensional orientation imaging of single molecules on Plasmonic Nanohole arrays using second harmonic generation. Nano Lett. 2019;19(9):6192–202.

    Article  Google Scholar 

Download references

Acknowledgements

We wish to thank Haifeng Hu for inspiring discussions.

Funding

This work was supported by the baseline of PSE, KAUST.

Author information

Authors and Affiliations

Authors

Contributions

Methodology, J.L, Q.G; J.L, W.Y and Q.G. conceived the experiments, and W.Y. and J.L. conducted the experiments. All authors analyzed the results. J.L, and Q.G wrote and reviewed the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Qiaoqiang Gan.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

43074_2022_80_MOESM1_ESM.docx

Additional file 1: Fig. S1. (Left) The experimental setup for the incoherent illumination diffuser imaging system. The Single-shot speckle from LED illuminated object behind a thin diffuser is captured by a camera. The object in the USAF-resolution test target is selected and controlled by an electrical displacement platform. (right) Experimental setup for rotating objects. Fig. S2. Experimentally captured speckles of different digital objects at different wavelengths. Digital objects from USAF test target are used for varied wavelength of incidence with (a) central wavelength 625 nm, (b) central wavelength 530 nm, and (c) central wavelength 450 nm. Experimental observations of self-imaging effects in speckles for different objects are verified under different experimental conditions. Original raw speckles of varied isolated object are enlarged for better vision. The bottom-left corners show the ground truth objects and the self-imaging unit picked out are shown with arrows. The circles superposed on the self-imaging units show the guide for the eye. The self-imaging phenomenon is observed straightforwardly in speckle. The original results are black and white, which are depicted with Pseudo-colors here for good seeing and comfortable look for (a). The original results for (b) and (c) are captured with the color camera, which are depicted with true colors. Fig. S3. Experimental observations of self-imaging in speckles with narrow-band LED illumination. Original raw speckles of isolated objects of (a) letter ‘A’, (b) object ‘’, (c) letter ‘k’, and (d) letter ‘S’, for diffusers of 220-grits. The ground truth objects are shown in the bottom-left corners. The self-imaging unit is observed straightforwardly in speckle. Fig. S4. Experimental observations of self-imaging in speckles with LED light illumination. Original raw speckles of varied isolated letters objects, for two types of diffusers. The ground truth objects are shown in the bottom-left corners. The self-imaging unit is observed straightforwardly in speckle. The circles superposed on the self-imaging units show the guide for the eye. Fig. S5. Experimental observations of self-imaging in speckles with LED through narrowband filters (a) at 530 nm, and (b) at 490 nm. Original raw speckles of varied isolated object letter ‘A’ for diffusers of 600-grits. The ground truth objects are shown in the bottom-left corners. The self-imaging is observed straightforwardly and clearly in speckle. Fig. S6. Experimental observations of self-imaging in speckles with broadband white light illumination. Camera captured original raw speckles of varied isolated object (a) letter ‘A’, and(b) letter ‘S’, for diffusers of 220-grits. The ground truth objects are shown in the bottom-left corners. The self-imaging unit is observed straightforwardly in speckle, where the white-circle superposed on the self-imaging units show the guide for the eye. Fig. S7. Experimental observations of self-imaging in speckles with broadband light illumination. Original raw speckles of isolated objects of (a) letter ‘A’, and(b) letter ‘S’, for diffusers of 600-grits. The ground truth objects are shown in the bottom-left corners. The self-imaging unit is observed straightforwardly in speckle, where the white-circle superposed on the self-imaging units show the guide for the eye. Fig. S8. Experimental observations of self-imaging in speckles with broadband light illumination. Captured original raw speckles of varied isolated object (a, c) letter ‘B’ and symbol ‘’ (b, d) for diffusers of 600-grits (first row) and 220-grits (second row). The ground truth objects are shown in the bottom-left corners. The self-imaging unit is observed straightforwardly in speckles, which is picked out and shown in the left-conner with yellow-box. The circles superposed on the self-imaging units show a guide for the eye. Table 1. Comparing the proposed method with the related speckle-imaging methods. Fig. S9. The diffuser imaging system. Fig. S10. The scheme of modified SAI and digital filter methods for improved self-imaging in speckle. Fig. S11. The block diagram for the phase-retrieval algorithm. Fig. S12. Experimental results of imaging of objects through thin diffuser using classic SAI (a-c) and improved SAI (d-f) reconstruction methods. (a) Speckle of digit ‘2’ for reconstruction. (b) Autocorrelations of speckle in (a). (c) recovery image using the direct SAI method. (d) homomorphic filtering processing of speckle in (a). (e) Autocorrelations of speckle in (d). (f) The reconstruction results using (d) and (e). Scale bars in (c, f): 10 pixels. (g, h) The cross-line plot of the intensity of direct SAI (first row) and modified SAI (second row) at the central vertical line position (shown as the colored-dashed vertical line in (b-c) and (e-f)). Supplementary Note 1. Experimental Methods. Supplementary Note 2. Single-wavelength LED source illumination, for two types of diffusers (220-grit, and 600-grit). Supplementary Note 3. Broadband white light LED experimental results for two types of diffusers (220-grit, and 600-grit). Supplementary Note 4. Comparison in pros and cons with previously reported methods. Supplementary Note 5. Spatial resolution of the proposed imaging method. Supplementary Note 6. Random diffuser design and simulation. Supplementary Note 7. Robust way to extract the image of the target without prior information. Supplementary Note 8. Phase-retrieval algorithm and reconstructed results using speckles of a digit ‘2’ (in Fig. 3(b) in the main text).

Additional file 2.

Additional file 3.

Additional file 4.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, J., Yang, W., Song, G. et al. Directly and instantly seeing through random diffusers by self-imaging in scattering speckles. PhotoniX 4, 1 (2023). https://doi.org/10.1186/s43074-022-00080-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43074-022-00080-2