Skip to main content
PhotoniX
Download PDF

Different channels to transmit information in scattering media

PhotoniX volume 4, Article number: 10 (2023) Cite this article

Abstract

A communication channel should be built to transmit information from one place to another. Imaging is 2 or higher dimensional information communication. Conventionally, an imaging channel comprises a lens with free space at its both sides, whose transfer function is usually known and hence the response of the imaging channel can be well defined. Replacing the lens with a thin scattering medium, the image can still be extracted from the detected optical field, suggesting that the scattering medium retains or reconstructs not only energy but also information transmission channels. Aided by deep learning, we find that unlike the lens system, there are different channels in a scattering medium: the same scattering medium can construct different channels to match the manners of source coding. Moreover, it is found that without a valid channel, the convolution law for a spatial shift-invariant system (the output is the convolution of the point spread function and the input object) is broken, and in this scenario, information cannot be transmitted onto the detection plane. Therefore, valid channels are essential to transmit information through even a spatial shift-invariant system. These findings may intrigue new adventures in imaging through scattering media and reevaluation of the known spatial shift-invariance in various areas.

Introduction

Communication is the basis of daily life and modern civilization. A channel is the path to transfer information from one terminal to another. How to transmit information through a channel optimally has been well studied in information theory [1, 2]. In the context, optimum means the obtained code could determine the information unambiguously, isolating it from all others in the set, and consists of a minimal number of symbols. It also provides methods to separate real information from noise and to determine the channel capacity required for optimal transmission conditioned on the transmission rate. Note that all information has to be transferred into temporal dimensional symbols first in information theory. Imaging is a kind of two or higher dimensional information communication. To enhance imaging capabilities, understanding of the imaging channel is essential. Conventionally, an imaging channel comprises a lens with free space or other light guides at its both sides. As the transfer function of each part of the optical path is usually known, the response of a conventional imaging channel can be well defined. If the lens is replaced with a thin scattering medium, such as a diffuser, the image can still be extracted from the detected optical field [3,4,5,6]. This suggests that the scattering medium retains or reconstructs transmission channels not only for energy (optical intensity) but also for information (imaging). Some researchers have explored the transmission channels in this process [7, 8], which, however, has been largely involved with energy only. Regarding the characteristics of information (imaging) transmission channels in a scattering medium, whether and how they are different from those in a conventional imaging system, none study, to the best of our knowledge, has been reported to date.

Deep learning has been widely applied in many fields including imaging through scattering media due to its data driven and physical-model free feature [9,10,11,12,13,14,15]. Usually, a trained neural network is only applicable to a particular scenario with limited generalization capability. In addition to serving as a pure image reconstruction method, we have demonstrated that deep learning can also be used as a tool to explore unknown principles in physics. With sufficient data for training, it can extract all information encoded in random speckles [16]. Hence, it is reasonable to use deep learning as a criterion to check whether there is information encoded in a pattern. Consequentially, if an image is successfully predicted from a pattern recorded on the receiving plane of a system, it means that there is valid channel in the system to transmit information; otherwise, no valid channel exists in the system. In this study, by designing an experiment consisting of 4 combinations of illumination sources and transmission media and utilizing deep learning as the criterion to justify whether there is information encoded in recorded patterns under the condition of sufficient training data for each case, we found that a scattering medium could construct different channels to transmit image information and its micro structures contributed a new type of channel. To further verify the discovery, it was hypothesized that image information could not be transmitted through a medium without such micro structure. We demonstrated the hypothesis with phase-grids. Moreover, it is a basic law that an output of a spatial shift-invariant system is the convolution of its point spread function (PSF) and the input target [17,18,19]. In other words, it is considered by default that information can be transmitted through a spatial shift-invariant system and is encoded in the detected pattern by convolution. However, we found that without valid channel under incoherent illumination, the convolution law was broken, the output of the spatial shift-invariant systems did not equal the convolution of the PSF and the target, and image information could not be transmitted through the systems.

Methods

Figure 1 shows the experimental setup used in this study. In Fig. 1a, a green laser (MGL-III-532-200 mW, Changchun New Industries Optoelectronics Tech) was expanded to illuminate a reflective digital micromirror device (DMD, V-7001 VIS, ViALUX) that has a pixel pitch of 13.7 μm. Handwritten digits from the MNIST database [20,21,22,23,24] were used as the objects, which were reshaped into 64 × 64 arrays and loaded on the central 64 × 64 pixels of the DMD in subsequence. The reflected light traveled through an aperture of a diameter of 10 mm and reached a digital camera (DCU224M, Thorlabs) with a pixel pith of 4.65 μm. Instead of a lens, the aperture was used to construct the channel for imaging, and diffraction patterns of input objects were recorded on the camera. Here, the aperture and the free spaces of its both sides built up the channel. Next, we inserted a homemade 220-grit ground glass diffuser into the optical path just next to the aperture, i.e., adding a random phase distribution caused by thickness and/or refractive index variations to the plane of the aperture, and the resultant speckle patterns were also recorded by the camera. The averaged speckle size is 13.3 μm, i.e., 2.8 times of the pixel size of the camera, to balance the sampling rate and the number of independent speckles of a recorded speckle pattern. Now, the diffuser and the free spaces at its both sides constituted the channel, and its effective diameter of the diffuser was limited by the aperture. The object distance from the DMD to the diffuser z1 = 16 cm, and the image distance from the diffuser to the camera z2 = 25 cm. Since the free spaces were identical in these two cases, for simplification whenever we mention channel below, it only refers to the aperture or the diffuser. To avoid geometry projections on the aperture and camera planes, the object loaded on the DMD cannot be too large.

Fig. 1
figure 1

Schematics of experimental setup. a Experimental setup to validate different channels in a scattering medium. The switch of channels is realized by inserting a rotating diffuser and a stationary diffuser into the optical path. b Experimental setup to demonstrate no valid channels in phase-grids, although they constitute shift-invariant systems and can convey light energy to the camera. The lower row shows the structures of the two phase-grids loaded on the SLM. d is the cell period, and a is the width of the phase square with a value of π in each cell. The color bar on the right denotes the phase value. The insert on the left is the zoom-in of the dash red square. Abbreviations: DMD - digital micromirror device; SLM - spatial light modulator

Full size image

Next, we inserted a rotating diffuser into the laser beam to create a pseudo thermal source, and repeated the aforementioned data acquisition step for the aperture and the diffuser, respectively. For the obtained 4 groups (laser light + aperture, laser light + diffuser, thermal light + aperture, and thermal light + diffuser) of data, each has 20,000 pairs of DMD objects and camera intensity patterns. Among them, 18,000 patterns were used to train and the rest 2000 to test a UNet neural network [16, 25,26,27]. From Ref. [16], it is known that with sufficient data to train the network, it can completely extract all image information encoded in the detected speckle patterns. Furtherly, it is suggested that whether image can be reconstructed is an indication of the existence of valid information transmission channel or not.

In the other grid experiment as shown in Fig. 1b, in order to eliminate the rain effect in the recorded patterns (as shown in Fig. 2 for the group of thermal light + aperture) caused by the rotating diffuser, a LED light (M530L4-C1, Thorlabs) coupled with a spectral filter (wavelength = 532 nm, line width = 10 nm, FL532-10, Thorlabs) was used as the illumination source. Different phase-grids were displayed on a spatial light modulator (SLM, LETO-2, Holoeye) that has 1920 × 1080 pixels (each pixel is 6.4 μm in length). The structures of the grids are illustrated in the lower row of Fig. 1b. Two phase-grid patterns were used to replace the diffuser in Fig. 1a. Parameters of the first grid pattern (Grid I) included a = 12.8 μm, b = 51.2 μm, and d = 64 μm, while the other pattern (Grid II) had a = 256 μm, b = 1024 μm, and d = 1280 μm. Due to the limited size of the SLM, the diameter of the grid patterns occupied 1000 pixels, corresponding to 6.8 mm, which was slightly smaller than the size of the aperture in Fig. 1a. The reflected light from the DMD first impinged on the SLM, then reached the camera, with which the PSFs of the system and the intensity patterns corresponding to different objects displayed on the DMD were recorded. A point consisted by 2 × 2 pixels was loaded on the DMD to obtain the corresponding PSF. The recorded intensity patterns were also used to train the UNet network [16], but no image information could be successfully extracted or reconstructed, which will be discussed in detail in the next section. For comparison, we also loaded a random phase map sharing the same statistical features with the diffuser onto the SLM, and then repeated the grid experiment. It should be noted that the recorded intensity patterns were first cropped into1024 × 1024 arrays, and then further down-sampled to 512 × 512 arrays for network training and testing to enhance the speed of deep learning.

Fig. 2
figure 2

Experimental results of different channels to transmit information under coherent and incoherent illumination. Two digits “5” and “8” are used as the ground truths. For each case, the recorded patterns and the corresponding reconstructed images are shown side by side. (Upper panels) Under coherent illumination, the recorded patterns after the aperture are the diffraction patterns of the input objects. The trained network can reconstruct the corresponding images. After inserting the diffuser, the camera records high contrast speckle patterns, and the aperture channel is encoded by the randomly distributed refractive index on the diffuser. Information can still be delivered to the detection plane and extracted by deep learning. (Lower panels) Under incoherent illumination, through the aperture, no image can be extracted from the pattern. But with the presence of the diffuser, images can again be well predicted from the speckle patterns, although the speckles are of lower contrast than those formed under coherent illumination. The rain effect on the recorded patterns, i.e., the pattern has an appearance of rain falling on a window, in the aperture case is due to the rotating of the diffuser to generate the pseudo thermal source

Full size image

The same UNet as used in Ref. [16] was adopted for image extraction in this study. The inputs of the network were preprocessed 512 × 512 recorded patterns, which was not necessarily speckle patterns. The outputs of the trained network were the reconstructed images with an array size of 512 × 512. The UNet network uses Python compiler based on Keras/Tensorflow 2.0 Library, and the GPU edition is a NVIDIA RTX 3060 laptop. The total number of training epochs is 50, and the learning rate at the beginning is set as 2 × 10− 4. After 5 epochs, if the loss function value does not decrease, the learning rate will be adjusted to one tenth of the previous one until the learning rate is reduced to 2 × 10− 6. If the loss function value does not decrease after 15 epochs, the training will be terminated.

For a spatial shift-invariant optical processing system, under incoherent illumination the output intensity pattern [28]

$$I\left(x,y\right)=\iint {\left|h\left(x-{x}^{\prime },y-{y}^{\prime}\right)\right|}^2{\left|E\left({x}^{\prime },{y}^{\prime}\right)\right|}^2{dx}^{\prime }{dy}^{\prime }.$$
(1)

It is a convolution of the input signal intensity |E(x′, y′)|2 with respect to the intensity impulse response |h(x − x′, y − y′)|2. Then, for an optical imaging system with spatial shift-invariance, the output [29,30,31]

$$I\left(x,y\right)=O\left(x,y\right)\ast PSF\left(x,y\right)={\int}_{-\infty}^{\infty }O\left({x}^{\prime },{y}^{\prime}\right) PSF\left(x-{x}^{\prime },y-{y}^{\prime}\right)d{x}^{\prime }d{y}^{\prime },$$
(2)

where ∗ denotes the convolution operation, O(x, y) is an intensity object and PSF(x, y) = |h(x, y)|2 is the PSF of the imaging system. Under incoherent illumination, the intensity pattern on the camera plane equals the convolution of the object and the PSF. That is to say, the shift-invariant system can provide valid channel, and information of the object is delivered through the channel and encoded in the detected pattern. However, the experiment of shift-invariant grid system tells a different story. Without valid channel to transmit information, the recorded pattern is not the calculated convolution of the object and the PSF, and no valid information is encoded in the recorded pattern.

In order to justify whether the recorded pattern and the calculated convolution pattern are equal, their structural similarity (SSIM) was calculated [32].

$$SSIM\left(x,y\right)=\frac{\left(2{\mu}_x{\mu}_y+{c}_1\right)\left(2{\sigma}_{xy}+{c}_2\right)}{\left({\mu}_x^2+{\mu}_y^2+{c}_1\right)\left({\sigma}_x^2+{\sigma}_y^2+{c}_2\right)},$$
(3)

where μx and μy are the averages, \({\sigma}_x^2\) and \({\sigma}_y^2\) are the variances of x and y, respectively, σxy is the covariance of x and y. c1 = (k1L)2 and c2 = (k2L)2 are the constants used to maintain stability and avoid division by zero [33], and L = 2Bit − 1 is the dynamic range of the pixel value. Generally, for 8 Bit data, the L value is 255. By searching for information, generally it is most suitable to compare pictures when k1 is set as 0.01 and k2 as 0.03 [34].

Simulation based on wave optics was also conducted to verify the hypothesis, considering that the simulation was noise free. The configuration and parameters were the same as those for experiment. In simulation, we created 500 frames of independent speckle illuminations on the object. For each speckle illumination, the propagation of complex field was calculated with the angular spectrum method, and the corresponding intensity distribution on the camera plane was recorded. The final intensity pattern was an average of all 500 frames of intensity distributions. It should bear in mind that the intensity pattern obtained from simulation is a response of a physical process, while the convolution of an object and a PSF is simply a mathematical operation. The mathematical operation is independent from the physical process. We can still implement the mathematical operation even when the physical process cannot happen.

Results

Under coherent illumination, the trained UNet can reconstruct the image well (Fig. 2), the aperture is a valid channel to transmit image information from the object plane to the camera plane. After inserting the diffuser, although the recorded patterns are seemly random speckles, images can still be predicted well by the network, which means that image information is still delivered onto the detection plane. That is to say, the channel functions properly. The main difference is that the refractive index distribution of the diffuser encodes the aperture channel, as also inferred from Ref. [16]. Under incoherent illumination, through the aperture channel, no image can be reconstructed by deep learning. Apparently, there is no image information in detected patterns, and the aperture channel fails in transmitting information. In this case the aperture is an invalid channel. However, after inserting the diffuser in, images can be extracted from recorded intensity patterns, which means that a new type of channel appears due to the diffuser to enable successful information transmission.

More specifically, there are different channels to transmit information in the scattering medium, which is different from conventional optical components, such as the aperture. Under coherent illumination, the whole space within the aperture composes the channel, and the random phase distribution introduced by the diffuser encodes the channel. Under incoherent illumination, the aperture channel is not valid anymore, since no image is successfully predicted from deep learning. However, after inserting the diffuser, images can be reconstructed, suggesting that there is information being received on the receiving plane. That is to say, a valid channel is constituted again. Is it the encoded aperture channel? The answer is No; if a channel is invalid, it is unlikely encoding will make it work. A reasonable guess is that there is a new channel or there are new channels. We believe that the new type of channel is formed by the micro structures of the diffuser.

Results of further experimental demonstration with phase-grids under incoherent illumination are shown in Fig. 3. As seen, when the point shifts, the PSF of the grid system shifts accordingly yet remains its shape in Fig. 3a. The correlation coefficient of two PSFs at positions y and y + ∆y is insensitive to the displacement ∆y. Apparently, the grid system is shift-invariant. In Fig. 3c, Row A shows the recorded PSFs when a point was displayed on the DMD in different systems (Grid I, Grid II, and diffuser). For a spatial shift-invariant system, the output on the camera plane for an object on the DMD should be the convolution of the intensity object and the shift-invariant PSF under incoherent illumination [28,29,30,31, 35, 36] according to the convolution law. But the recorded patterns in Row C (responses of real physical processes) are different from the calculated convolution patterns in Row B for the two grids, with significantly lower SSIM factors (see Table 1), comparing to the diffuser. In Row D, the image is successfully recovered by deep learning through the diffuser but not the phase-grids. The different results demonstrate the hypothesis that image information could not be transmitted through a medium without similar micro structures of the diffuser, which constitute the new type of channel under incoherent illumination. The results from deconvolution in Row E are consistent with Row D, which constitutes one more evidence. Rows F&G confirm that a convolution pattern contains image information by default, and deep learning can extract all information encoded in a pattern with sufficient data. From the comparison of Row D to Row F, we can say that without valid channel to transmit information onto the camera plane the recorded pattern contains no information of the target and does not equal the convolution pattern. In other words, the convolution law fails when there is no valid channel in the spatial shift-invariant system. We also know that images can be extracted by deep learning even when objects go beyond the memory effect ranges [37,38,39,40], i.e., the spatial shift-invariance doesn’t hold any more [35, 41, 42]. Apparently, channel is more essential in transmitting information.

Fig. 3
figure 3

Experimental results of grid and diffuser channels under incoherent illumination. a Recorded PSFs of grids 1&2 at different point positions, which confirm that the grid systems are shift-invariant. The correlation coefficient curves are flat over the scan range, which is already bigger than input objects. b Photograph of the input digit “5”. c Row A: the PSFs of different systems (Grid I, Grid II, and diffuser). From the zoom-ins we can see many fine structures. Row B: the calculated convolution of digit “5” and the corresponding PSFs. Row C: the corresponding recorded patterns by the camera. In the column for the diffuser, after contrast enhancement by the imadjust function in Matlab, fine structures, which cannot be seen in the original recorded pattern 1), become visible in 2). Rows D & E are extracted images from recorded patterns correspondingly by deep learning and deconvolution, respectively. DL is short for deep learning. Rows F & G are reconstructed images from the calculated convolution patterns by deep learning and deconvolution, respectively. The red dash square inserts show the zoom-in of original areas correspondingly. Compared to deconvolution operation, deep learning shows advantages in noise suppressing in reconstructed images

Full size image
Table 1 SSIM between the recorded pattern and the calculated convolution pattern
Full size table

Figure 4 shows the simulation results, in comparison to the experimental results in Fig. 3. Due to less noise, the patterns in both Rows B and C have better contrast to see their finer structures. For the diffuser, the two patterns look quite similar, while for the grids, such similarity disappears. It confirms that no valid channel existing in phase-grids to transmit image information to the camera plane, and there is no information in recorded patterns, which don’t equal the convolution of the object and the PSFs any more.

Fig. 4
figure 4

Corresponding simulation results of grid and diffuser channels under incoherent illumination. No image can be reconstructed from recorded patterns by either deep learning or deconvolution for both grids. There is no valid channel in phase-grids to transmit information, although the system constructed by it is shift-invariant. Row A: the PSFs of different systems (Grid I, Grid II, and diffuser). Row B: the calculated convolution of digit “5” and the corresponding PSFs. Row C: the corresponding recorded patterns by the camera. Rows D: images extracted from Row C via deep learning. Row E: images from Row C via deconvolution. Rows F: images from Row B via deep learning. Row G: images from Row B via deconvolution. The red dash square inserts show the zoom-in of original areas correspondingly, too

Full size image

In Table 1, the SSIM values of the two patterns for the diffuser are obviously bigger than the two grids. The reason for the extremely low SSIM = 0.0285 for grid II is that in simulation the convolution pattern has higher intensity around the center and a dark background while it is reversed for the intensity distribution in the recorded pattern. The diffuser provides valid channel to transmit information, so the recorded pattern can be considered as equal to the convolution of the input object and the PSF. The unique micro structures of the diffuser, rather than the structures of the grids, build the channel to transmit image information under incoherent illumination. The higher SSIM value in experiment is the result of the uniform backgrounds caused by ambient light in the recorded and calculated convolution patterns.

Discussions

One may argue that the wide spread of the PSF is the reason of the failure in image reconstruction for the aperture in Fig. 2, i.e., the resolution on the recorded plane is not sufficient to differentiate the structure of the image. If it is correct, we should be able to reconstruct images from the recorded patterns in both Figs. 3 and 4, since the fine structures of the PSFs of the grids provide the resolution to distinguish small details, as shown in the deconvolution results. In fact, we cannot extract any information about the object from the recorded pattern by both deep learning and deconvolution. Hence, the failure of imaging is due to lack of effective information in the recorded pattern but not the limited resolution.

Comparing with conventional methods such as wavefront shaping and scattering matrix measurement, deep learning provides a powerful tool to extract interested information and explore unknown principles instead of being distracted by speckle patterns in complex environments. A few more emphases should be added for the validity of using deep learning as a criterion to justify whether there is information concealed or encrypted in an intensity pattern. Owing to the data driven feature, deep learning has been widely applied in many areas, especially for situations when the physical mechanism of a system is unknown yet or too complicated to mathematically compute. In other words, the applicability of the deep learning method is not case sensitive, as long as sufficient dataset can be obtained to train and test the network, it is able to extract concealed or encrypted information. We also tested a generative adversarial network (GAN), which has different internal structures, for comparison. The same results and conclusion can be achieved, which confirms that deep learning is a valid criterion.

There are different ways for source coding. In imaging, coherent and incoherent illuminations provide two different means to encode object information. In coherent illumination, the information is encoded on wavefront, while in incoherent illumination, it is encoded on intensity. In order to effectively transmit image information, the channel should match the coding way of the source. The micro structures of the diffuser construct a new type of channel matched the incoherent encoding to efficiently transmit information. However, the model of the channel hasn’t yet been established. A possible model is a phase pinhole, since random phase distribution can be decomposed into numerous randomly distributed phase pinholes of different shapes and diameters [43], and each phase pinhole can image the object independently under incoherent illumination. On the other hand, channel bandwidth is critical to determine the quality of reconstructed image. Scattering media, like apertures, lenses and other optical components, construct imaging channels, and different channels in the medium has different capacities. Understanding imaging from the information angle may bring breakthroughs to some long pursued but yet unsolved problems.

Currently, all prevailing techniques of imaging through scattering media are trying to find the connection of an input and the corresponding output of a scattering medium. In scattering matrix measurement and speckle autocorrelation imaging, mathematic relations are given. In optical phase conjugation, an empirical relation based on optical reciprocity is the key. These relations are oversimplified with no internal process in physics. The scattering medium is a black box in all these techniques. We believe that cracking the black box and understanding what happens inside is the right route to eventually realize imaging through thick scattering media. The discovery of different channels is the first step to unveil the mask on information transmission in imaging through scattering media. It may intrigue new adventures on imaging through scattering media. Besides, the failure of convolution law for the shift-invariant grid systems also implies that there are deeper level mechanisms about the spatial shift-invariance unexplored.

Conclusions

In summary, deep learning is applied as a criterion to check whether there is information encoded in a pattern, and as a tool to completely extract information, if any, from the pattern to reconstruct an image. Aided by deep learning, we find that imaging channels is sensitive to illumination modes, which determine manners of source coding in optical imaging, and there are different types of channels in a scattering medium. Under coherent illumination, the channel is the whole space within the aperture while the index distribution of the medium inside the aperture encodes the channel. Under incoherent illumination, the encoded aperture channel is ceased, but a new type of channel formed by the micro structures of the scattering medium is activated to transmit information. It is further confirmed that the micro structures constructing the channels have their own characteristics, which are not shared by the phase-grids. Without valid channels, information cannot transmit through a medium even if it constitutes a shift-invariant system. These results refresh our understandings of scattering, imaging and spatial shift-invariance. It may also intrigue further investigation and applications of deep learning as a powerful tool to study unknown physical principles and/or mechanisms, modeling of transmission channels in scattering media, as well as deeper principles beneath the spatial shift-invariance.

Availability of data and materials

Data and materials are available from corresponding authors upon reasonable request.

Abbreviations

PSF:

Point spread function

DMD:

Digital micromirror device

LED:

Light emitting diode

SLM:

Spatial light modulator

SSIM:

Structural similarity

References

  1. MacKay DJC. Information theory, Inference and Learning Algorithms. Cambridge: Cambridge University Press; 2003.

    MATH  Google Scholar 

  2. Thomas M. Cover, elements of information theory. 2nd ed. Hoboken: Wiley; 2006.

    Google Scholar 

  3. Katz O, Heidmann P, Fink M, Gigan S. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations. Nat Photonics. 2014;8:784–90.

    Article  Google Scholar 

  4. Zhipeng Y, Li H, Zhong T, Park J-H, Cheng S, Woo CM, et al. Controlling light in complex media via wavefront shaping: a versatile tool to deal with scattering in multidiscipline. Innovation. 2022;3(5):623–37.

    Google Scholar 

  5. Mitsuo T, Singh AK, Narayana ND, Giancarlo P, Wolfgang O. Holographic correloscopy-unconventional holographic techniques for imaging a three-dimensional object through an opaque diffuser or via a scattering wall: a review. IEEE Trans Industr Inform. 2015;12(4):1631–40.

    Google Scholar 

  6. Joseph W. Goodman, speckle phenomena in optics: theory and applications. Englewood: Roberts & Company Publishers; 2007.

    Google Scholar 

  7. Yılmaz H, Hsu CW, Yamilov A, Cao H. Transverse localization of transmission eigenchannels. Nat Photonics. 2019;13:352–8.

    Article  Google Scholar 

  8. Yılmaz H, Hsu CW, Goetschy A, Bittner S, Rotter S, Yamilov A, et al. Angular memory effect of transmission Eigenchannels. Phys Rev Lett. 2019;123:203901.

    Article  Google Scholar 

  9. Li S, Deng M, Lee J, Sinha A, Barbastathis G. Imaging through glass diffusers using densely connected convolutional networks. Optica. 2018;5:803–13.

    Article  Google Scholar 

  10. Li Y, Xue Y, Tian L. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica. 2018;5:1181–90.

    Article  Google Scholar 

  11. Horisaki R, Takagi R, Tanida J. Learning-based imaging through scattering media. Opt Express. 2016;24:13738–43.

    Article  Google Scholar 

  12. Luo Y, Yan S, Li H, Lai P, Zheng Y. Towards smart optical focusing: deep learning-empowered dynamic wavefront shaping through nonstationary scattering media. Photon Res. 2021;9:B262–78.

    Article  Google Scholar 

  13. Sun L, Shi J, Xiaoyan W, Sun Y, Zeng G. Photon-limited imaging through scattering medium based on deep learning. Opt Express. 2019;27:33120–34.

    Article  Google Scholar 

  14. Lyu M, Wang H, Li G, Situ G. Learning-based lensless imaging through optically thick scattering media. Adv Photon. 2019;1(3):036002(1-10).

  15. Rawat S, Wendoloski J, Wang A. cGAN-assisted imaging through stationary scattering media. Opt Express. 2022;30:18145–55.

    Article  Google Scholar 

  16. Zhang X, Gao J, Song C, Zhang D, Zhuang S, Han S, Lai P, Liu H. Roles of scattered and ballistic photons in imaging through scattering media: a deep learning-based study. arXiv:2207.10263 [physics.optics]. https://doi.org/10.48550/arXiv.2207.10263.

  17. Häusler G, Lange E. Feedback network with space invariant coupling. Appl Opt. 1990;29:4798–805.

    Article  Google Scholar 

  18. Arguello H, Pinilla S, Peng Y, Ikoma H, Bacca J, Wetzstein G. Shift-variant color-coded diffractive spectral imaging system. Optica. 2021;8:1424–34.

    Article  Google Scholar 

  19. Arigovindan M, Shaevitz J, McGowan J, Sedat JW, Agard DA. A parallel product-convolution approach for representing depth varying point spread functions in 3D widefield microscopy based on principal component analysis. Opt Express. 2010;18:6461–76.

    Article  Google Scholar 

  20. Deng M, Li S, Zhang Z, Kang I, Fang NX, Barbastathis G. On the interplay between physical and content priors in deep learning for computational imaging. Opt Express. 2020;28:24152–70.

    Article  Google Scholar 

  21. Moralis-Pegios M, Mourgias-Alexandris G, Tsakyridis A, Giamougiannis G, Totovic A, Dabos G, et al. Neuromorphic silicon photonics and hardware-aware deep learning for high-speed inference. J Lightwave Technol. 2022;40:3243–54.

    Article  Google Scholar 

  22. Liu J, Chenghua F. MNIST data set recognition research based on TensorFlow framework. Int Core J Eng. 2021;7:410–4.

    Google Scholar 

  23. He Y, Duan S, Yuan Y, Chen H, Li J, Zhuo X. Semantic ghost imaging based on recurrent-neural-network. Opt Express. 2022;30:23475–84.

    Article  Google Scholar 

  24. Yan Zhang, Steve Farrell, Michael Crowley, Lee Makowski, and Jack Deslippe. A Molecular-MNIST Dataset for Machine Learning Study on Diffraction Imaging and Microscopy. Biophotonics Congress: Biomedical Optics 2020 (Translational, Microscopy, OCT, OTS, BRAIN), OSA Technical Digest (Optica Publishing Group, 2020), paper JTh2A.28.

  25. Zhang Z, Zheng Y, Tienan X, Upadhya A, Lim YJ, Mathews A, et al. Holo-UNet: hologram-to-hologram neural network restoration for high fidelity low light quantitative phase imaging of live cells. Biomed Opt Express. 2020;11:5478–87.

    Article  Google Scholar 

  26. Feng J, Deng J, Li Z, Sun Z, Dou H, Jia K. End-to-end res-Unet based reconstruction algorithm for photoacoustic imaging. Biomed Opt Express. 2020;11:5321–40.

    Article  Google Scholar 

  27. Deng J, Feng J, Li Z, Sun Z, Jia K. Unet-based for photoacoustic imaging artifact removal. Imaging and applied optics congress, OSA technical digest. Washington, DC: Optica Publishing Group; 2020. paper JTh2A.44.

  28. Yu FTS, Jutamulia S, Yin S. Introduction to Information Optics: Academic Press; US. 2001. p. 73–5. ISBN 978-0-12-774811-5.

    Google Scholar 

  29. Gureyev T, Nesterets Y, de Hoog F. Spatial resolution, signal-to-noise and information capacity of linear imaging systems. Opt Express. 2016;24:17168–82.

    Article  Google Scholar 

  30. Gureyev TE, Nesterets YI, de Hoog F, Schmalz G, Mayo SC, Mohammadi S, et al. Duality between noise and spatial resolution in linear systems. Opt Express. 2014;22:9087–94.

    Article  Google Scholar 

  31. Mahalanobis A, Vijaya Kumar BVK, Sims SRF. Distance-classifier correlation filters for multiclass target recognition. Appl Opt. 1996;35:3127–33.

    Article  Google Scholar 

  32. Kang S-J. SSIM preservation-based backlight dimming. J Display Technol. 2014;10:247–50.

    Article  Google Scholar 

  33. Bakurov I, Buzzelli M, Schettini R, Castelli M, Vanneschi L. Structural similarity index (SSIM) revisited: a data-driven approach. Expert Syst Appl. 2022;189:116087.

    Article  Google Scholar 

  34. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process. 2004;13(4):600–12.

    Article  Google Scholar 

  35. Liu H, Liu Z, Chen M, Han S, Wang LV. Physical picture of the optical memory effect. Photon Res. 2019;7:1323–30.

    Article  Google Scholar 

  36. Zhang R, Jinye D, He Y, Yuan D, Luo J, Daixuan W, et al. Characterization of the spectral memory effect of scattering media. Opt Express. 2021;29:26944–54.

    Article  Google Scholar 

  37. Scheibler S, Ackermann M, Malavalli A, Aegerter CM. Extending the field of view of imaging behind turbid media beyond the memory effect. OSA Continuum. 2019;2:1468–73.

    Article  Google Scholar 

  38. Guo E, Zhu S, Sun Y, Bai L, Zuo C, Han J. Learning-based method to reconstruct complex targets through scattering medium beyond the memory effect. Opt Express. 2020;28:2433–46.

    Article  Google Scholar 

  39. Levene M, Steckman GJ, Psaltis D. Method for controlling the shift invariance of optical correlators. Appl Opt. 1999;38:394–8.

    Article  Google Scholar 

  40. Silvera E, Kotzer T, Shamir J. Adaptive pattern recognition with rotation, scale, and shift invariance. Appl Opt. 1995;34:1891–900.

    Article  Google Scholar 

  41. Yanny K, Monakhova K, Shuai RW, Waller L. Deep learning for fast spatially varying deconvolution. Optica. 2022;9:96–9.

    Article  Google Scholar 

  42. Horisaki R, Tanida J. Multi-channel data acquisition using multiplexed imaging with spatial encoding. Opt Express. 2010;18:23041–53.

    Article  Google Scholar 

  43. Liu H, Lai P, Gao J, Liu Z, Shi J, Han S. Alternative interpretation of speckle autocorrelation imaging through scattering media. Photonic Sensors. 2022;12(3):220308.

    Article  Google Scholar 

Download references

Acknowledgements

We thank Dr. Guohai Situ for helpful discussions.

Funding

The work was supported by National Natural Science Foundation of China (NSFC) (81930048), Guangdong Science and Technology Commission (2019A1515011374, 2019BT02X105), Hong Kong Research Grant Council (15217721, R5029-19, C7074-21GF), and Hong Kong Innovation and Technology Commission (GHP/043/19SZ, GHP/044/19GD).

Author information

Authors and Affiliations

  1. Key Laboratory for Quantum Optics, Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai, 201800, China

    Xuyu Zhang, Jingjing Gao, Yu Gan, Chunyuan Song, Shensheng Han & Honglin Liu

  2. Engineering Research Center of Optical Instrument and System, The Ministry of Education, Shanghai Key Laboratory of Modern Optical Systems, University of Shanghai for Science and Technology, Shanghai, 200093, China

    Xuyu Zhang, Dawei Zhang & Songlin Zhuang

  3. Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing, 100049, China

    Jingjing Gao, Yu Gan, Chunyuan Song, Shensheng Han & Honglin Liu

  4. Hangzhou Institute for Advanced study, University of Chinese Academy of Sciences, Hangzhou, 310024, China

    Shensheng Han

  5. Department of Biomedical Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, China

    Puxiang Lai

  6. Hong Kong Polytechnic University Shenzhen Research Institute, Shenzhen, 518000, China

    Puxiang Lai & Honglin Liu

  7. Photonics Research Institute, The Hong Kong Polytechnic University, Hong Kong SAR, China

    Puxiang Lai

Authors
  1. Xuyu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jingjing Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Gan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chunyuan Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dawei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Songlin Zhuang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Shensheng Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Puxiang Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Honglin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

H. L. conceived the idea and designed the experiment and simulation. X. Z., J. G., Y. G., and C. S. implemented the experiment and simulation. X. Z., P. L., and H. L. analyzed the data. All contributed in writing and revising the manuscript. The author(s) read and approved the final manuscript.

Corresponding authors

Correspondence to Dawei Zhang, Puxiang Lai or Honglin Liu.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

All authors approve the publication.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Gao, J., Gan, Y. et al. Different channels to transmit information in scattering media. PhotoniX 4, 10 (2023). https://doi.org/10.1186/s43074-023-00087-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s43074-023-00087-3

Keywords