Harnessing disordered photonics via multi-task learning towards intelligent four-dimensional light field sensors

Abstract

The complete description of a continuous-wave light field includes its four fundamental properties: wavelength, polarization, phase and amplitude. However, the simultaneous measurement of a multi-dimensional light field of such four degrees of freedom is challenging in conventional optical systems requiring a cascade of dispersive and polarization elements. In this work, we demonstrate a disordered-photonics-assisted intelligent four-dimensional light field sensor. This is achieved by discovering that the speckle patterns, generated from light scattering in a disordered medium, are intrinsically sensitive to a high-dimension light field given their high structural degrees of freedom. Further, the multi-task-learning deep neural network is leveraged to process the single-shot light-field-encoded speckle images free from any prior knowledge of the complex disordered structures and realizes the high-accuracy recognition of full-Stokes vector, multiple orbital angular momentum (OAM), wavelength and power. The proof-of-concept study shows that the states space of four-dimensional light field spanning as high as 1680=4 (multiple-OAM) $\times$2 (OAM power spectra) $\times$15 (multiple-wavelength) $\times$14 (polarizations) can be well recognized with high accuracy in the chip-integrated sensor. Our work provides a novel paradigm for the design of optical sensors for high-dimension light fields, which can be widely applied in optical communication, holography, and imaging.

Introduction

Complex light field generation, modulation and detection are the cornerstone of modern optics and photonics research. The rise of metamaterials and metasurfaces using artificial subwavelength structures to tailor light-matter interactions has created unprecedented capability of light field manipulation either in the free space or on-chip circuits [1,2,3,4,5]. In particular, metasurface conventionally fabricated on an ultra-thin interface via nanofabrications, modulates the wavefront of light through spatially varying geometric structures [6, 7]. Since the optical response of a metasurface is engineered locally addressing large degrees of freedom, it has become a mainstay platform for harnessing the intrinsic high-dimension of light, such as amplitude, phase, polarization and wavelength, etc [8,9,10]. In contrast to the fast development of light field manipulation, the integrated optical detection systems for multi-dimensional light fields are only emerging recently [11,12,13], although they have important applications in optical communications, biomedical imaging and autonomous vehicles [12].

The conventional photodetectors only measure light intensity, and the intrinsic high-dimension of light collapse in the simply photo-electric conversion process. By leveraging the optical response of functional materials/microstructures, the novel integrated light-field sensors are developed, capable of resolving light polarization [14, 15], wavelength [16], incident angle [17] or orbital angular momentum (OAM) [18], etc. On the other hand, the direct integration of metasurface with photodetectors (arrays) for high-dimension light detection is attracting great attention [11, 19,20,21,22,23]. Nevertheless, so far these multi-dimensional light-field sensors require sophisticated designs and precise fabrications, and nearly all of them are optimized for no more than three featured properties of light [12,13,14, 16, 19,20,21,22,23], which are fundamentally limited by their internal degrees of freedom in the light transmission and detection systems [24,25,26]. Besides, the signal cross-talks in the measurements of high-dimension light due to the wave dynamics rapidly increase as the increase of detection freedom, and this challenges the conventional deterministic mapping or demultiplexing method based on the physics-driven models.

Light scattering by disordered media is a ubiquitous phenomenon that can be seen everywhere in life, such as sunset glows. These disordered structures of high internal degrees of freedom can support numerous transmitting channels of light. Although the light field can be evidently scrambled by the multiple scattering in disordered media, the optical information is not lost and thus contained in a speckle pattern since the scattering is elastic [27,28,29]. Basically, the input light field can be recovered from these speckle patterns if the transmission matrix is determined. However, when the system becomes large and complex (multiple scattering) and is further coupled with the high-dimension light, it will be challenging to solve the inverse problem from the conventional transmission matrix method [29]. In this work, we propose the concept of integrated four-dimensional light field sensors by harnessing light scattering in disordered structures via multi-task-learning (MTL) deep neural network (DNN) [30, 31] as illustrated in Fig. 1. The disordered nematic liquid crystals (NLC) forming schlieren textures via self-assembly provide more degrees of freedom of transmitting channels for light and generate speckle patterns. We reveal theoretically and experimentally that these speckle patterns are intrinsically correlated to the input high-dimension light field, i.e., polarization, wavelength, OAM and power, from the structural similarity (SSIM) index [32]. Furthermore, by leveraging a trained MTL-DNN, we demonstrate an intelligent four-dimensional light field sensor capable of simultaneously extracting the full-Stokes parameters, multiple-wavelength, multiple-OAM and optical power from the single-shot speckle patterns by a CMOS image chip. This data-driven and deep learning method overcomes the shortcomings of conventional light field decoding systems based on physical theories or numerical modelings since the signal processing can be automatically tracked to the optimal sensing data via a user-defined cost function [13, 33].

Fig. 1

Schematic of integrated four-dimensional light field sensors via disordered nematic liquid crystal (NLC) medium, imaging chip and deep learning. The input high-dimensional light field of various wavelength, polarization, orbital angular momentum (OAM) and power, are scattered and encoded by a disordered NLC film, forming diversified speckle patterns and being captured in a CMOS imaging chip. The speckle image data can be regarded as states in a n-dimension vector space. The multi-task-learning (MTL) deep neural network (DNN) as a decoder maps the measured vector to the high-dimension space of light field including OAM, wavelength, power and polarization, through a user-define cost function

Results and discussion

Properties of speckle patterns correlated to high-dimension of light

The explicit calculation of light propagation and scattering in disordered NLC film is complex and can not provide clear physics intuition (see Methods). Here we use a simplified model of square NLC lattices of random director distributions (Fig. 2a), and calculate the far-field speckles as shown in Fig. 2b (Supplementary Section 2). The lattice constant of 20 $\upmu$m is determined from the autocorrelation function of measured NLC director distributions (Supplementary Fig. S1). We implement SSIM index (Supplementary Section 3) to quantitatively evaluate the correlations of the speckle patterns to high-dimension light field [32]. The larger SSIM difference of speckles among different input light fields indicates higher optical-field sensitivity of disordered media.

Fig. 2

Principles of light field interactions with disordered NLC medium. a Simplified model composed of randomly LC directors arranged in a square lattice. Inset: Schematic of in-plane director angle $\theta$ relating to the x-axis. b Simulated in-plane random distributions of NLC directors (left panel), and their far-field diffracted speckle patterns (right panel) with Gaussian beam input (waist radius of 1 mm). The ordinary refraction index and extraordinary refraction index of NLC medium are set as 1.517 and 1.741, respectively; and the NLC film thickness is set as 20 $\upmu$m. The scale bar is 2 mm. c Calculated structural similarity (SSIM) index of speckle patterns for various linear-polarization light input in a range of disorder strength ($\varepsilon$). d Calculated SSIM contrast for various disorder parameter ($\varepsilon$) under linear-polarization light input. e,f Simulated and experimental results of SSIM index of speckle patterns for different polarized light input (e) and different OAM-state input (f). The light wavelength is fixed as 660 nm. g Simulated and experimental results of SSIM for speckle patterns under different light wavelength input

We first explore the influence of disorder strength on light-field sensing. The simulated NLC film is characterized by the disorder parameter $\varepsilon \ (0<\varepsilon \le 1)$, and it indicates the range of the random rotation angle of NLC director $\theta _{\varepsilon }(x,y)$, which obeys a uniform distribution function $f_{\varepsilon }(\theta )=\{1/\varepsilon \pi , 0<\theta \le \varepsilon \pi ; 0,$ otherwise} [34] (Supplementary Fig. S2). In Fig. 2c, we show the calculated SSIM of speckles as a function of the linear polarization angle of light under different disorder parameters. When $\varepsilon =0$, the orientation of the NLC directors is homogeneous, hence the element acts as a uniform medium and cannot respond to light polarization via SSIM. By incrementally increasing the disorder, the similarity between simulated speckle patterns is decreased gradually in a certain range of polarization angle, since the wavefront of input light is spoiled. The SSIM contrast, defined as the maximal SSIM change in a parameter space, is monotonously increased with the disorder strength and is greatest near the critical point ($\varepsilon =1$), as shown in Fig. 2d. A similar trend of SSIM on the disorder strength is found for OAM and wavelength parameters (Supplementary Fig. S3 and S4). We note that the fabricated NLC film approximately corresponds to a fully disordered structure ($\varepsilon \approx 1$, Supplementary Fig. S1).

Figure 2e shows the simulated (upper panel) and measured (lower panel) SSIM of speckle patterns under different polarized light incidences. The SSIM of speckles corresponding to a polarized light input is minimal when its linear polarization angle is 90 deg compared to the reference polarization state of 0 deg. The SSIM fluctuates with the change of input polarization states because the similarity among the speckles is fundamentally related to the distance of polarization states on Poincaré sphere (Fig. S5). Figure 2f shows the statistical response of speckle patterns with different OAM modes input, i.e. topological charge $l=\pm 1,\pm 3,\pm 5$. Since the helical-phase front of OAM modes is easily distorted by the anisotropic inhomogeneity of NLC film, SSIM rapidly decreases with the change of OAM modes numerically and experimentally. As for the light wavelength dependency, numerical results in Fig. 2g (upper panel) show that SSIM is monotonously decreased with the increase of wavelength, which is consistent with experimental measurements. Basically, the local phase retardation for transporting light in NLC medium can be written as $2\pi (n_o-n_e)d/\lambda$, where n_o and n_e are the ordinary index and extraordinary index, respectively; d is the film thickness, $\lambda$ is the light wavelength. Consequently, the far-field diffracting patterns shaped by disordered NLC medium depend on the input light polarization, phase distribution, wavelength and power theoretically.

Overall, from the SSIM index evaluations, the speckle patterns from a disordered NLC film are strongly correlated to the input light field properties. Nevertheless, it would be challenging to differentiate a multi-dimensional light field of multiple properties, by simply statistically analyzing the speckle patterns. First, the statistical features, such as SSIM index, can neither have single-valued mapping nor a monotonous relation to a single optical property of light. Regarding the infinite number of states for polarizations and OAM of light, their corresponding relations to SSIM make it impracticable to conduct the pre-calibrations of SSIM as indicated from Fig. 2e, f. Second, the random nature of NLC director distributions introduces strong coupling and mixing effects for the multi-dimensional light field input. For example, the multi-wavelength and multi-OAM states are constructively encoded in the diffracting patterns, and thus the deterministic one-to-more mapping simply from SSIM index is nearly impossible.

Methodology of MTL algorithms

To tackle these challenges, here we use deep learning methods to process the as-imaged speckle patterns in no need of knowing the physical properties of devices in advance, supporting a transferable architecture for the multi-dimensional light field sensing, as shown in Fig. 1. We adopt a hard-parameter sharing MTL-DNN architecture (Supplementary Section 6), which consists of a shared encoder and two task-specific heads. A multi-layer convolutional neural network (CNN) plays the role of shared encoder to extract general feature maps from the speckles, branching out into two independent multi-layer fully-connected layers for classification and regression, respectively. The classification head is implemented to recognize the discrete values of OAM modes and wavelengths, while the regression one predicts the continuous values of the power and polarization states of light.

It is well known that the form of loss function has a great impact on the power of training DNN models. In this study, we introduce the idea of homoscedastic uncertainty to adaptively decide the values of weights. The loss function of an MTL-DNN is constructed as the weighted linear sum of the losses for each individual task, which is sensitive to each task via the settings of weights [35]:

$$\begin{aligned} \mathcal {L}\left( \sigma _1,\sigma _2\right) =\frac{1}{2\sigma _1^2}\mathcal {L}_1+\frac{1}{{\sigma }_2^2}\mathcal {L}_2+\log {\sigma _1}+\log {\sigma _2} \end{aligned}$$

(1)

where the regression loss function $\mathcal {L}_1$ and the classification one $\mathcal {L}_2$ are weighted by observation noise parameters $\sigma _1$ and $\sigma _2$, respectively; $\sigma _1$ and $\sigma _2$ are used to capture relative confidence between tasks and tune the weights of the single-task losses during training processes (Supplementary Section 6). Since the relative confidence reflects the homoscedastic uncertainty of prediction outputs for each task, the program tends to suppress the effect of the task with high uncertainty, but increase the weight of the task with low uncertainty. This strategy makes the learning process more balanced as well as effective.

In the following sections, we separately optimize the MTL-DNN model for three intelligent sensing functions, i.e., full-Stokes vector sensing, multiple-OAM sensing and four-dimensional light field sensing.

Characterizing the performance of intelligent full-Stokes polarimetry

Polarization is an intrinsic characteristic of electromagnetic waves that describes the geometric path traced by the wave’s electric field vector [36]. The conventional polarization measurement in free space is mainly based on the division-of-amplitude or division-of-time approach [20, 23]. The Stokes vector is widely used to describe the polarization behavior of light, mainly because polarization ellipse is an amplitude description of polarized light and cannot be directly measured [37]. In this section, we demonstrate intelligent full-Stokes polarimetry by leveraging DNN. Theoretically, when a coherent light beam passes through the disordered NLC film, its polarization state and optical power are implicitly encoded in the speckle patterns. Because of the continuous values of the Stokes vector (polarization state) and power, we employ the regression head of the MTL-DNN model to decipher the optical properties of unknown light beams. The predicted label vector $(S_0, S_1, S_2, S_3)^T$ of four Stokes parameters are constructed, where $S_1$, $S_2$ and $S_3$ are obtained at the output layer of the regression head, and $S_0$ is calculated by the equation $S_0=(S_1^2+S_2^2+S_3^2)^{1/2}$ considering the coherent light input. The datasets of captured 20,000 speckle images are divided into three subsets, i.e., training set (80%), validation set (10%) and testing set (10%). After training procedures, we plot the normalized Stokes vector for different input/predicted polarizations of various optical power as shown in Fig. 3a. Figure 3b further plots the projected light polarization of test results onto the $S_1/S_0-S_2/S_0$, $S_2/S_0-S_3/S_0$ and $S_1/S_0-S_3/S_0$ planes for better clarity and error analysis. The results show that the predicted Stokes vectors are in good agreement with the input parameters. Figure 3c shows the predicted optical power $S_0$ as a function of the input values, and the average error for the predicted power of light compared to the measurement is about 5%. Figure 3d shows the relationship between the loss function and the number of training epochs. With the increase in the number of iterations for the training dataset, the loss is decreased significantly and converged to around 0.005. We define the error on the Stokes vector reconstruction as $\sigma =\textrm{arccos}(({S_1S^{'}_1+S_2S^{'}_2+ S_3S^{'}_3})/({S_0S^{'}_0}))$, where $S_i$ and $S^{'}_i$ (i=0,1,2,3) are input and predicted Stokes vector [23]. It is clear that $\sigma$ corresponds to the angle between the input and the reconstructed Stokes vector on the Poincaré sphere. Therefore, the proposed intelligent full-Stokes polarimetry is capable of reconstructing the light polarization with a high accuracy and the average deviation angle $\left\langle {\sigma }\right\rangle$ is as small as $7^{\circ }$, which is comparable to the state-of-the-art results of computational polarimetry [15, 23].

Fig. 3

Full-Stokes polarimetry by leveraging DNN. a Polarization states from the DNN predicted results (solid circles) and their corresponding input values (hollow circles) are plotted in the Poincaré sphere. The symbols’ colors differentiate the optical power for the input and predicted states. b The results from (a) are projected onto the normalized $S_1-S_2$, $S_1-S_3$ and $S_2-S_3$ planes for better clarity and analysis. c Predicted power ($S_0$) as a function of incidence power. The red area corresponds to a 5% error in power measurements. d The loss function versus epochs for the training and testing datasets

Characterizing performance of intelligent multiple-OAM detection

Light with helical phase wavefront featuring exp($il\varphi$) (where l and $\varphi$ represent the topological charge and azimuthal angle, respectively) carries OAM, which has boosted advanced optical applications in communications, holography encryption, tweezers, high-order quantum entanglement and nonlinear optics [38,39,40,41,42]. The measurement of the topological charge of the OAM beam is a nontrivial task for many optical applications. The conventional measurements of topological charge are based on the deterministic interference and diffraction properties of OAM modes, which often require a precise alignment of the optical path and are difficult for compact integration [39, 43]. Recently, the speckle-based deep learning methods for OAM mode classification are being developed, and they have found applications in the OAM-multiplexed data transmission [44,45,46,47,48]. However, it is still challenging to realize simultaneous recognition of multiple-OAM states and their power spectra of different OAM channels, mainly limited by the single-task learning configuration, although the measurement of multiple-OAM spectra is crucial for high-capacity information processing systems [49, 50].

Here we demonstrate a novel MTL-based intelligent multiple-OAM sensing system to classify the multiple-OAM states and estimate their power spectra as shown in Fig. 4a. The major components of MTL-DNN consist of a four-layer CNN and two independently task-specific fully-connected layers (FCs) for the classification of OAM modes and the regression of their power spectra. In essence, the feature maps extracted from the input speckle images via the four-layer CNN are shared by the task-specific FCs; meanwhile, the task-specific FCs are used for learning a nonlinear mapping from features to individual targets. Several dropout layers are embedded between every two three-layer FCs to prevent overfitting and improve the generalization ability.

Fig. 4

Intelligent multiple-OAM states sensing. a Schematic of multiple-OAM state detection empowered by MTL-DNN. The multiple-OAM state composing three weighted OAM modes is produced via a superposed phase plate. The multiple-OAM state is scattered by the disordered NLC device, and the generated speckle images are input to the MTL-DNN. The multiple-OAM states and their power spectra are recognized by two independently task-specific fully-connected layers. b The loss function on the training epochs numbers for train and test datasets of multiple-OAM states: (i) regression loss function $\mathcal {L}_1$; (ii) classification loss function $\mathcal {L}_2$ in semi-log axis; (iii) total loss function $\mathcal {L}$. c The confusion matrix of 20 multiple-OAM states labeled 1-20, reporting accuracy of 100%. The right panel lists the OAM mode compositions for the different multiple-OAM states. d Four typical input and predicted spectra of multiple-OAM states. The predicted results agree reasonably well with the input spectra

Experimentally, the calculated phase profile is loaded into a spatial light modulator (SLM), and the diffracted multiple-OAM beam (Supplementary Section 5, Supplementary Fig. S7) is then launched to the disordered NLC medium to generate speckle patterns in the far-field regime. To reduce the performance degradation caused by discrete phase pattern and phase-only modulation on generated collinearly multiple-OAM beams, we design OAM base with a state interval of 2 for proof-of-concept demonstration [51]. We recorded 25,000 speckle images covering 20 different multiple-OAM states along with various power spectra. In the training stage, different multiple-OAM light field information and their corresponding speckle images are used to train the MTL-DNN model. Labeled samples are randomly split into training, validation and testing subsets in the ratio of 8:1:1. A MTL loss function is well-defined to adaptively balance the weights of two tasks during the training procedure. After the optimization process, the network is able to recover the multiple-OAM states information directly from a single speckle pattern. From Fig. 4b, it shows that the loss function of the regression and classification tasks converge to $\sim 3\times 10^{-3}$ and $\sim 1\times 10^{-8}$, respectively. And thus the total loss function of the MLT-DNN for multiple-OAM recognition is $\sim$0.02. Figure 4c shows the truth table for multiple-OAM states classification, and the recognition accuracy of twenty kinds of multiple-OAM states reaches 100%. In addition, Fig. 4d shows a comparison between the reconstructed OAM power spectra and the input results. It can be found that all the components of predicted results agree well with the input values. As far as we know, the proposed intelligent light-field sensor is for the first time demonstrated to realize the multiple-OAM spectra resolution, which will stimulate applications in OAM-based holography, imaging and communications.

Intelligent single-shot reconstruction of four-dimensional light fields via integrated sensors

In this section, we discuss the potential of using disordered NLC film and MTL-DNN to achieve the complete single-shot reconstruction of realistic four-dimensional light fields including polarization, multiple-OAM, multiple-wavelength, and power. The disordered NLC medium of high degrees of freedom provides rich scattering channels to the high dimension of light, which lays the foundation for direct single-shot recognition. We assemble the disordered liquid crystal cell and commercial imaging chip for proof-of-concept demonstration, as shown in Fig. 5a. The high-dimension light fields generated from the home-made optical system (Fig. 5b, see Methods) are directly imaged and decoded in the integrated sensors. We would like to note that the integrated sensing scheme significantly reduces the light propagation distance of the millimeter-scale and influence the diffracted speckle patterns compared with that of long-distance diffraction of tens of centimeter-scale (Supplementary Fig. S9), while their predicted results via the MTL-DNN are comparable (Supplementary Fig. S10). The decoding of high-dimensional properties of light from the single-shot speckles is not a trivial task considering the strong linear and nonlinear coupling in the light scattering and detection process. To cope with this difficulty, we continue to use the MTL-DNN model for previous multiple-OAM detection while making slight modifications in its output layers to meet specific requirements. We apply linear extrapolation and impose random noise on the expanded datasets. The signal-to-noise ratio (SNR) is defined as $\text {SNR}=-10\lg {({\mu }/{I_s})}$, where $\mu$ and I_s represent the variance of noise and intensity of original measured image. Here, random noise with a SNR of 5-15 dB is imposed to generate a total of 120,000 speckle images based on the original 336 measured images. The extrapolation is appropriate based on two methods: (1) The speckle images are scaled linearly with the excitation power; (2) Polychromatic speckles are considered as a linear superposition of monochromatic speckles. These techniques are widely used in machine learning studies when a larger training set was desired [15].

Fig. 5

Demonstration of integrated four-dimensional light field sensors. a Photograph of integrated light field sensors composing a liquid crystal cell and imaging chip. The inset is the crossed-polarized optical image of the disordered liquid crystal film, showing schlieren textures. The black arrows are the polarizer and analyzer directions. The scale bar is 150 $\upmu$m. b Schematic of the experimental setup. SLM: spatial light modulator, AOTF: acoustic-optical tunable filter, SC laser: super-continuum laser. c,d The typical input (left panel) and predicted results (right panel) of multiple-OAM modes in combinations of four polarizations ($P_1$, $P_4$, $P_7$, $P_9$) at the wavelength of (c) 650 nm and (d) 660 nm. The labeled light wavelength is predicted by the MTL-DNN rather than predetermined. e,f The typical input (left panel) and predicted results (right panel) for the poly-chromatic light field in combinations of four polarizations ($P_1$, $P_4$, $P_7$, $P_9$) for (e) multiple-OAM state 1 and (f) multiple-OAM state 2. These polarization states $P_i$ are illustrated in Fig. 2e. The input power spectra of multiple-OAM state 1 are $l=-1$ (20%), 1 (50%), 3 (30%); the input power spectra of multiple-OAM state 2 are $l=-3$ (50%), -1 (30%), 3 (20%). The labeled multiple-OAM state is predicted by the MTL-DNN rather than predetermined. g The reconstruction error of polarization, power, wavelength and multiple-OAM on the signal-to-noise ratio (SNR)

After training MTL-DNN model, the loss function stops at $\sim$0.08 with classification loss of $\sim$0.03 and regression loss of $\sim$5$\times 10^{-3}$ (Fig. S8). Using the trained model, the input multi-dimensional light fields can be completely recovered from single-shot speckle images. Figure 5c-f show the comparison of input and predicted results of 16 typical states of high-dimension light fields composing different combinations of polarization, multiple-OAM, multiple-wavelength and intensity, which clearly indicate the high recognition accuracy of the deep learning network. Since it is not intuitive to simultaneously illustrate the predicted results of such four properties of light, we have to fix one property of light and plot the remaining dimensions. To further validate the effectiveness of the proposed intelligent sensor, the mean squared error (MSE) between the input value and predicted value are implemented. The reconstruction error of polarization, power, wavelength and multiple-OAM as a function of SNR for high-dimension light is shown in Fig. 5g. The results indicate that the light-field sensor can realize high-dimension light field detection with high fidelity at a relatively high noise level (SNR $\ge$10 dB). In the current proof-of-concept study, the states space of a four-dimensional light field is as high as 1680, spanning by 4 multiple-OAMs, 2 OAM-power-spectra and 15 single-/multiple-wavelength and 14 polarizations, which can be well recognized with high accuracy. The states space can be readily extended (Supplementary Section 8) if more experimental data are included in the training network. As far as we know, this is the first demonstration of complete light field construction of four universal degrees of freedom, i.e., wavelength, polarization, OAM, and power via an integrated sensor, which provides a new paradigm shift in the complex light-field manipulations and detections.

Conclusions

In summary, we have demonstrated an integrated intelligent four-dimensional light field sensor enabled by disordered anisotropic medium and deep learning. Our results unambiguously show that the complete information of light including wavelength, polarization, OAM, and power can be decoded simultaneously via a single-shot speckle image from the disordered anisotropic medium. Compared with the conventional light field sensing systems by cascading numerous bulky opto-electro-mechanical elements along the optical path, this developed concept of the intelligent optical sensor provides a disruptive technology for the ultimate multi-dimensional light field detection. The current disordered NLC film is fabricated via self-assembly, and we envision that for practical applications, the photo-patterning technique can be implemented to generate arbitrarily-designed mesostructured NLC film with high repeatability [52, 53]. On the other hand, the developed speckle-based light field-sensing configuration can also be transferred to the multimode fiber scheme [46, 54, 55], and the extended degrees of freedom for high-dimension light encoding and decoding with ultrahigh data capacity can be realized. Furthermore, it is anticipated to construct a photonic neural network to implement machine learning, paving the way for on-chip all-optical light field sensing.

Methods

Device fabrications

The nematic liquid crystal (NLC) film (E7, Jiangsu Hecheng Display Technology) is sandwiched between two glass plates coated with indium-tin-oxide. The film thickness is controlled by spacers with a thickness of 20 $\upmu$m.

Optical measurements

The spatial director distributions of the NLC film is measured by a commercial birefringence imaging microscope (Crystalent-50, Ningcui). The multi-dimensional light field is generated by a home-built optical system, which consists of a wavelength-tunable module, a phase modulation module and polarization tuning module. The wavelength-tunable module is a super-continuum laser (YSL SC-Pro) filtered by an acoustic-optical tunable filter (AOTF, YSL AOTF-Pro bandwidths $\sim$4 nm). The spatial light modulator (SLM, UPOlabs HDSLM80R-Plus) is used to control the phase wavefront of the beam. The polarization state of light is modulated by a $\lambda /2$ plate and a $\lambda /4$ plate. The diffracted multi-dimensional light field in disordered NLC film is captured by a home-built imaging system (MIchrome 5 Pro, Shanghai Taizi Technology).

Light propagation in anisotropic disordered medium

We consider the Helmholtz equation that governs light propagation for the electric field $\vec {E}_\omega (\vec {r})$:

$$\begin{aligned} -\nabla \times \nabla \times \vec {E}_\omega (\vec {r})+k^2_0 (\varvec{\varepsilon }^r+\varvec{\varepsilon }^s(\vec {r})) \vec {E}_\omega (\vec {r})=0 \end{aligned}$$

(2)

where $k_0$ is the vacuum wave number and dielectric function $\varvec{\varepsilon }(\vec {r})=\varvec{\varepsilon }^r+\varvec{\varepsilon }^s(\vec {r})$ is embedded in an infinite homogeneous reference medium $\varvec{\varepsilon }^r$. When an incident field $\vec {E}^0_\omega$ is considered, the scattering problem can be treated by setting up equivalent relations for the incident field that satisfies: $-\nabla \times \nabla \times \vec {E}^0_\omega +k^2_0 \varvec{\varepsilon }^r \vec {E}^0_\omega =0$ in the homogeneous and source-free reference system. Introducing the operators $\varvec{L}$, $\varvec{e}_r$, and $\varvec{e}_s$ for simplifying the notation $-\nabla \times \nabla \times$, $k^2_0 \varvec{\varepsilon }^r$, and $k^2_0 \varvec{\varepsilon }^s(\vec {r})$, respectively, we can rewrite the Eq. 2 as:

$$\begin{aligned} (\varvec{L}+\varvec{e}_r+\varvec{e}_s)\vec {E}_\omega (\vec {r})=0 \end{aligned}$$

(3)

The incident light field $\vec {E}^0_\omega$ in the homogeneous and source-free reference system satisfies:

$$\begin{aligned} (\varvec{L}+\varvec{e}_r)\vec {E}^0_\omega =0 \end{aligned}$$

(4)

The Green’s tensor $\varvec{G}(\vec {r}, \vec {r}^{\prime }, \omega )$ describes the impulse response of a given physical system. This dyadic tensor is closely related to the field susceptibility of the system. The Green’s function $\varvec{G}$ associated with the complete system is defined by

$$\begin{aligned} (\varvec{L}+\varvec{e}_r+\varvec{e}_s)\varvec{G}=\delta (\vec {r}-\vec {r}^{\prime })\varvec{I} \end{aligned}$$

(5)

where $\varvec{I}$ is the unit tensor. By setting Eq. 3 equal to Eq. 4 and using Eq. 5, we obtain the following formula:

$$\begin{aligned} \vec {E}_\omega (\vec {r})=\vec {E}^0_\omega -\varvec{G}\varvec{e}_s\vec {E}^0_\omega \end{aligned}$$

(6)

We define the generalized propagator as $\vec {K}(\vec {r},\vec {r}^{\prime },\omega )=1-\varvec{G}\varvec{e}_s$. Thus the scattered field of Eq. 6 in the $\vec {r}$ representation becomes [56]:

$$\begin{aligned} \vec {E}_\omega (\vec {r})=\int d \vec {r}^{\prime } \vec {K}(\vec {r},\vec {r}^{\prime },\omega ) \vec {E}^0_\omega (\vec {r}^{\prime }) \end{aligned}$$

(7)

Data preparation

The color images collected by the camera will be converted into gray values according to the formula $I(x,y) = 0.3I_R(x,y) +0.59I_G(x,y)+ 0.11I_B(x,y)$, where $I_R$, $I_G$, and $I_B$ are red-, green-, and blue-channel of images, respectively. The converted and subsampled (408 $\times$ 342 pixels) images are imported into the deep-learning network for training.