Dual-plane coupled phase retrieval for non-prior holographic imaging

Introduction Accurate detection of wave propagation in spatial and temporal domains is essential for the investigation of wave–matter interactions. Holographic imaging records the entire complex-amplitude wavefront by interference [1, 2], becoming a powerful technique of quantitative phase imaging. Holographic recording and replaying of waves have been widely applied in various fields, such as quantitative phase measurements for stain-free biological cells [3–7], wavefront shaping [8, 9], particle analysis and 3D tracking [10, 11], etc. Two main holographic configurations exist, i.e., off-axis and in-line holography. Off-axis holography can achieve separation of the object wave and conjugate wave by introducing a tilted reference wave. It is widely used in phase microscopic imaging and instruments [12–16]. The maximum bandwidth of the sample should be less Abstract

than a half of the bandwidth of the detector [17]. Nevertheless, the in-line configuration is affected by the fact that the main ray of the object wave and its conjugate wave share the same optical path. The full bandwidth utilization of the detector can be achieved but the superposition of twin images interferes with the quantitative amplitude and phase map. It has been applied in live cell observation [18,19] and the imaging of whole pathological sections [20][21][22][23]. Several strategies have been reported for extracting the object phase including single-plane phase retrieval [24][25][26][27][28], multi-image phase retrieval [22,23,[29][30][31][32][33][34][35][36][37][38][39][40][41] and non-iterative method [42][43][44][45][46]. The single-plane method carries out an iterative procedure between the measured plane and the estimated plane. It requires prior constrains of the object to escape local convergence in the iterative procedure. The multi-distance phase retrieval (MPR) method can achieve high-fidelity and stable reconstruction at the expense of measuring many observation planes through axial scanning [30,31,[34][35][36][37][38][39][40][41]. Objectbased constraints can be transferred to detection-based constraints without prior assumption. The accuracy of MPR highly depends on the number of measurements. When the paraxial approximation is satisfied, the distribution of the object's diffraction field at different distances is approximately a transversal elongation effect; thus, the hologram at another distance can be obtained by stretching the hologram [47].
Phase retrieval (PR) solves the inverse problem of nonconvex optimization under the known constraints with an iterative procedure [48]. The convergence of nonconvex optimization suffers from phase-stagnation problem with a limited initial guess. The support prior, sparsity assumption, non-negative assumption or random modulation can help to escape local convergence [49][50][51][52][53][54]. The phase solution of transport of intensity equation (TIE) between multiple planes can be considered as the initial phase guess to assist the convergence of iterative reconstruction [22,55,56]. Accurately solving the TIE initial-phase (TP) estimation requires imposing boundary conditions and certain approximations [57]. Higher convergence speed can be obtained by using off-axis optimization phase in the hologram plane while prior knowledge and non-negative constraints still need to be imposed on the object [49,[58][59][60][61]. Here, we propose a two-plane coupled phase retrieval (TwPCPR) method to achieve stable and versatile spatial complex-amplitude reconstruction. No constraint or prior knowledge of the object is needed in the proposed approach. Two in-line holograms at two distances z 1 and z 2 and one off-axis hologram at z 1 (or z 2 ) are recorded with the hybrid optical setup. It realizes a fast-convergence holographic calculation method.
Essentially, all the constraints of the TwPCPR method are derived from measurements without any prior constraints of the object, making it expand the ability to become a wavefront sensor. Off-axis optimization phase provides an effective initial guess to avoid stagnation and minimize the required measurements of the MPR method. Two imaging sensors capture holograms simultaneously to avoid the mechanical shift. We demonstrate the superiority of the proposed method by reporting imaging reconstruction of a variety of objects, i.e., an amplitude resolution target, phase resolution targets and biological samples, using both numerical simulations and experiments.

The optical setup coupled DH system
The geometrical configuration of the lensless hybrid holographic system is shown in Fig. 1(a), and it also includes microscopic mode, respectively. The light source is a He-Ne laser with a wavelength of 632.8 nm. The illumination plane wave is divided into two beams by using beam splitter (BS) 1. The imaging sensor chip has the resolution of 1200×1920 with a pixel size of 5.86 μm (QHY174, QHYCCD Co., Ltd.). The total field of view (FOV) of this lensless system is 7.032×11.251 mm 2 . Two sensors are used to detect holograms simultaneously without mechanical shift. The system includes an inline device and an off-axis device. They are combined by BS2 and switched by a shutter. In the lensless system, the sample is placed in front of BS2, and the reference wave illuminates from its adjacent surface. The distance between the object and the sensor can be adjusted by changing the size of the BS. In the microscopic system, the parameters of the microscope objective (MO) are 10× Mitutoyo Plan Apo infinity corrected objective with 0.28 NA (MY10X-803, Thorlabs). The MO is combined with a tube lens (TL) to form a microscopic imaging system. The focused positions magnified object can be adjusted by axially moving the sample. The sample is placed before the MO and the focused image can be directly obtained by the detectors.

The TwPCPR principle
The distribution of the hologram is I = | P(O)+ R| 2 . O and R are the distributions of the object and reference waves, respectively, and P is the propagation vector from the object plane to the hologram plane. The MPR method receives N holograms as input, and the phase field is initially unknown. The complex-amplitude field is solved to make the current estimate closely fit the input actual captured holograms, and the quantification is given by the objective function as follows [48]: where ‖. ‖ is the Euclidean norm, I k is the intensity of the k th captured hologram, and O is the current estimated function corresponding to the object. Operator P k is the transmission process from the object plane to the k th hologram plane. N is the number of detections. The process provides a relatively correct solution in early iterations but then overshoots, which can often be attributed to the nonconvex characteristic of phase retrieval [48]. The convergence depends on more known detection-based constraints. The lateral displacement error accumulates between each hologram as the number of measured holograms increases. Fewer constraints are desired to reduce the accumulation of errors, but having a weak convergence of iteration. Efficient sub-pixel registration can be achieved in the lateral displacement between different holograms [62][63][64].
The in-line and off-axis recording modes can be switched by the presence or absence of a reference wave by using the shutter, where the reference wave is R x, y = A exp i 2π x sin (θ x ) + y sin θ y . Here, θ x is the angle between the projection of the reference wave illumination direction on the x-z plane and the z axis, and θ y is the angle between the projection of the reference wave illumination direction on the y-z plane and the z axis. The process of the TwPCPR method is shown in Fig. 1b1. Two in-line holograms at z 1 and z 2 and one off-axis hologram at z 1 (or z 2 ) are obtained The TwPCPR process, where ① is the in-line recording at distance z 1 , ② is the in-line recording at distance z 2 , and ③ is the off-axis recording at distance z 2 . b2 Bandwidth limitation of holography in normalized coordinates, where Δ is the pixel size of the camera, NA is numerical aperture, and λ is wavelength. b3 Reconstructed Peak Signal-to-Noise Ratio (PSNR) for different holographic imaging methods. c Reconstructions for different holographic imaging methods for the reconstruction. Half of irradiation energy is lost before the camera by using BS. The bandwidth limitations of different holographic methods are shown at the bottom of Fig. 1b2. The numerical aperture (NA) of the system determines the optical bandwidth of the object wave. The reconstruction is also limited by the sensor pixel size based on the Nyquist theorem. The highest frequency in the horizontal or vertical direction of the wave collection in detector can be described as min{1.22NA/ (Mλ), 1/(2Δ)} under the coherent diffraction limit [3], where M is the magnification, and Δ is the pixel size of the camera. One-dimensional is considered for analysis here.
4∆ , the object wave can be completely separated from its conjugate wave and autocorrelation (AC) term in the spatial frequency domain (SFD). If 2∆ , the object wave can be separated from its conjugate wave, but it overlaps with its AC term to form a slight off-axis hologram [65]. For lensless imaging systems, the off-axis bandwidth requirement is usually not satisfied (See supplementary). However, an estimated low-frequency complex-amplitude envelope of the sample can be extracted from it. In Fig. 1b3 and c, an object is assumed with a spectrum of full camera's bandwidth. 2Δ is the smallest detection period of the camera based on the Nyquist theorem. The in-line reconstruction is obtained by using back-propagation method. Full bandwidth reconstruction can be achieved but twin image is superimposed on the reconstructed object. The slight off-axis reconstruction is the inverse Fourier transform after filtering the 1 st order spectrum under the circular pupil function with the radius of a quarter of camera's bandwidth. Twin image is almost removed, but high-frequency details are missing and some artifacts remain in the object due to incomplete-bandwidth reconstruction. A faithful reconstruction of the TwPCPR method is shown as a result of combining the advantages of the two holographic configurations. It works for more extended samples without any prior assumptions in the object plane.
The specific optimization process is described by the following steps: Step 0: Initialization. The size of each hologram is M 1 × M 1 pixels. The in-line holograms I 1 and I 2 are located at object-detector distances z 1 and z 2 , respectively. The off-axis hologram is located at z 1 (or z 2 ). In the MPR method, the initial phase φ 0 are set to constant or random. In the TwPCPR method, the initial phase φ 0 comes from off-axis hologram. The diffraction wave is filtered in the SFD of the off-axis hologram. First, the off-axis hologram is performed by Fourier transform. Second, the 1 st order spectrum is filtered by using a pupil function. The radius of the pupil function is one-sixth of the bandwidth for the camera (See supplementary). Thirdly, the inverse Fourier transform is performed in the filtering spectrum and the phase is initial phase φ 0 . Then, the initial wave field at distance z 1 is Step 1: The complex-amplitude field U 2 = t 2 exp(iφ 2 ) at the distance z 2 is obtained from the complex-amplitude field U 1 by using the angular spectrum propagation method [2].
Step 2: The amplitude of U 2 is replaced with the square root of the intensity of the hologram I 2 . The phase value is maintained. Operator P z 1 −z 2 represents the function of propagation from distances z 1 and z 2 . The complex-amplitude field is obtained: Step 3: The complex-amplitude field U 1 = t 1 exp (iφ 1 ) is obtained from the complexamplitude field U 2 by using the angular spectrum propagation method.
Step 4: The amplitude of U 1 is replaced with the square root of the intensity of the hologram I 1 . The phase value is maintained. The complex-amplitude field U 1 is obtained: The next iteration is calculated through steps 1-4. Then, back-propagation to the object plane is performed to achieve the reconstruction from the complex amplitude U 1 or U 2 . The distances z 1 and z 2 are calculated by using the auto-focusing algorithm [10,66]. The interval between two hologram planes can be determined in this system.

Samples for testing
Three types of samples are considered: an amplitude USAF-1951 resolution target, two phase resolution targets and two biological tissues, respectively. The amplitude USAF target is fabricated by etching lines onto a glass plate using nano processing methods. The phase object is the Quantitative Phase Microscopy Target (QPMT, Benchmark Technologies). It is a completely transparent measurable artifact which provides a wellcharacterized optical path difference for quantitative phase imaging. The optical path difference is achieved by increasing the thickness of pattern on the glass plate. It is made of an acrylate polymer (The refractive index is 1.52) on Corning Eagle XG Glass. The phase USAF-1951 resolution target includes group 6-10 with the feature height of 250 nm. The other QPMT is an optically focus star on simulate circular structures with the feature height of 250 nm. The third type of sample includes two biological sections, one of which is a muscle cell section on the glass slide. It includes the transverse and longitudinal morphology of the skeletal muscle cells mainly composed of muscle fibers. The other is the mitotic section of horse roundworm fertilized egg section. It is drawn from the cross-sectional of the uterus of horse roundworm. The uterus contains cells at various stages of division.

Results and discussion
The TwPCPR method Figure 2 shows the reconstruction of complex objects by using the proposed method and the MPR method. For the MPR method, constant initial phase and TIE initial phase (TP) . are both considered. The simulation parameters are as follows: the wavelength is 500 nm, the pixel pitch is 6 μm, and the propagation distances of the three planes are 4.00, 4.05, and 4.10 cm, respectively. Figures 2a1 and a2 show the amplitude and phase of the object 1. Figure 2  The object 1 is centrally isolated in the image FOV. The hologram can be approximately regarded as the flat boundary, which is recovered regardless of the selected boundary conditions [57]. The universal solution is used to obtain the TP here [68]. The 2-MPR and 3-MPR method remain the crosstalk between the amplitude and phase. A higher convergence speed and better reconstructed quality can be achieved by increasing the sampling plane in the MPR method [31,67]. The usual expectation is a reduction in the complexity with fewer detection planes but higher reconstructed quality. The TP can speed up convergence of MPR method and it effectively avoids the stagnation in the MPR method [22,56,57]. The TP and the off-axis initial guess perform initial input to minimize the required measurements. The cross-sections of the reconstructed amplitude and phase are shown below Fig. 2c1-d4, respectively. The phase errors of TP originate from the nonlinear components related to the finite difference approximation [55,56]. The estimation using TP provides limited convergence assistance in this parameter of the system. Figure 2f1 and f2 show the amplitude and phase of the object 2 without flat boundary. Figure 2(g) is one of the three in-line holograms of the original scene. The whole diffraction field with 750×750 pixels is shown. Figure 2h1-h4 and i1-i4 are the reconstructed amplitude and phase by using the 2-MPR, 3-MPR, 3-MPR-TP and TwP-CPR method with 20 iterations, respectively. The TwPCPR method shows the advantage of parameter-free of the imaging with insensitivity to the interval between different planes, boundary conditions, object assumptions and paraxial approximation. It is reflected in the reconstructed MSE of the amplitude and phase with different numbers of iterations in Fig. 2e1, e2, j1 and j2, respectively. The proposed TwPCPR method can achieve a lower MSE than those of the MPR methods after a few iterations.  Figure 3b1 is one of the raw in-line holograms. The distance between the two diffraction planes of two sensors is 2.5 mm. For the MPR method, the distance interval between different hologram planes is 0.5 mm. Figure 3b2 is the raw offaxis hologram. Figure 3c1 is the reconstruction from the in-line hologram. It is the result from Fig. 3b1 by using back-propagation to the object plane. Figure 3c2 is the reconstruction from the off-axis hologram. The radius of the pupil function is one-sixth of the camera's bandwidth (See supplementary). The advantages and disadvantages of the two types of holographic imaging are clearly shown. The in-line hologram provides a highresolution reconstruction but fails to give a correct estimate of the object, while the offaxis hologram has the opposite function. The in-line reconstruction in Fig. 3c1 shows the best resolvable feature with all of group 5 (line width from 8.77 to 15.6 μm) and even in group 6, element 3 (line width 6.2 μm, close to the pixel size). The off-axis reconstruction in Fig. 3c2 shows the best resolvable feature with group 4, element 4 (line width 22.1 μm). The in-line reconstruction makes full use of the bandwidth of the entire camera to achieve the extreme resolution but with twin image. Figure 3(d) is the reconstruction achieved by using the TwPCPR method with only 5 iterations. Figure 3e1-e4 are enlargements of Fig. 3(d) by using the 2-MPR, 3-MPR, 3-MPR-TP and TwPCPR method, respectively. The TwPCPR method outputs contrast-enhanced images in which the artifacts are remarkably removed. The resolution of this system is close to the pixel size of the detector. The best resolvable feature corresponds to group 6 element 4 of USAF 1951 (6.2 μm line width). The minimum distance between the object and the sensor is the width of the BS.

Reconstruction for biological section
To further demonstrate the capability of complex-amplitude reconstruction, an experiment on skeletal muscle cells is performed in Fig. 4. The transverse and longitudinal morphology of the skeletal muscle cells are prepared in one specimen. The whole specimen can be detected in the area of sensor. Figure 4(a) shows the muscle cells on a glass slide. Figure 4b1 is the raw measured in-line hologram. Figure 4b2 is the raw off-axis hologram. Figure 4(c) shows the amplitude-reconstruction achieved by using the TwP-CPR method with 5 iterations. Figures 4d1 and d2 are the reconstructed amplitudes by using the 3-MPR and 3-MPR-TP with 50 iterations, respectively. Figures 4(e)-(f ) are the corresponding phase, respectively. The reconstructed phase is calculated by the phase unwrapping algorithm [69]. Observation field by using TwPCPR method includes the transverse and longitudinal morphology of the muscle cells. They can be clearly distinguished from the reconstructed amplitude and phase. Twin images are covered in the detailed texture in Fig. 4f1-f2. The reconstructions achieved by using the TwPCPR method show that it eliminates the noise artifacts and accomplishes contrast-enhanced reconstruction with fewer iterations.

Reconstruction for microscopic imaging on quantitative phase target
At the aim to verify the universality of the TwPCPR method, the geometrical configuration of the hybrid DH microscopy imaging system is shown in Fig. 5(a). The principle of configuration is similar to Fig. 1(a), excepts that a microscopic system is considered between the sample and BS2. The distance difference between the two the two diffraction planes of two sensors is 3.3 mm. For the MPR method, the interval between different hologram planes is 1 mm. The bandwidth limit of this system belongs to the scope of in-line holography in Fig. 1b2

Reconstruction for microscopy imaging on biological sections
We demonstrate the biological imaging experiments of the TwPCPR method based on the hybrid DH microscopic system in Fig. 5(a). Figure 6(a) shows the reconstructed amplitude of the mitotic section of horse roundworm fertilized egg on the glass slide by different method. Figure 6(b) shows the corresponding phase. Some cells are distributed along the edge of original FOV without flat boundary condition. The red area (a1)-(a2) and (b1)-(b2) show the reconstructed amplitude of anaphase mitosis and telophase mitosis of the horse roundworm fertilized egg, respectively. The cell and nucleus  (a1) and (b1). The thickness of the nucleus increases significantly. A new nucleus is gradually formed in the telophase and the two daughter cells begin to separate, which is clearly shown in red area (a2) and (b2). The reconstructed phase by using MPR method only represent edge contours of the cells. The results of the quantitative phase microscopy and biological experimental validation show that the TwPCPR method can be combined with modern microscopy systems to achieve robust reconstruction of complex-amplitude with fast convergence. The advantage of non-prior TwPCPR reconstruction without constrains on the object is that the spatial complex-amplitude field can be reconstructed even if the exact location of the object cannot be obtained, such as atmospheric turbulence and air disturbance. The TwPCPR method can also achieve precise wavefront detection of phase disturbance. But the TwPCPR method still suffers from a few limitations that need to be addressed in the future. Currently, this method captures three images to accurately recover a field. Time division method are required to obtain the off-axis hologram though two cameras are used without real-time calibration and registration Fig. 6 Experimental reconstruction of the mitotic section of horse roundworm fertilized egg on the glass slide by using the 2-MPR method with 50 iterations, 3-MPR method with 50 iterations and TwPCPR method with 5 iterations in microscopic imaging system. a-b The reconstructed amplitude and phase by using TwPCPR method with 5 iterations, respectively. a1-a2 The reconstructed amplitude of anaphase mitosis and telophase mitosis of the horse roundworm fertilized egg in (a), respectively. b1-b2 The corresponding reconstructed phase in (a1)-(a2), respectively of multi-image. The distance between the object and the sensor can be adjusted by changing the size of the BS. But it limits the numerical aperture of the lensless system due to cube shape of BS. With the development of integrated optics, BS by specially designing with large area and small thickness may be help to perform the TwPCPR method in pixel super-resolution with fast convergence, and then it can apply in lensless on-chip microscopy. Moreover, this method can only handle static aberrations because we calibrate aberrations prior to the measurement, and we assume that they remain unchanged during the acquisition process. An optimization framework to automatically separate the object and aberrations without calibration is great interest to its applications in high-precision and real-time measurement.

Conclusions
We demonstrate that the proposed TwPCPR method performs a reliable, rapidly converging iterative procedure by involving only two in-line holograms and one off-axis hologram between only two planes. Full-bandwidth reconstruction can be realized based on in-line holography. It realizes a fast-convergence holographic calculation method. The reported results show robust reconstructions of the complex-amplitude fields of different types of objects by combining the advantages of in-line and off-axis holography. Off-axis optimization phase provides an effective initial guess to avoid stagnation and minimize the required measurements of multi-plane phase retrieval. This universal complex-amplitude reconstruction method works well for more extended samples. The TwPCPR method does not require any prior assumptions of objects including support, non-negative, sparse constraints, etc. It realizes both amplitude and phase solutions for complex-amplitude wave field analysis and imaging. Experimentally, the high phase reconstructed accuracy enables this method to have important applications in high-resolution wavefront sensing. It can be easily combined with modern microscopy system to realize complex-amplitude microscopy imaging. The versatility of TwPCPR method can significantly improve the performance of existing technologies such as adaptive optics. This technique helps to perform improvements in DH and quantitative phase imaging, thus enlarging and empowering applications in wavefront sensing, computational microscopy and biological tissue analysis.