Holographic capture and projection system of real object based on tunable zoom lens

Abstract

In this paper, we propose a holographic capture and projection system of real objects based on tunable zoom lenses. Different from the traditional holographic system, a liquid lens-based zoom camera and a digital conical lens are used as key parts to reach the functions of holographic capture and projection, respectively. The zoom camera is produced by combing liquid lenses and solid lenses, which has the advantages of fast response and light weight. By electrically controlling the curvature of the liquid-liquid surface, the focal length of the zoom camera can be changed easily. As another tunable zoom lens, the digital conical lens has a large focal depth and the optical property is perfectly used in the holographic system for adaptive projection, especially for multilayer imaging. By loading the phase of the conical lens on the spatial light modulator, the reconstructed image can be projected with large depths. With the proposed system, holographic zoom capture and color reproduction of real objects can be achieved based on a simple structure. Experimental results verify the feasibility of the proposed system. The proposed system is expected to be applied to micro-projection and three-dimensional display technology.

Introduction

With the rapid development of the information age, people’s demand for information display is increasing gradually. The next generation of display technologies such as virtual reality, micro-projection display and three-dimensional (3D) display are gradually appearing in various applications [1,2,3]. The traditional micro projectors are based on amplitude modulation and they usually use multiple solid lenses to form a projection lens [4, 5]. In contrast, holographic projectors have higher light efficiency and the feasibility of real 3D images by encoding corresponding grayscale images on a spatial light modulator (SLM) [6, 7]. Therefore, micro-projection technology based on the holography has attracted much attention. Holographic capture and projection technology for real objects in real-time has important application value in military, medical and other fields.

Although holographic projection technology has made some progresses, there are still some issues to be solved:

• 1) It is difficult to acquire the image source in real time. For the imaging capture process, in order to realize high quality holographic projection effect, we hope to reproduce real or virtual objects with full color and ideal size. In the process of image acquisition, virtual objects can be obtained by modeling, and real objects are usually captured by a CCD camera [8]. The traditional zoom camera is achieved by changing the distances between the solid lenses [9], so that detailed information of the image can be captured. But the bulky size is inconvenient to be designed for micro projectors. To acquire a 3D object, multiple cameras are required for shooting [10, 11]. Therefore, the current cameras are difficult to meet the needs of real-time acquisition for different scenes [12].

• 2) The size of reproduced image is relatively small. For the holographic reconstruction process, a solid lens is usually used to implement the Fourier transform [13]. When the position of the receiving screen changes, the image becomes unclear. Therefore, it is necessary to change the position and size of the reproduced image by adjusting the position or focal length of the solid lens [14, 15]. Some researchers used scaled Fresnel diffraction to realize zoomable holographic projection [16]. Liquid crystal lens has also been used in the holographic system to adjust the size of the reproduction [17]. However, the size of the liquid crystal lens is small and the aberration exists, so it is difficult to achieve color reproduction.

• 3) On the other hand, the chromatic aberration in the system also affects the effect of holographic projection. Color reproduction method based on time-division multiplexing is to load red, green and blue holograms in turns onto the same SLM [18, 19]. This method requires the switching time of the light source and the hologram to be strictly consistent. Color reproduction method based on spatial multiplexing is to divide an SLM into three parts or using three SLMs. When three color light beams are used to illuminate the corresponding regions, color reconstructed image can be seen due to the accurate coincidence in space [20, 21]. Some researchers put forward the methods of frequency shift and image shift to realize the perfect coincidence of color image [22]. However, the system uses a 4f lens and the size of the reproduction cannot be changed. At present, in order to achieve color zoom projection without chromatic aberration, the system is usually more complex. In addition, these holographic zoom systems currently reproduce virtual objects. If the real scene is acquired, the system will be even larger.

Adaptive liquid lenses have been studied recent years due to the unique advantages of large focal length tuning, fast response, and light weight [23,24,25,26]. In 2014, two liquid lenses were produced and used together with a digital lens, then the holographic projection system with an optical zoom function can be realized [27]. In 2018, an optical see-through head mounted display was proposed by using a liquid lens [28]. Although the liquid lens can change the size and position of the reproduced image adaptively, the depth of the reproduced image is constant. When the receiving screen is moved away from the focal plane, the reconstructed image becomes blurred.

In order to solve the above problems, in this paper, we propose a holographic capture and projection system of real objects based on adaptive lenses. Different from the traditional holographic system, a tunable zoom camera is produced based on liquid lenses in order to capture the real objects. The liquid lens is electrically driven, so the zoom camera has a fast response speed. Real objects can be captured and the detail part can be optically magnified by adjusting the focal length of the zoom camera. Moreover, in the holographic reproduction, we use a digital conical lens instead of the other lenses. By loading the phase of the conical lens on the SLM and adjusting the focal length of the corresponding colors, color holographic projection of the real object can be realized without chromatic aberration. Compared with the previous systems that use liquid lens or digital lens for reconstruction [29], the size and position of the reconstructed image can be changed easily without any optical components. The structure of the proposed system is simplified to a great extent and the reconstructed image can be projected with a large depth.

Structure and operating principle

Structure of the proposed system

Figure 1 is the schematic diagram of the proposed system. It consists of a zoom camera, three lasers, three filters, three solid lenses, a mirror, three beam splitters (BSs), an SLM, a computer and a receiving screen. In the process of acquiring images, the zoom camera is used to capture the image of the real object. The zoom camera is connected to the computer, then the information of the real object can be transferred to the computer. The hologram of the object can be generated through the computer. The lasers, filters and solid lenses are used to generate the collimated light. The mirror and the BSs are used to adjust the angle of light so that the collimated light can illuminate the SLM. When the hologram is loaded on the SLM, the diffracted light is reflected by the BS. Finally, the reconstructed image can be seen on the receiving screen.

Fig. 1

Schematic diagram of the proposed system

Principle of the zoom camera

In the image acquisition part, a zoom camera based on adaptive liquid lenses is designed by Zemax and its structure is shown in Fig. 2. The zoom camera consists of two electrowetting liquid lenses and four solid lenses. The two liquid lenses can not only play a role of zoom part, but also keep the image plane fixed during the zoom process. The two liquid lenses are both actuated by electrowetting and have the functions of variable focal length. The focal length of the liquid lens is changed by tuning the liquid-liquid interface due to the electrowetting effect. According to Young-Lippmann equation, the relationship of the contact angle and the applied voltage U can be described as follows:

$$ \cos {\theta}_{\mathrm{Y}}=\mathit{\cos}{\theta}_0+\frac{U^2\varepsilon }{2d{\gamma}_{12}}, $$

(1)

where θ₀ is the initial contact angle without external voltage, θ_Y is the contact angle when the voltage is applied to the liquid lens, U is the external voltage applied to the ITO electrode, d is the thickness of the dielectric insulator, ε is the dielectric constant of the dielectric insulator and γ₁₂ is the surface tension between the two liquids that filled in the liquid lens chamber.

Fig. 2

Structure of the zoom camera. a Zoom state when effective focal length is F₁; b zoom state when effective focal length is F₂

The effective focal length f of the two variable liquid lenses can be expressed by the following equations:

$$ \frac{1}{f}=\phi ={\phi}_1+{\phi}_2-l{\phi}_1{\phi}_1, $$

(2)

$$ {\phi}_1=\frac{n_2-{n}_1}{r_1}, $$

(3)

$$ {\phi}_2=\frac{n_2-{n}_1}{r_2}, $$

(4)

where n₁ and n₂ are the refractive indexes of the two liquids filled in the liquid lens, l is the distance between the two liquid lenses, and r₁, r₂ are the radii of the liquid-liquid interfaces under the voltages of U₁ and U₂, as shown in Fig. 2a. When the two liquid lenses are applied with different voltages of U₃ and U₄, the effective focal length of the system can be changed from F₁ to F₂, as shown in Fig. 2b. The proposed zoom camera can vary the optical power with the fixed back focal distance L.

Principle of the holographic reconstruction

In the holographic reconstruction part, three colors of the collimated light illuminate one third of the SLM area, respectively. When the collimated light is used to illuminate the SLM loaded with hologram, the reconstructed image can be displayed on the receiving screen after the modulation of the SLM. In the traditional Fourier holographic system, a solid lens (assuming that the focal length is f₀) is used for Fourier transform and the receiving screen is placed at the focal plane of the solid lens. Then the light field on the receiving screen U_f0 (u, v) can be expressed as follows:

$$ {U}_{f_0}\left(u,v\right)=\frac{e^{i k{f}_0}}{i{f}_0\lambda}\exp \left[\frac{i\pi}{f_0\lambda}\left({u}^2+{v}^2\right)\right]\iint {U}_0\left(x,y\right)\exp \left[\frac{-2 i\pi}{f_0\lambda}\left( xu+ vy\right)\right] dxdy $$

(5)

where λ is the wavelength of the parallel light source, k = 2π/λ, U₀(x, y) is the hologram distribution. For different wavelengths, the corresponding focal lengths are different accordingly (f_r, f_g, f_b are the focal lengths for red, green and blue colors respectively). So, the position and size of three color-reconstructed images are different, as shown in Fig. 3.

Fig. 3

Principle of the chromatic aberration

In the proposed system, a digital conical lens is used to replace the traditional solid lens. Figure 4 shows the principle of the conical lens. The phase of the conical lens φ (d_c) with the focal length f₀ can be expressed as follows:

$$ \phi \left({d}_c\right)=\frac{\pi }{\lambda}\frac{{d_c}^2}{f_0+\frac{z}{r^2}{d_c}^2} $$

(6)

where d_c is the radial coordinate, z is the focal depth, and r is the radius of the conical lens. High horizontal resolution requires a large numerical aperture, while longer focal depth requires a small numerical aperture. A conical lens can have the both advantages. So, when the conical lens is used in the holographic reconstruction, we can see the clear image in the focal depth of z. In the proposed system, the SLM is used to record the phase information of the conical lens. According to Eq. (6), the phase of conical lens can be calculated. The final phase loaded on the SLM φ’ can be expressed as follows:

$$ \phi \hbox{'}={\phi}_o+\phi \left({d}_c\right) $$

(7)

where φ_o is the phase information of the recorded object. In this way, we can use the SLM to realize the function of the digital conical lens.

Fig. 4

Principle of the conical lens

In order to keep the position of three colors reconstructed images in the same position, the focal depth z of conical lens is used to compensate for the difference between the focal lengths of the three color images. According to Fig. 3, the blue reconstructed image has the shortest reconstructed distance. So, the focal depth z of the conical lens satisfies z ≥ f_r - f_b. By setting the corresponding parameters, the digital conical lens is generated and added to holograms of three colors. In this way, three color reconstructed images can coincide in the same position without axial chromatic aberration. Moreover, by changing the focal length of the digital conical lens, the size and the position of the image can be changed easily.

Simulation, experiments and results

In order to verify the feasibility of the proposed system, the optical experiment is built. The wavelengths of the lasers used in the experiment are 671 nm, 532 nm and 473 nm, respectively. The lasers are manufactured by Changchun New Industries Optoelectronics Technology Co., Ltd. The filters and the solid lenses are manufactured by Daheng New Epoch Technology Inc. The focal length of the solid lens is 300 mm. The SLM is manufactured by Xi’ an CAS Microstar Optoelectronic Technology Co., Ltd. and the refresh rate of the SLM is 60 Hz. The resolution and pixel size of the SLM are 1920 × 1080 and 6.4 μm, respectively. We chose a commercial liquid lens Arctic 39 N0 produced by Corning, US. The size of the CCD is 1/2.5” and the pixel size is 2.2 μm.

Simulation and experiments of the tunable zoom camera

In the Zemax simulation, the focal length of the proposed zoom camera can be tuned from ~ 26.08 mm to ~ 32.26 mm. In the zoom process, f/# varies with the focal length, airy disk and spot diameter, as shown in Table 1. As can be seen from Table 1, the spot diameters are smaller than the airy disk during the focal length tuning process, which indicates that the zoom camera has a reasonable high image quality. We also simulate the point spread function (PSF) and modulation transfer function (MTF) at the effective focal lengths of 26.08 mm, 27.90 mm and 32.26 mm, as shown in Fig. 5. From Fig. 5 we can see that the spatial frequency can reach to 60 lp/mm when MTF > 0.5 (the center field) during zooming, which means the camera has a high resolution in the center field. Although the other field, the zoom camera has a relatively large astigmatism especially at the focal length of 32.26 mm, as shown in Fig. 5c. We can add more liquid lenses in the zoom camera to balance the aberration.

Table 1 Detailed parameters of the proposed zoom camera

Fig. 5

PSF and MTF of the zoom camera with different effective focal lengths. aF = 26.08 mm; bF = 27.90 mm; cF = 32.26 mm

Then we fabricate the zoom camera to evaluate the optical performance. The zoom camera consists of a liquid lens cavity, a back-lens group with two solid lenses, a front-lens group with two solid lenses, two commercial liquid lenses and a CCD circuit board, as shown in Fig. 6.

Fig. 6

Zoom camera fabrication

In the first experiment, we put two panda dolls 100 mm away from the zoom camera and optimize the radii of the two liquid lenses in order to get the optimized solutions for six focal lengths in Zemax, as shown in Fig. 7a. Then the applied voltages can be got based on the optimized solutions. When the optimized voltages are applied to the two electrowetting liquid lenses, we can obtain the magnified images, as shown in Figs. 7b-f. During the driving process, the effective focal lengths can be varied from 26.08 mm to 32.26 mm. The dynamic response video of the image capture process is included in Additional file 1: Media S1. The curve radius of the two liquid lenses during the focal length varying is also measured, as shown in Fig. 8.

Fig. 7

Captured images of two panda dolls with different effective focal lengths. aF = 26.08 mm; bF = 27.31 mm; cF = 27.90 mm; dF = 29.97 mm; eF = 31.87 mm; fF = 32.26 mm

Fig. 8

Curve radius of the two liquid lenses during the focal length varying

Experiments of the holographic reconstruction

In the second experiment, when the object is captured, the scene information of red, green and blue colors is separated. The holograms of the recorded object for three colors can be generated by the iterative Fourier transform algorithm. The phase information of the digital conical lens can be generated according to Eq. (5). The final hologram can be generated by adding the phase of the digital conical lens to the that of the recorded object, as shown in Fig. 9.

Fig. 9

Process of the hologram

To verify the advantage of the digital conical lens, a solid lens and a digital lens with the same focal length are used for experimental comparison. The focal lengths of the solid lens, digital lens and digital conical lens are set to be 500 mm. The focal depth of the digital conical lens is set to be 200 mm. Then the reconstructed image can be seen on the receiving screen, as shown in Fig. 10. When the receiving screen is placed at the focal plane of the corresponding lens, the result by using the solid lens, digital lens and digital conical lens are shown in Figs. 10a-c, respectively. When the receiving screen moves backward from the focal plane position, the results are shown in Figs. 10 d-f. It can be clearly seen that at this time, the reproduced images of the solid lens and digital lens appear to be blurred, while the reproduced image by using the digital conical lens is clear. It can be seen the reconstructed image can be projected in a wider depth range by using the digital conical lens. Then we change the parameters of the digital conical lens and compare the reconstructed image of the panda. The focal length is set to be 600 mm and the focal depth is 500 mm. When the position of the receiving screen changes, the details of the panda can be reproduced clearly with a larger depth, as shown in Fig. 11. So, by changing the focal length and focal depth of the conical lens, the size and depth of the reconstructed image can be adjusted easily.

Fig. 10

Results of the reproduced image with different lenses. a Result with the solid lens when the receiving screen is in the focal plane; b result with the digital lens when the receiving screen is in the focal plane; c result with the digital conical lens when the receiving screen is in the focal plane; d result with the solid lens when the receiving screen moves backwards; e result with the digital lens when the receiving screen moves backwards; f result with the digital conical lens when the receiving screen moves backwards

Fig. 11

Results of the reproduced image when the parameters of the digital conical lens are changed. a Result when the reproduction distance is 700 mm; b Result when the reproduction distance is 800 mm; c Result when the reproduction distance is 900 mm

In order to eliminate the lateral chromatic aberration, the sizes of the three color components are scaled at the process of color separation. We verified the three colors separately. Since the conical lens has the large depth, when the position of the receiving screen is fixed, the reproduced images of the three colors can be clearly displayed, as shown in Figs. 12a-c. Therefore, axial chromatic aberration can be eliminated. In order to achieve color coincidence based on an SLM, the SLM is divided into three parts in space and each part is illuminated with the corresponding color light respectively, as shown in Fig. 12d. Figure 12e is the color reconstructed image, and the result shows that three color reconstructed images can coincide in the same position without chromatic aberration. When the focal length of the zoom camera changes, the size of the captured object is different accordingly. In this way, the magnified scene of the object can be captured. The results of the holographic reconstructed image on the receiving screen for different captured object are shown in Fig. 13. Fig. 13 shows the partial reconstruction of the object. With the proposed system, we can take the detail of the object by optical zoom and project it simultaneously.

Fig. 12

Results of three color-reconstructed images. a Green result; b blue result; c red result; d experimental diagram of the SLM; e color result

Fig. 13

Results of the reproduced images when the focal length of the zoom camera changes. a Reconstructed image when F = 26.08 mm; b reconstructed image when F = 31.87 mm

Discussion

The proposed zoom camera has the resealable fast response time (within 200 ms), thus it can also be used as a depth acquisition camera. When only one liquid lens is actuated under the voltages of 36 V and 55 V, the results of the captured images are shown in Fig. 14. It can be seen clearly that by adjusting the focal length of the liquid lens, objects of different depths can be photographed. The dynamic response video of the image capture process with single liquid lens is also included in Additional file 2: Media S2. Figure 15 is the reconstructed images of the captured images when only one liquid lens is actuated. At present, the switching time of a single liquid lens is ~ 200 ms. When the switching time is fast enough, we can consider using it to obtain the information of the 3D object. There are already technologies that can control the response time of a liquid lens within a few tens of milliseconds. We believe that with the optimization of the system, zoom cameras are expected to be applied to the acquisition of 3D objects in the future.

Fig. 14

Results of the captured images when only one liquid lens is actuated. a Applied voltage U = 36 V; b applied voltage U = 55 V

Fig. 15

Reconstructed images when only one liquid lens is actuated. a Applied voltage U = 36 V; b applied voltage U = 55 V

In the proposed system, as the digital conical lens has a large focal depth, the reconstructed image of the object can be clearly seen in the focal depth, as shown in Fig. 10c and f. On the other hand, by changing the focal length of the digital conical lens, the size and the position of the reconstructed image can also be adjusted easily, as shown in Fig. 11. In the holographic reconstruction, the reproduced image is disturbed by zero-order light and high-order diffraction images. Figures 10, 11, 12, 13 show the first-order diffraction images. We can load the offset on the hologram to separate the reproduced image and zero-order light, then the undesirable light can be eliminated using an aperture or a filter in the system. Compared with the existing holographic projection system, the proposed system is designed with a zoom camera, which is very small in size and fast in response time. Therefore, the proposed system can easily capture the details of the object without moving the position of the zoom camera. In addition, compared with the previous systems, we use digital conical lens instead of solid lens or other lens for projection. For the same focal length, the digital conical lens has a large depth of focus and the projected image is clear in the focal depth range. Color holographic projection can be realized without chromatic aberration. The size and position of the projected image can be changed according to the requirement easily. In the process of generating the hologram, the iterative Fourier transform algorithm is used to calculate the phase information of the object. Of course, if we use GPU or other acceleration algorithms, the calculation speed can be faster. For the zoom camera, we are developing a circuit board and a control software for tuning the focal lengths of the liquid lenses, synchronously. Through the above methods, the whole response time of the system can be improved effectively. With the decrease of switching time of zoom camera and the improvement of hologram calculation, the real time capture and projection of 3D object can be realized eventually. In the next work, we will continue our research to improve the performance of the system. We believe that our work can promote the development of micro-projection technology and 3D technology.

Conclusion

In this paper, a holographic capture and projection system of real objects based on zoomable lenses is proposed. A liquid lens-based zoom camera and a digital conical lens are used as key parts to reach the functions of holographic capture and projection, respectively. The liquid lens is electrically driven, so the zoom camera has a fast response speed and light weight. As another tunable zoom lens, the digital conical lens has a large focal depth and is used in the holographic system for adaptive projection. By adding the phase of the conical lens to that of the captured object, the reconstructed image can be projected with a large depth. With the proposed system, holographic zoom capture and color reproduction of real objects can be achieved based on a simple structure. The proposed system is expected to be applied to the real-time acquisition and reproduction of 3D objects.