Artificial potential field-empowered dynamic holographic optical tweezers for particle-array assembly and transformation

Abstract

Owing to the ability to parallel manipulate micro-objects, dynamic holographic optical tweezers (HOTs) are widely used for assembly and patterning of particles or cells. However, for simultaneous control of large-scale targets, potential collisions could lead to defects in the formed patterns. Herein we introduce the artificial potential field (APF) to develop dynamic HOTs that enable collision-avoidance micro-manipulation. By eliminating collision risks among particles, this method can maximize the degree of parallelism in multi-particle transport, and it permits the implementation of the Hungarian algorithm for matching the particles with their target sites in a minimal pathway. In proof-of-concept experiments, we employ APF-empowered dynamic HOTs to achieve direct assembly of a defect-free 8 × 8 array of microbeads, which starts from random initial positions. We further demonstrate successive flexible transformations of a 7 × 7 microbead array, by regulating its tilt angle and inter-particle spacing distances with a minimalist path. We anticipate that the proposed method will become a versatile tool to open up new possibilities for parallel optical micromanipulation tasks in a variety of fields.

Introduction

Assembling building blocks like micro- and nano-particles into organized patterns is a subject that has fascinated scientists from a spectrum of disciplines. Physicists arrange atom arrays for quantum operations [1,2,3], while biologists create cell patterns to study intercellular communication [4,5,6]. Chemists and material scientists are devoted to assembling molecules and colloids into structures that can mimic or surpass nature’s complexity and functionality [7,8,9,10,11]. Engineers explore new forms of displays in which the pixels are made of patterned particles [12,13,14].

Among state-of-the-art techniques, dynamic holographic optical tweezers (HOTs) have stood out as an assembly method featuring both programmable and reconfigurable properties, because it permits simultaneous trapping of multiple particles and their independent control in real-time [15]. Thus far, however, its advanced use has been hampered by the collision problem. In the assembling process, collisions between particles may occur, especially for transporting massive particles. Collisions could cause particles to be knocked out of their traps or all fall into a single trap, leading to defects in the patterns (Fig. 1a). An effective and traditional route to form a defect-free pattern is to carefully steer each particle toward its target position one by one [1, 2], which however is inefficient or time-consuming. Potential collisions also impose a challenge in the flexible transformation of a pattern, such that usually only simple types of dynamics are allowed, such as rigid translation and rotation. Central to these challenges is the long-standing question of how to make full use of parallel manipulation by overcoming collisions.

Fig. 1

Schematics illustrating the superiority of APF-empowered dynamic HOTs over the conventional counterpart in parallel manipulation. a Conventional dynamic HOTs cannot eradicate potential inter-particle collisions, which may give rise to defects in the assembled microbead patterns and limit the flexibility of pattern transformation. b By contrast, the APF-empowered method provides a collision-free solution for parallel manipulation by creating a virtual repulsive field around each trap, which promises the direct assembly of perfect patterns and allows high-efficiency elastic transformation with a minimalist path

Fortunately, the well-known artificial potential field (APF) [16] has provided a virtual force method to avoid obstacles for macroscopic manipulators and mobile robots. The APF method is widely used in the cooperative work of multiple robots and the co-navigation of multi-unmanned aerial vehicles [17,18,19]. Yet this artificial intelligence technique remains unexploited in optical tweezers. In this work, we integrate the APF into dynamic HOTs for direct assembly and transformation of microbead arrays (Fig. 1b). The Hungarian algorithm [20] is adopted for matching the microbeads to target sites along a minimal path. We show that APF-empowered dynamic HOTs enable rapid assembly of large-scale defect-free patterns in a parallel way, and endows the pattern transformation with a high degree of freedom. This is attributed not only to the virtual force due to the APF, which protects the particles against collisions, but also to the optical gradient force [21,22,23,24,25] exerted by HOTs, which overwhelms stochastic Brownian effects so that the APF scheme can work as expected. All the experimental results are in accordance with theoretical expectations, validating the effectiveness of the proposed method for collision avoidance in dynamic HOTs.

Results

The formation of microbead arrays using dynamic HOTs involves three main stages: target assignment, collision-free path planning, and hologram generation.

Target assignment

We temporarily ignore potential collisions to assign destinations to beads. The coordinates of all target sites are stored in the set $T=\{{\mathbf{t}}_{i}|1\le i\le N\}$, and the coordinates of all trapped beads are put into the set $B=\{{\mathbf{b}}_{j}|1\le j\le N\}$. The number of target sites is N, which is equal to the number of trapped beads. The goal is to find a one-to-one matching $f:T\to B$, which minimizes the total distance between the target sites and the beads, ${d}_{\text{total}}={\sum }_{i}|{\mathbf{t}}_{i}-f({\mathbf{t}}_{i})|$. This is a minimum-weight matching problem of bipartite graphs, which can be solved by the Hungarian algorithm. For an N × N cost matrix D, whose element ${d}_{i,j}=|{\mathbf{t}}_{i}-{\mathbf{b}}_{j}|$ denotes the distance between the target site t_i and the bead b_j, the Hungarian algorithm has a time complexity of O(N³). Figure 2a (left panel) illustrates the minimum-weight match of a target array with N = 25 randomly distributed beads. We have implemented the Hungarian algorithm by using MATLAB software and run it on a CPU (Intel Xeon E5-1620 v3). In this example, the running time of the algorithm is approximately 1.4 ms. The fast computational speed allows the Hungarian algorithm to handle large values of N.

Fig. 2

Path planning for both conventional and APF-empowered dynamic HOTs. a The left shows a minimum-weight match between a 5 × 5 target array and the same number of random beads by using the Hungarian algorithm with the objective function ${d}_{\text{total}}={\sum }_{i}|{\mathbf{t}}_{i}-f({\mathbf{t}}_{i})|$. The right shows bead positions after 13 travel steps along straight lines, where collisions occur (cyan circles). The red balls indicate beads, the magenta circles indicate target sites, the arrows indicate one-to-one matches, and the cyan circles indicate collision events. b Experimental results corresponding to (a). The final bead array exhibits defects (Video 1). There are three merging sites (red arrows) and three missing sites (yellow dashed circles). The target sites are spaced 1.5 μm apart and the beads are 1 μm in diameter. c Simulated and experimental trajectories. d The top shows distances from bead positions to target sites with time. The merging beads could not reach their targets, so the distances do not decrease to 0 after 5.8 s. The bottom shows the arrival percentage of beads and the distribution of trajectory lengths. e Collision-free path planning with the Hungarian algorithm followed by the APF method. The target sites and the initial positions of the beads are the same as (a). The solid purple lines indicate the travel trajectories. f Enlarged views of the boxes in (e). The green arrows indicate the changes in trajectories due to repulsive forces. g Experimentally obtained bead array without defects (Video 2). h Simulated and experimental trajectories. i Experimental trajectory analysis. All beads arrive at their target sites after 10.2 s, and 36% of the beads have a trajectory length of 5 μm

In the simulation, the target sites are spaced 1.5 μm apart, the beads have a diameter of 1 μm, and the step size is 0.25 μm. Figure 2a (right panel) shows the straight-line travel paths. There are three collision events (cyan circles) after 13 travel steps, two of which are two-bead collisions and another is a three-bead collision. The experimental results validate the simulated collisions, and the final bead array has defects (Fig. 2b, Video 1). The trajectories of the colliding beads deviate from the predefined straight lines (Fig. 2c), and merging beads could not arrive at their target sites after 5.8 s (Fig. 2d, top and lower left panels). The trajectory length is in the range of 2 ~ 7 μm (Fig. 2d, lower right panel).

Collision-free path planning

After the destinations of beads have been assigned, they need to be moved to the corresponding target sites. Taking straight lines as travel paths may lead to collisions (cyan circles in Fig. 2a,b), which will result in beads being knocked out of their respective traps, or all falling into a single trap. In massive microbead transport, the APF can effectively reduce collisions, especially the complicated collisions occurring among multiple beads. The APF sets repulsive forces between the beads, thus allowing for collision avoidance.

Consider again the example shown in Fig. 2a. We keep target sites and initial positions of beads as previously shown, and generate a collision-free path planning by using the Hungarian algorithm followed by the APF method. The Hungarian algorithm uses the objective function d_total. The purple lines in Fig. 2e indicate the collision-free travel trajectories. When the beads are close to collisions, the APF generates collision-avoidance tendencies via repulsive forces, which is manifested by the change of travel trajectories from straight lines to curves (Fig. 2f). A perfect bead array has been obtained from the experiment (Fig. 2g, Video 2). The experimental trajectories almost overlapped with the simulated ones (Fig. 2h). 100% of the beads have reached the target after 10.2 s (Fig. 2i, left and middle panels), and 36% of the beads have a travel length of about 5 μm (Fig. 2i, right panel). We implement the APF by using MATLAB software and run it on the CPU (Intel Xeon E5-1620 v3), too. In this example, the number of steps that the beads travel is 51 (Fig. 2g), and the running time of the APF is 8.4 ms.

By optimizing the match between beads and targets, the number of collisions can be reduced at the beginning (Extended Data Fig. 2). We thus modify the objective function of the Hungarian algorithm with ${d}_{{\text{t}}{\text{otal}}}^{(2)}=\sum_{i}{|{\mathbf{t}}_{i}-f({\mathbf{t}}_{i})|}^{2}$ to generate an alternative match as reported in ref. [20]. Accordingly, the element of cost matrix D becomes ${d}_{i,j}^{2}$. The match created with ${d}_{\text{total}}^{(2)}$ shows some improvement in terms of collision avoidance. Only a two-bead collision event occurs after 12 travel steps (Extended Data Fig. 2a). The experimental results are in good agreement with the simulation, and the final array has one empty site (Extended Data Fig. 2b, Video 3). The experimental trajectories are generally consistent with the simulations, except for the two colliding beads (Extended Data Fig. 2c). Statistical analysis of the experimental trajectories shows that 96% of the beads have arrived at their targets after 5.2 s, and the trajectory length ranges from 2 to 6 µm (Extended Data Fig. 2d). The APF still helps the beads avoid collisions. The path of the beads is re-planned by using the APF method (Extended Data Fig. 2e). The two originally colliding beads travel along the curved trajectories due to the repulsive force (Extended Data Fig. 2f). The final array shows no defects (Extended Data Fig. 2g, Video 4). The experimental trajectories are consistent with the simulated ones (Extended Data Fig. 2h). All beads have reached the targets after 9.6 s, and 40% of the beads have a trajectory length of about 5 μm (Extended Data Fig. 2i).

The objective function d_total equals the sum of distances, while ${d}_{\text{total}}^{(2)}$ denotes the sum of squared distances. However, ${d}_{\text{total}}^{(2)}$ is more effective to reduce collision events, thus helping to alleviate the workload of the APF at the later stage. In the following results, ${d}_{\text{total}}^{(2)}$ is taken as the objective function of Hungarian algorithm before potential collisions are avoided by using the APF method.

Next, we perform a collision-free path planning for massive beads. As shown in Fig. 3a, 100 beads are randomly distributed in an area of 30 μm × 30 μm, and the target is a 10 × 10 array with a spacing distance of 1.5 μm. The beads are 1 μm in diameter, the step size is 0.25 μm and the number of travel steps equals 70. Figure 3a shows the travel trajectories of 100 beads. The enlarged views show the trajectories of the two-bead collision avoidance. The green arrow indicates that the trajectories are bent due to repulsive forces. Note that most of the trajectories are straight and only bend when the beads are subjected to repulsive forces. Trajectory lengths range from 1 to 11 μm (Fig. 3b). In this example, the total running time of the APF is about 140.4 ms. The quantitative relationship between the single-step running time and the number of optical traps (beads) is $\frac{t}{{n}_{\text{step}}}=\left(1.92{N}_{\text{trap}}^{2}+9.12{N}_{\text{trap}}+135.34\right)\cdot {10}^{-4}$, where n_step is the number of travel steps and N_trap is the number of optical traps (Fig. 3c). The single-step running time is typically within 2 ms for N_trap ≤ 100.

Fig. 3

Collision-free path planning for 100 random beads forming a 10 × 10 target array by using the APF method. a Simulated trajectories of beads. Enlarged views of the boxes show the trajectories of the two-bead collision avoidance, with green arrows indicating the changes in trajectories due to repulsive forces. b Distribution of simulated trajectory lengths. c Quantitative relationship between the single-step running time and the number of optical traps

Formation of microbead arrays

After trapping the initial microbeads, the three stages of target assignment, collision-free path planning, and hologram generation are sequentially undergone. Subsequently, the phase-only liquid crystal spatial light modulator (SLM) updates the hologram at each timestep to achieve parallel movement of all optical traps. Figure 4a demonstrates that 64 random beads move to target sites according to preplanned paths, ultimately forming an 8 × 8 array with a spacing distance of 1.5 μm. (Video 5). The step size s = 0.25 μm and the number of travel steps is 27. The beads usually travel in constant step size, only decreasing the step size at the collision positions (Fig. 4b). All beads have reached their destinations after 5.4 s, and 53% of the beads have a trajectory length of about 3 μm (Fig. 4c).

Fig. 4

Formation of multiple microbead patterns starting from random initial positions. a Forming sequences of an 8 × 8 bead array (Video 5). Initial 64 beads are randomly distributed. The final array has been assembled after 27 steps. The spacing distance of beads is 1.5 μm. b Distance from bead positions to targets, as well as speed of 25 randomly selected beads. c Percentage of beads reaching their targets, and distribution of trajectory lengths. d-f Another three bead patterns after 23, 35, and 47 steps, respectively (Video 6, Video 7, Video 8). The insets show the target logos. The microbeads are 1 μm in diameter, and the scale bar is 2 μm

Notice that beads move with slight shaking. This is mainly due to two reasons: first, beads have an inherent Brownian motion, and second, optical traps flicker briefly during the response time of the SLM, that is, the traps become cluttered when the hologram switches. The stability of manipulations could be improved by decreasing the step size or by using an SLM with a higher response frequency, which would result in smoother trajectories. In addition, suppressing environmental vibration, especially severe vibration, is important to minimize bead escape.

The time for the optical traps to move one step is about 100 ms (the response time of the SLM) and the exposure time of the camera is 100 ms. One raw image is acquired after each step of movement of the optical traps. Therefore, the time interval between two adjacent images is 200 ms.

Figure 4d-f shows a gallery of microbead patterns with user-defined geometries containing 60, 58, and 48 beads, respectively (Video 6, Video 7, and Video 8). The single-frame images and trajectory analysis are shown in Extended Data Fig. 3-5. The forming sequences allow observing the assembly process of patterns (Extended Data Fig. 3a, 4a, 5a). The distance plots present the number of beads subjected to repulsive forces (Extended Data Fig. 3b, 4b, 5b). The trajectory length distributions illustrate the traveling distance of most beads (Extended Data Fig. 3c, 4c, 5c). There are two challenges to generating larger patterns. The first is the difficulty in trapping a large number of individual beads at the beginning of experiments. The second is the limited output power of the laser, as well as the finite damage threshold of the optical elements.

Fig. 5

Successive transformations of a 7 × 7 microbead array (Video 9). a Initial bead array formed from random positions (gray circles). The solid lines indicate the travel trajectories. b Bead array after a transformation. This transformation style yields shorter paths compared to the rigid rotation. 51% of the beads have a trajectory length of about 1.5 μm. c Bead array after a two-fold enlargement of the spacing distance. d-f Bead arrays after rotations of 15°, 30° and 45°, respectively. The gray circles in (b-f) indicate the positions where transformations start, and the solid lines in (b-f) indicate the travel trajectories extracted from the fluorescence images. The spacing distance of beads in (a,b) is 1.5 μm, while in (c-f) it is 3 μm. The microbeads are 1 μm in diameter, and the scale bar is 2 μm

Transformation of microbead arrays

A microbead array can be transformed into other micropatterns by setting new target sites. Figure 5a presents a 7 × 7 microbead array formed from random positions (gray circles). The solid lines indicate the travel trajectories extracted from the fluorescence images. The first transformation is a rotation of 45° of the array (Fig. 5b). 51% of the beads have a trajectory length of about 1.5 μm. The second transformation is a two-fold enlargement of the spacing distance (Fig. 5c). The next followed transformations are rotations of 15°, 30°, and 45°, respectively, as shown in Fig. 5d-f. The gray circles in Fig. 5b-f indicate the positions where transformations start. The step size s = 0.25 μm and the number of travel steps is 103. Dynamic transformations are recorded in Video 9.

Specifically, it is noted that the transformation here is not the overall rotation of an array (Fig. 5b). The path for the transformation is generated by both the Hungarian algorithm and the APF method. For the transformation, the maximum length of the trajectories is 3($\sqrt{2}$-1)·1.5 μm = 1.86 μm, while the maximum length of the overall rotation is 3 $\sqrt{2}$·π/4·1.5 μm = 5.00 μm. Compared to the rigid rotation, this style of transformation yields shorter paths and therefore takes less time to create a new array.

The arrays shown in Fig. 5 are all square. However, other geometric patterns can also be created by transformations. As shown in Fig. 6a, 48 beads first form two concentric circles from random positions, and then transform into octagonal, hexagonal, and quadrilateral patterns sequentially. Each of the two circles is transformed into an octagon (Fig. 6a, upper right panel). The trajectory length ranges from 0.8 to 1 μm (Fig. 6b, second panel). Figure 6a (lower left panel) shows that two beads move from the outer ring to the inner ring in the octagonal-to-hexagonal transformation. The longest trajectories are about 3 μm, and 58% of the beads have a trajectory length of 1.5 μm (Fig. 6b, third panel). In contrast, Fig. 6a (lower right panel) shows that two beads move from the inner ring to the outer ring during the transformation of hexagons to quads. The longest trajectories are about 2.5 μm, and 52% of the beads have a trajectory length of 1.5 μm (Fig. 6b, fourth panel). Dynamic transformations are recorded in Video 10.

Fig. 6

Successive transformations of geometric patterns (Video 10). a 48 beads first form two concentric circles from random positions, and then transform into octagonal, hexagonal, and quadrilateral patterns sequentially. The gray circles indicate the positions where beads start, and the solid lines indicate the travel trajectories extracted from the fluorescence images. The spacing distance of beads is 1.5 μm. The microbeads are 1 μm in diameter, and the scale bar is 2 μm. b Distributions of trajectory lengths corresponding to (a)

Up to this point, we have demonstrated theoretically and experimentally methods for matching microbeads and targets, as well as for generating collision-free paths. The Hungarian algorithm shortens the total length of the path, and the APF method circumvents potential collisions between the beads. Further, we have performed parallel micromanipulation of massive microbeads using the proposed dynamic HOTs. The collision-free paths facilitate to create defect-free bead arrays or patterns as expected.

Discussion

The time taken by the Hungarian algorithm and the APF method is short and negligible, while the hologram generation consumes a lot of time. Therefore, GPU acceleration is used to generate multi-frame holograms. Specifically, the time of hologram generation depends on the number of beads N and the step size s. On one hand, a greater N will undoubtedly increase the computation time. On the other hand, a smaller s will result in more travel steps, and thus, a longer computation time. However, a small step size helps improve the stability of the bead movement, yielding smoother trajectories.

The step size is limited by the size of Gauss-point optical traps and microbeads. In the current setup, the wavelength λ = 1.064 μm and NA = 1.27 of the water-immersed objective lens (Apo Plan IR, 60 × /NA1.27, Water Immersion, Nikon Corp., Japan). The diffraction limit is therefore 0.5λ/NA = 0.42 μm, equal to the full width at half maximum (FWHM) of Gauss-point optical traps. The beads used in our experiments have a diameter of 2a = 1 μm. Consequently, the theoretical maximum value of the step size is s_max = FWHM + a = 0.92 μm. The moving speed of trapped beads is equal to s·f_slm, where f_slm is the response frequency of the SLM. For the currently used SLM, the maximum value of the response frequency is about 10 Hz. Therefore, the theoretical limit of the moving speed that can be achieved by the current system is about 9 μm/s. Using a faster SLM, the maximum moving speed will ultimately be determined by the balance between viscous forces and optical forces acting on trapped beads. Although digital micromirror devices (DMDs) are faster, they are less efficient in terms of light energy utilization. In addition, acousto-optic modulators (AOMs) and galvanometers, etc., are not as flexible as liquid crystal SLMs in achieving complicated optical manipulations and multifunctional applications.

The travel trajectories are two-dimensional curves. However, 3D trajectories could be easily realized by adding the term exp(ik_zz_m) associated with the axial coordinate z_m into Eq. (7). The APF method is still suitable for collision avoidance in the depth direction. For further applications, such as those involving particles of different sizes, shapes or materials, things become complicated. On the one hand, the particular shape and size of each particle needs to be considered when designing the virtual repulsive force of the APF. On the other hand, when designing a hologram, it is necessary to assign appropriate light energy to particles of different sizes and materials.

In conclusion, we have developed dynamic HOTs to achieve defect-free assembly and transformation of particle arrays, with minimal path lengths and a maximized degree of parallelism of optical manipulation. At the center of our strategy is the use of APF, which enables collision-free parallel control of particles. Experiments with fluorescent PS microbeads verified the practical performance of APF-empowered dynamic HOTs. Our work evolves functionally simple optical tweezers into powerful multi-armed robots, which will facilitate exciting applications in the generation of atomic arrays for quantum computation [26, 27], the study of cell sorting and intercellular interactions, and the assembly of optomechanical meta-devices [28] for advanced micromanipulation [29,30,31,32]. It will also inspire the development of other active assembly techniques, such as time-shared scanning optical tweezers [33, 34] and holographic acoustic tweezers [12, 35].

Methods

The principle of APF

The APF method mainly applies repulsive fields to realize collision avoidance. Each bead creates its repulsive field with the potential of

$$\begin{array}{c}U_\text{rep}\left(\mathbf q,{\mathbf b}_j\right)=\left\{\begin{array}{lc}\frac12\eta\left(\frac1{\rho_j}-\frac1{\rho_0}\right)^2&\text{ if }\rho_j\leq\rho_0\\0&\text{ if }\rho_j>\rho_0\end{array}\right.,\end{array}$$

(1)

where η is the repulsion coefficient, ρ₀ is the acting range of the force, and ρ_j =|q-b_j| is the distance from any position q = (x, y) to the bead b_j. The repulsive force is

$$\begin{array}{ll}{\mathbf F}_\text{rep}\left(\mathbf q,{\mathbf b}_j\right)&\begin{array}{lc}=&-\nabla U_\text{rep}\left(\mathbf q,{\mathbf b}_j\right)\end{array}\\&=\left\{\begin{array}{lc}\eta\left(\frac1{\rho_j}-\frac1{\rho_0}\right)\frac1{\rho_j^2}\nabla\rho_j&\text{if}\ \rho_j\leq\rho_0\\0&\text{if}\ \rho_j>\rho_0\end{array}\right..\end{array}$$

(2)

When the distance ρ_j is less than a given value ρ₀, the beads will tend to avoid collisions. Each bead will move in the direction away from its neighboring beads, thus maintaining a suitable distance between beads, and avoiding collisions.

In addition, since beads need to reach their respective destinations, it is required to set up the gravitational field at target sites with a potential of

$$\begin{array}{c}{U}_{\text{att}}\left(\mathbf{q},{\mathbf{t}}_{i}\right)=\frac{1}{2}\xi {\rho }_{i}^{2},\end{array}$$

(3)

where ξ is the gravitational coefficient and ρ_i =|q-t_i| is the distance from the current point q to the target site t_i. The gravitational force is

$$\begin{array}{c}{\mathbf{F}}_{\text{att}}\left(\mathbf{q},{\mathbf{t}}_{i}\right)=-\nabla {U}_{\text{att}}\left(\mathbf{q},{\mathbf{t}}_{i}\right)=-\xi {\rho }_{i}\nabla {\rho }_{i}.\end{array}$$

(4)

For any bead b_m, it is subject to both the gravitational force F_att(b_m, t_i) of the target site and the repulsive force of the other beads ${\mathbf{F}}_{\text{rep},m}=\sum_{j\ne m}{\mathbf{F}}_{\text{rep}}({\mathbf{b}}_{m},{\mathbf{b}}_{j})$, with a resultant force of

$$\begin{array}{c}\begin{array}{cc}{\mathbf{F}}_{\text{res}}\left({\mathbf{b}}_{m}\right)=& {\mathbf{F}}_{\text{att}}\left({\mathbf{b}}_{m},{\mathbf{t}}_{i}\right)+{\mathbf{F}}_{\text{rep},m}\\ =& {\mathbf{F}}_{\text{att}}\left({\mathbf{b}}_{m},{\mathbf{t}}_{i}\right)+\sum_{j\ne m}{\mathbf{F}}_{\text{rep}}\left({\mathbf{b}}_{m},{\mathbf{b}}_{j}\right).\end{array}\end{array}$$

(5)

So now, we can update the position of bead b_m based on the resultant force F_res(b_m) on it:

$$\begin{array}{c}{\mathbf{b}}_{m}\left(t+\tau \right)=\left\{\begin{array}{ll}{\mathbf{b}}_{m}\left(t\right)+s\frac{{\mathbf{F}}_{\text{res}}\left({\mathbf{b}}_{m}\left(t\right)\right)}{\left|{\mathbf{F}}_{\text{res}}\left({\mathbf{b}}_{m}\left(t\right)\right)\right|}& \text{ if }\left|{\mathbf{F}}_{\text{rep},m}\right|=0\\ {\mathbf{b}}_{m}\left(t\right)+\frac{s}{2}\frac{{\mathbf{F}}_{\text{res}}\left({\mathbf{b}}_{m}\left(t\right)\right)}{\left|{\mathbf{F}}_{\text{res}}\left({\mathbf{b}}_{m}\left(t\right)\right)\right|}& \text{ if }\left|{\mathbf{F}}_{\text{rep},m}\right|>0\end{array}\right.,\end{array}$$

(6)

where s is the step size when the repulsion |F_rep,m|= 0. When the repulsion is greater than 0, potential collisions are avoided by reducing the step size to s/2 or less. τ is the time interval between adjacent steps. Each bead updates its position according to Eq. (6) until it reaches the target site. The APF parameters affect the trajectories of particles. The ρ₀ determines the area affected by the repulsive force and should be smaller than the spacing of target sites. The η and ξ define the magnitude of the repulsive and gravitational forces, respectively. The repulsive force should be in the same order of magnitude as the gravitational force to keep trajectories smooth.

Hologram generation

By modulating the wavefront of the incident light in the back focal plane of the objective lens, Gauss-point optical trap array can be created in its front focal plane. Several methods have been proposed to generate phase-only holograms [36]. Although iterative methods have a higher quality of optical traps, they are also more time-consuming. For current dynamic manipulations, which require a large number of holograms (tens to hundreds), we adopted the randomized superposition (SR) method among direct algorithms. Here, the phase distribution ϕ(r) in the back focal plane is given by:

$$\begin{array}{c}\phi \left(\mathbf{r}\right)={\text{a}}{\text{r}}{\text{g}}\left(\sum_{m=1}^{N}{\text{exp}}\left(\text{i}\frac{2\uppi }{\lambda f}\mathbf{r}\cdot {\mathbf{b}}_{m}+\text{i}{\theta }_{m}\right)\right),\end{array}$$

(7)

where r = (x, y) is the coordinate vector in the back focal plane, λ is the wavelength of the trapping light, f is the focal length of the objective lens, b_m is the coordinates of a bead, and θ_m is a random phase assigned to the bead. The random phase θ_m improves diffraction efficiency and uniformity, especially for optical traps with a highly symmetric geometry. A zero-order beam would create an unmodulated Gaussian optical trap. As a result, beads that pass through its neighborhood will fall into the same trap and be lost. A blazed grating is used in the experiments to separate the target traps from the zero-order beam.

In the case where the number of beads N = 100 and the size of holograms is 1080 × 1080 pixels, the time to generate a hologram is about 10.8 ms. We performed 100 calculations in MATLAB software with the GPU (NVIDIA Quadro K2200) and then averaged them. In addition, all holograms had been computed in advance to achieve fast movement of beads. Therefore, the refresh rate of bead positions is limited only by the SLM response frequency.

Sample preparation

Fluorescent polystyrene (PS) microbeads (FluoSpheres Carboxylate 1.0 μm Yellow-green 505/515 nm, Thermo Fisher Scientific Inc., USA) were used as trapping objects in the experiments. We used slides, coverslips, and double-sided tape to make sample chambers in which a PS solution suspended in water was injected and then sealed with glue. The concentration of beads is minimized so that only a few beads are visible in the field of view. The depth of a sample chamber is controlled to be around 10 μm so that the beads are roughly in the same layer.

Target site generation

For arrays and geometric patterns, target coordinates were automatically generated by using MATLAB (R2021a). For complex patterns (Fig. 4b-d), target coordinates were manually designed by using SolidWorks (2018). Because particles with too small a spacing are prone to form barriers, the APF falls into a local optimum. The spacing of target sites we designed is not less than 1.5 times the diameter of beads (1 µm in our case).

Experiment setup

The HOTs setup was used to conduct optical trapping experiments as reported in our previous work [37]. As shown in Extended Data Fig. 1, the trapping beam from a 1064-nm laser (5000 mW; VFLS-1064-B-SF-HP, Connet Laser Technology Co., Ltd., China) was modulated by a phase-only liquid crystal SLM (1920 × 1080 pixels, 8-μm pixel pitch; PLUTO-2-NIR-049, Holoeye Photonics AG, Germany) and relayed through a 4-f system into the back focal plane of the objective lens (Apo Plan IR, 60 × /NA1.27, Water Immersion, Nikon Corp., Japan). The Gauss-point optical trap array were finally generated in the front focal plane after the Fourier transform. The excitation beam from a 473-nm laser (500 mW; gem 473, Laser Quantum Ltd., UK) was expanded and collimated by a telescope and then utilized to excite the fluorescence of the sample. The fluorescent beam from the sample was separated by the dichroic mirrors DM1 (Di03-R488/561-t1-25 × 36, Semrock Inc., USA) and DM2 (Di02-R1064-25 × 36, Semrock Inc., USA) and finally detected by an sCMOS camera (2048 × 2048 pixels, 6.5 μm pixel pitch; ORCA-Flash 4.0 V2 C11440-22CU, Hamamatsu Photonics K.K., Japan). A motorized XY translation stage (PZ-2000FT, Applied Scientific Instrumentation Inc., USA) and a piezoelectric Z-axis stage (PZ-2500FT, Applied Scientific Instrumentation Inc., USA) were used to move the sample in three dimensions. A home-made software developed with Qt (5.14.2) and MATLAB (R2021a) was employed to synchro control the whole system.

Initial trapping

At the beginning of experiments, microbeads were automatically trapped by feature recognition. This was done by first acquiring a fluorescence image, then identifying the microbeads in a certain region at the center of the field of view and obtaining the positions of their centers of mass, and finally generating the corresponding hologram and addressing it to the SLM. In addition, adding more beads or deleting trapped beads could be achieved by mouse-clicking on the camera's live display window. The initial positions of microbeads were randomly distributed. Figure 4a shows an example where beads in the 20 μm × 20 μm area at the center of the field of view were automatically trapped. Adding additional beads by clicking with the mouse, eventually a total of 64 beads were trapped. For all trapping experiments, the optical power of each trap was typically 7 ~ 10 mW.

Fluorescent image acquisition

A 473-nm laser was used to excite the fluorescence of the PS beads, and the fluorescence images were acquired by using an sCMOS camera. The image acquisition was synchronized with the movement of the microbeads. An image was recorded for each travel step of microbeads. The time for microbeads to move one step was about 100 ms, and the exposure time of the camera was 100 ms.

Statistics and reproducibility

All running times were obtained by averaging 100 calculations. Imaging data were processed with Fiji (2.3.0) or MATLAB (R2021a) and were plotted with Origin (2018). Videos were created by using experimental images with Adobe Premiere (2018).