- Perspective
- Open access
- Published:
Lensless fiber endomicroscopy in biomedicine
PhotoniX volume 5, Article number: 18 (2024)
Abstract
Lensless fiber endomicroscopy, an emergent paradigm shift for minimally-invasive microscopic optical imaging and targeted light delivery, holds transformative potential, especially in biomedicine. Leveraging holographic detection and physical or computational wavefront correction, it enables three-dimensional imaging in an unprecedentedly small footprint, which is crucial for various applications such as brain surgery. This perspective reviews the recent breakthroughs, highlighting potential emerging applications, and pinpointing gaps between innovation and real-world applications. As the research in this realm accelerates, the novel breakthroughs and existing frontiers highlighted in this perspective can be used as guidelines for researchers joining this exciting domain.
The increasing demand in medical diagnostics and treatment for minimally-invasive procedures coupled with high-resolution imaging has been a driving force in fiber-based endomicroscopy. Conventional endoscopic imaging methods face a trade-off between imaging resolution, field of view, number of modalities and invasiveness. Complex mechanical-scanners or imaging lenses at the endoscope distal side, i.e. the application side inside the body, are often required for high-resolution imaging, which typically results in more invasive probes, potentially causing increased tissue damage, which must be minimized, especially in brain surgery.
Lensless fiber endomicroscopes, empowered by advanced computational imaging methods [1,2,3,4] and programmable optics [5, 6], can overcome this trade-off and enable high-resolution imaging through tiny probes which show negligible tissue-damage. Utilizing a flexible ultra-thin fiber, such lensless endomicroscopes can easily navigate through small openings or penetrate in a needle-like minimally-invasive manner through delicate tissues, making them ideal for biomedical microscopic imaging in situ. Imaged volumes may include tiny cavities such as in the lung, prostate, kidney, bladder, blood vessels, eye and ear especially the cochlea, as well as delicate tissue for instance in the brain for tumor classification [6], neuroimaging [7,8,9] and stimulation [10]. Lensless fiber endomicroscopes open the door for dynamic imaging in vivo, providing real-time insights into cellular and even subcellular processes. Most importantly, fiber-based endoscopes enable controlled light delivery, allowing optical excitation, nonlinear contrast mechanisms, micro-targeted optogenetic stimulation [10], optical manipulation [11], photo-therapy and further applications like photo-acoustics. These emerging capabilities could revolutionize early disease detection, targeted biopsies, and the monitoring of treatment responses, significantly improving patient care in clinical settings.
In this perspective, we focus on emerging lensless fiber endomicroscopic imaging technologies employing either multicore fibers (MCFs) or multimode fibers (MMFs). We outline the strengths and limitations of these imaging methods, while exploring their potential for transfer to practical applications in real-world scenarios. This perspective will discuss potential future advancements in lensless fiber endomicroscopes, exploring the potential technological breakthroughs and their transfer to applications towards clinics.
Commercial endoscopes often use miniaturized cameras and lenses at the probe’s tip and offer relatively high-resolution 2D imaging at a fixed focal length. However, such ‘chip on tip’ designs limit advanced imaging capabilities due to constraints in illumination control. In contrast, lensless fiber endomicroscopes use optical fibers as waveguides for both light delivery and detection, allowing light sources and detectors to be positioned externally, eliminating the need for integrating complex components on the probe, significantly minimizing footprint. While miniature lenses or metalenses can be fabricated at a conventional single-core fiber end, distal scanners would still be required for imaging [12]. Avoiding distal scanners requires multiple spatial transmission channels offered by an MMF or MCF, and holographic detection or control of the light fields at the proximal side.
MMFs can transmit thousands of spatial modes simultaneously, thus providing a high space-bandwidth product, i.e. effective number of simultaneously imaged pixels, which potentially enables high-resolution diffraction-limited imaging at probe diameters of only 60 μm [14]. However, the complex light propagation due to modal coupling and modal dispersion presents the most prominent challenge for imaging through MMFs. Especially, since it depends also on wavelength, polarization and fiber conditions such as temperature and bending [7, 9, 14]. While the propagation through MMFs is complex, it is not random. It is characterized by a transmission matrix (TM), which defines the relationship between the input and output fields. The TM can be directly measured by characterizing the light propagation through the fiber, for instance via holography. In general, as many measurements as transferable modes are required for full characterization, however, if the TM is known or can be approximated, the fiber distortions can be corrected for either by programmable optics, such as spatial light modulators [14] or computationally via direct calculation [15] or deep neural networks [16].
MCFs, also called coherent fiber bundles, typically consists of thousands of independent single-mode fiber cores, compactly arranged within a diameter of a few hundred microns. MCFs use each core as a separate spatial channel to transfer information in parallel. The cores ideally exhibit negligible crosstalk compared to the modal-coupling in MMFs, at the price of a lower fill factor due to the required core-to-core spacing. Each core can effectively act as an individual pixel, allowing direct transmission of intensity information independent of fiber bending (Fig. 1d). However, the core-to-core spacing leads to under-sampling of spatial information. Neural networks have been employed to address this issue and ‘inpaint’ the missing pixels for e.g. cancer diagnostics [17]. However, similar to MMFs, intercore dispersion in MCFs leads to bend-dependent spatial-phase distortions, making light field transmission through MCFs with amplitude, phase and polarization a challenge too. Overcoming this challenge is possible by wavefront corrections utilizing a spatial light modulator (Fig. 1c). One such approach is digital optical phase conjugation (DOPC) [5, 18]. Different from MMFs which commonly require a full TM-based correction, a single phase mask is typically sufficient for correcting the distortions of a static MCFs, enabling precise and high-fidelity focusing and imaging by exploiting the memory effect. For dynamically-bent MCFs, an in situ estimation of the distortions is required. This can be obtained by nonlinear signal optimization [19], or a virtual guide star [5]. The capability to undo MCF phase-distortions facilitates the use of MCFs in advanced imaging modalities, such as scanning fluorescence [3], two-photon [19], and holographic imaging [1,2,3,4], significantly enhancing their effectiveness and image quality. The ability to correct the excitation-path distortions by wavefront-shaping opens the path for converting any fiber to a scanning laser endoscope using no mechanical scanners at the distal end. Lensless, video-rate, cellular-resolution imaging [5] has already demonstrated great potential for early cancer detection and tumor diagnosis [17].
Excitation-path correction also allows to work within the limited spectral bandwidth of the wavefront correction, which is dictated by the wavelength dependence of the light propagation through long fibers [19]. Importantly, raster-scanned focused excitation enables nonlinear imaging, such two-photon excited fluorescence [19], and coherent anti-stokes Raman Scattering (CARS) [20], offering significant applications in deep-tissue imaging and cell structure visualization. Two-photon imaging provides deeper tissue penetration and three-dimensional imaging, which is particularly useful in urology for bladder-wall examinations where deeper tissue analysis is required with minimal probe size. As a label-free technique, CARS offers chemically-specific imaging, crucial for observing molecular details without external markers. The fiber-optic micromechanical characterization of tissues by Brillouin scattering using flexible catheters is also very interesting, but has not yet been translated into practical applications [21].
Wavefront shaping can be used to correct the detection path as well. While the correction is employed for spatially-incoherent light [22], the narrow-band nature of the correction is beneficial for coherent reflection or transmission-contrast, opening the path to label-free quantitative phase imaging [1]. Phase-contrast is often a key in revealing cellular and tissue morphology. While MMFs present challenges due to their complex light propagation, MCFs present a more feasible alternative for in situ corrections. Recent advancements include proximal holography for reflection-matrix measurement and decomposition [3] and distal phase-shifting holography [2], enhancing imaging capabilities through dynamically bent MCFs. Holographic lensless endomicroscopy significantly enhances the functionality of endoscopes by enabling refocusing capabilities and extending the working distance. This allows for detailed imaging over a broader range of depths, crucial for applications like complex tissue analysis and intricate surgical procedures. Its ability to computationally adjust the focus after image capture provides versatility in examining areas that are difficult to access, making it especially useful in neurosurgery, inner ear investigation, and early cancer diagnostics.
The ability to correct the excitation-path distortions allows not only tight focusing for imaging, but also engineered light field delivery to any chosen targets without distal optical elements. A straightforward potential application is in vivo tumor ablation, the targeted destruction of cancerous cells by concentrating light with extreme precision, while minimizing damage to surrounding healthy tissues [23]. Another attractive emerging application is optogenetics with cellular resolution selectivity, where wavefront shaping allows the precise delivery of light to control genetically modified cells, often neurons. This ability is essential for effectively stimulating or inhibiting neuronal activity with high spatial precision, offering significant implications for neuroscience research and the development of therapies for neurological disorders [10]. Both, ablation and optogenetics can benefit significantly by the combination of manipulation and imaging with the same probe, for direct process control. Finally, a wavefront-shaped focus can be used to realize dynamically-controlled optical tweezers, a tool employed in the manipulation of cells, nanoparticles, and microrobots with remarkable accuracy. One example is the rotation of cells in three dimensions for precise optical tomography, providing detailed 3D visualization of subcellular structure, enhancing our understanding of cancer cells toward flow cytometry in clinics [11].
Lensless fiber endomicroscopy, while in its nascent stages, presents a world of opportunities. As the field progresses, a balance between innovation, evaluation, and a focus on real-world applications will be critical. This perspective underscores the importance of not only championing technological advancements but also critically evaluating their broader clinical and societal implications.
From deep brain imaging to real-time cancer diagnosis and in vivo intestinal imaging, the potential applications of these technologies are vast, as the diverse application fields listed in Fig. 2. However, the journey from lab-scale innovations to real-world medical applications is fraught with challenges. The reduction of the fiber endoscope’s footprint, achieved by removing the distal optics, is accompanied by a decrease in the detection aperture, which can lead to reduced coupling efficiency and signal-to-noise ratio (SNR) according \(SNR\sim NA^2,NA\sim R/Z\), with R as radius of the fiber and Z as the distance from the fiber facet to the tissue surface. In order to get a sufficient SNR, the distance Z has to be reduced, which also makes the field of view smaller. Furthermore, the current solutions are often limited to narrow linewidth light field correction, reducing realizable imaging modalities for fluorescence. Lastly, the realized systems are complex, either in hardware or in computation, which so far hindered translation to clinical applications. Addressing these challenges may require new fiber designs, such as 3D-printed diffractive optical elements (DOEs) on the fiber tip [24, 25], or innovative computational methods such as deep learning [26]. Additionally, incorporating advanced functionalities like microfluidics, active fibers, and microactuators will expand the scope, applicability, and versatility of lensless fiber endomicroscopy.
In the design and application of lensless fiber endomicroscopes, a critical trade-off exists between robustness, resolution, and diameter. These factors are interdependent and must be carefully balanced to meet the specific needs of different application scenarios. Firstly, the diameter of the fiber plays a crucial role in determining both the resolution and robustness. MMFs usually have smaller diameters down to 60 μm, providing high potential for high resolution imaging. However, MMFs are generally more sensitive to mechanical bending and environmental changes, which can adversely affect image quality. In contrast, MCFs tend to offer greater robustness but at the cost of relatively lower resolution with a slightly larger diameter down to 250 μm. Furthermore, the application scenario greatly influences the choice of fiber. For instance, in medical applications such as intravascular or gastrointestinal imaging, where the fiber may be subjected to significant bending and twisting, a more robust fiber system may be preferable even if it sacrifices some resolution. This ensures reliability and consistency of imaging performance, which is critical in a clinical setting. On the other hand, applications like precision industrial inspection or laboratory research might prioritize high resolution over robustness because the fibers can be controlled and protected from extreme environmental conditions.
Once the technical challenges of lensless fiber endomicroscopy are overcome, it’s crucial to bridge the gap between innovation and practical implementation. This involves ensuring ease of use, cost-effectiveness, and compatibility with existing medical workflows and regulations. Future developments should focus on making these devices more user-friendly for clinicians, reducing costs to promote broader adoption, and ensuring compatibility with current medical imaging and diagnostic practices, especially regarding deep learning. Collaborations between researchers, clinicians, and industry are crucial in translating these advanced technologies from the lab to the clinic, ensuring they meet practical clinical needs while leveraging their advanced imaging capabilities.
As we embrace the future of lensless fiber endomicroscopy, we stand on the brink of a transformative era in medical imaging. The potential for minimally invasive, high-resolution diagnostics and treatments is immense. With continued innovation, collaboration, and a focus on practical implementation, this technology promises to revolutionize patient care and open new horizons in medical science. The journey ahead is filled with possibilities, and the medical community eagerly anticipates its impactful integration into clinical practice.
Conclusions
Lensless fiber endomicroscopy represents a significant shift towards minimally invasive optical imaging and targeted light delivery, with vast implications in biomedicine. Integrating advanced optical technologies such as holographic detection and wavefront correction facilitates optical manipulation and three-dimensional imaging through the ultra-thin fiber. Lensless fiber endoscopes enable microscopic imaging to extremely narrow or previously inaccessible areas, opening new perspectives for critical medical procedures, from brain surgery to vascular diagnostics. We highlight recent advancements and potential applications, while also identifying gaps between innovation and practical implementation. We are confident that the insights offered here will pave the way for further advancements in fiber endomicroscopy, steering future research in biomedicine.
Availability of data and materials
This paper is based on previously published studies, data, and materials, all of which are cited within the manuscript. As such, this paper does not contain any new data.
References
Sun J, Wu J, Wu S, Goswami R, Girardo S, Cao L, Guck J, Koukourakis N, Czarske JW. Quantitative phase imaging through an ultra-thin lensless fiber endoscope. Light Sci AppL. 2022;11(1 11):1–10.
Badt N, Katz O. Real-time holographic lensless micro-endoscopy through flexible fibers via fiber bundle distal holography. Nat Commun. 2022;13(1):6055.
Choi W, Kang M, Hong JH, Katz O, Lee B, Kim GH, Choi Y, Choi W. Flexible-type ultrathin holographic endoscope for microscopic imaging of unstained biological tissues. Nat Commun. 2022;13(1):4469.
Du Y, Turtaev S, Leite IT, Lorenz A, Kobelke J, Wondraczek K, Čižmár T. Hybrid multimode-multicore fibre based holographic endoscope for deep-tissue neurophotonics. Light Adv Manuf. 2022;3(3):408–16.
Kuschmierz R, Scharf E, Koukourakis N, Czarske JW. Self-calibration of lensless holographic endoscope using programmable guide stars. Opt Lett. 2018;43(12): 2997.
Wen Z, Dong Z, Deng Q, Pang C, Kaminski CF, Xu X, Yan H, Wang L, Liu S, Tang J, Chen W, Liu X, Yang Q. Single multimode fibre for in vivo light-field-encoded endoscopic imaging. Nat Photonics 2023. 2023;17(8):8.
Vasquez-Lopez SA, Turcotte R, Koren V, Plöschner M, Padamsey Z, Booth MJ, Čižmár T, Emptage NJ. Subcellular spatial resolution achieved for deep-brain imaging in vivo using a minimally invasive multimode fiber. Light Sci Appl 2018. 2018;7(1 7):1–6.
Papadopoulos IN, Farahi S, Moser C, Psaltis D, Flusberg B, Cocker E, Piyawattanametha W, Jung J, Cheung E, Schnitzer M, Mehta A, Aksay E, Stepnoski R. High-resolution, lensless endoscope based on digital scanning through a multimode optical fiber. Biomedical Opt Express. 2013;4(2):260–70.
Stibůrek M, Ondráčková P, Tučková T, Turtaev S, Šiler M, Pikálek T, Jákl P, Gomes A, Krejčí J, Kolbábková P, Uhlířová H, Čižmár T. 110 µm thin endo-microscope for deep-brain in vivo observations of neuronal connectivity, activity and blood flow dynamics. Nat Commun 2023. 2023;14(1 14):1–9.
Accanto N, Blot FGC, Lorca-Cámara A, Zampini V, Bui F, Tourain C, Badt N, Katz O, Emiliani V. A flexible two-photon fiberscope for fast activity imaging and precise optogenetic photostimulation of neurons in freely moving mice. Neuron. 2023;111(2):176-e1896.
Sun J, Yang B, Koukourakis N, Guck J, Czarske JW. AI-driven projection tomography with multicore fibre-optic cell rotation. Nat Commun. 2024;15(1):147.
Gissibl T, Thiele S, Herkommer A, Giessen H. Two-photon direct laser writing of ultracompact multi-lens objectives. Nat Photonics 2016. 2016;10(8):8.
Wang C, Dudman JT, Ji N, Jiang W, Aponte Y, Bocarsly ME. Minimally invasive microendoscopy system for in vivo functional imaging of deep nuclei in the mouse brain. Biomed Opt Express. 2015;6(11):4546–56.
Turtaev S, Leite IT, Altwegg-Boussac T, Pakan JMP, Rochefort NL, Čižmár T. High-fidelity multimode fibre-based endoscopy for deep brain in vivo imaging. Light Sci Appl. 2018;7(1):1–8.
Choi Y, Yoon C, Kim M, Yang TD, Fang-Yen C, Dasari RR, Lee KJ, Choi W. Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber. Phys Rev Lett. 2012;109(20): 203901.
Moser C, Psaltis D, Kakkava E, Borhani N. Learning to see through multimode fibers. Optica. 2018;5(8):960–6.
Wu J, Wang T, Uckermann O, Galli R, Schackert G, Cao L, Czarske J, Kuschmierz R. Learned end-to-end high-resolution lensless fiber imaging towards real-time cancer diagnosis. Sci Rep. 2022;12(1):18846.
Thompson AJ, Paterson C, Neil MAA, Dunsby C, French PMW. Adaptive phase compensation for ultracompact laser scanning endomicroscopy. Opt Lett. 2011;36(9):1707–9.
Weiss U, Katz O. Two-photon lensless micro-endoscopy with in-situ wavefront correction. Opt Express. 2018;26(22): 28808.
Trägårdh J, Pikálek T, Šerý M, Meyer T, Popp J, Čižmár T. Label-free CARS microscopy through a multimode fiber endoscope. Opt Express. 2019;27(21): 30055.
Xiang Y, Basirun C, Chou J, Warkiani M, Török P, Wang Y, Gao S, Kabakova I. Background-free fibre optic brillouin probe for remote mapping of micromechanics. Biomed Opt Express. 2020;11(11):6687–98.
Yeminy T, Katz O. Guidestar-free image-guided wavefront shaping. Sci Adv. 2021;7(21):5364–83.
Kakkava E, Romito M, Conkey DB, Loterie D, Stankovic KM, Moser C, Psaltis D. Selective femtosecond laser ablation via two-photon fluorescence imaging through a multimode fiber. Biomed Opt Express. 2019;10(2):423.
Kuschmierz R, Scharf E, Ortegón-González DF, Glosemeyer T, Czarske JW, Kuschmierz R, Scharf E, Ortegón-González DF, Glosemeyer T, Czarske JW. Ultra-thin 3D lensless fiber endoscopy using diffractive optical elements and deep neural networks. Light: Adv Manuf. 2021;2(4):1–10.
Fröch JE, Huang L, Tanguy QAA, Colburn S, Zhan A, Ravagli A, Seibel EJ, Böhringer KF, Majumdar A. Real time full-color imaging in a Meta-optical fiber endoscope. eLight. 2023;3(1):13.
Sun J, Zhao B, Wang D, Wang Z, Zhang J, Koukourakis N, Czarske J, Li X. Calibration-free quantitative phase imaging in multi-core fiber endoscopes using end-to-end deep learning. Opt Lett. 2024;49(2):342–5.
Acknowledgements
We would like to acknowledge the kind invitation and support from the PhotoniX editorial office.
Funding
Open Access funding enabled and organized by Projekt DEAL. German Research Foundation (DFG) grant CZ55/40, CZ55/47, European Research Council (ERC) Horizon 2020 research and innovation program (grant no. 101002406), Shanghai Artificial Intelligence Laboratory, National Key R&D Program of China (2022ZD0160100), Else Kröner Fresenius Center for digital Heath (EKFZ) and Competence Center for Biomedical Computational Laser Systems (BIOLAS).
Author information
Authors and Affiliations
Contributions
J.S., J.C. conceptualized the manuscript. J.S., R.K., O.K. prepared the manuscript. N.K., J.C. revised the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
Authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Sun, J., Kuschmierz, R., Katz, O. et al. Lensless fiber endomicroscopy in biomedicine. PhotoniX 5, 18 (2024). https://doi.org/10.1186/s43074-024-00133-8
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1186/s43074-024-00133-8