Transformer-based robotic ultrasound 3D tracking for capsule robot in GI tract

Otuya DO, Verma Y, Farrokhi H, Higgins L, Rosenberg M, Damman C, Tearney GJ (2018) Non-endoscopic biopsy techniques: a review. Expert Rev Gastroenterol Hepatol 12(2):109–117. https://doi.org/10.1080/17474124.2018.1412828

Article  CAS  PubMed  Google Scholar 

Hu C, Meng MQ, Mandal M (2005) Efficient magnetic localization and orientation technique for capsule endoscopy. In: 2005 IEEE/RSJ international conference on intelligent robots and systems, IROS. IEEE, Edmonton, pp 628–633. https://doi.org/10.1109/IROS.2005.1545490

Wang Q, Yang L, Yu J, Chiu P, Zheng YP, Zhang L (2020) Real-time magnetic navigation of a rotating colloidal microswarm under ultrasound guidance. IEEE Trans Biomed Eng 67(12):3403–3412. https://doi.org/10.1109/TBME.2020.2987045

Article  PubMed  Google Scholar 

Kimmey MB, Martin RW, Haggitt RC, Wang KY, Franklin DW, Silverstein FE (1989) Histologic correlates of gastrointestinal ultrasound images. Gastroenterology 96(2 PART 1):433–441. https://doi.org/10.1016/0016-5085(89)91568-0

Article  CAS  PubMed  Google Scholar 

Nasir AI (2018) Artifact in the image of ultrasound. Aust J Basic Appl Sci 12(12):131–143. https://doi.org/10.22587/ajbas.2018.12.12.21

Article  Google Scholar 

Pourtaherian A, Ghazvinian Zanjani F, Zinger S, Mihajlovic N, Ng GC, Korsten H, With P (2018) Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks. Int J Comput Assist Radiol Surg 13(9):1321–1333. https://doi.org/10.1007/s11548-018-1798-3

Article  PubMed  PubMed Central  Google Scholar 

Yang H, Shan C, Kolen AF, With P (2019) Catheter localization in 3D ultrasound using voxel-of-interest-based ConvNets for cardiac intervention. Int J Comput Assist Radiol Surg 14(6):1069–1077. https://doi.org/10.1007/s11548-019-01960-y

Article  PubMed  PubMed Central  Google Scholar 

Yang L, Zhang M, Yang Z, Yang H, Zhang L (2022) Mobile ultrasound tracking and magnetic control for long-distance endovascular navigation of untethered miniature robots against pulsatile flow. Adv Intell Syst 4(3):2100144. https://doi.org/10.1002/aisy.202100144

Article  Google Scholar 

Du X, Wang Q, Jin D, Chiu P, Pang CP, Chong K, Zhang L (2022) Real-time navigation of an untethered miniature robot using mobile ultrasound imaging and magnetic actuation systems. IEEE Robot Autom Lett 7(3):7668–7675. https://doi.org/10.1109/LRA.2022.3184445

Article  Google Scholar 

Lu Y, Zhao H, Becker AT, Leclerc J (2022) Steering rotating magnetic swimmers in 2.5 dimensions using only 2D ultrasonography for position sensing. IEEE Robot Autom Lett 7(2):3162–3169. https://doi.org/10.1109/LRA.2022.3146560

Article  Google Scholar 

Pane S, Faoro G, Sinibaldi E, Iacovacci V, Menciassi A (2022) Ultrasound acoustic phase analysis enables robotic visual-servoing of magnetic microrobots. IEEE Trans Rob 38(3):1571–1582. https://doi.org/10.1109/TRO.2022.3143072

Steinsvik EK, Hatlebakk JG, Hausken T, Nylund K, Gilja OH (2021) Ultrasound imaging for assessing functions of the GI tract. Physiol Meas. https://doi.org/10.1088/1361-6579/abdad7

Article  PubMed  Google Scholar 

Yang H, Shan C, Kolen AF, De With P (2021) Efficient medical instrument detection in 3D volumetric ultrasound data. IEEE Trans Biomed Eng 68(3):1034–1043. https://doi.org/10.1109/TBME.2020.2999729

Article  PubMed  Google Scholar 

De Oliveira A, Batista J, Misra S, Venkiteswaran VK (2022) Ultrasound tracking and closed-loop control of a magnetically-actuated biomimetic soft robot. IEEE Int Conf Intell Robots Syst 2022–Octob:3422–3428. https://doi.org/10.1109/IROS47612.2022.9981635

Article  Google Scholar 

Luo W, Li Y, Urtasun R, Zemel R (2016) Understanding the effective receptive field in deep convolutional neural networks. Adv Neural Inf Process Syst (Nips), 4905–4913

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 2017–Decem(Nips):5999–6009

Google Scholar 

Bhojanapalli S, Chakrabarti A, Glasner D, Li D, Unterthiner T, Veit A (2021) Understanding robustness of transformers for image classification. In: Proceedings of the IEEE international conference on computer vision, pp 10211–10221. https://doi.org/10.1109/ICCV48922.2021.01007

He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. CoRR arXiv:1512.03385

Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby N (2020) An image is worth 16X16 words: transformers for image recognition at scale. In: ICLR 2021 - 9th international conference on learning representations. arXiv:2010.11929

Liu X, Esser D, Wagstaff B, Zavodni A, Matsuura N, Kelly J, Diller E (2022) Capsule robot pose and mechanism state detection in ultrasound using attention-based hierarchical deep learning. Sci Rep. https://doi.org/10.1038/s41598-022-25572-w

Article  PubMed  PubMed Central  Google Scholar 

Shokrollahi P, Lai YP, Rash-Ahmadi S, Stewart V, Mohammadigheisar M, Huber LA, Matsuura N, Zavodni A, Parkinson J, Diller E (2021) Blindly controlled magnetically actuated capsule for noninvasive sampling of the gastrointestinal microbiome. IEEE/ASME Trans Mechatron 26(5):2616–2628. https://doi.org/10.1109/TMECH.2020.3043454

Article  Google Scholar 

Comments (0)

No login
gif