Emerging Trends in Artificial Intelligence-Based Urological Imaging Technologies and Practical Applications
Article information
Abstract
The integration of artificial intelligence (AI) into medical imaging has notably expanded its significance within urology. AI applications offer a broad spectrum of utilities in this domain, ranging from precise diagnosis achieved through image segmentation and anomaly detection to improved procedural assistance in biopsies and surgical interventions. Although challenges persist concerning data security, transparency, and integration into existing clinical workflows, extensive research has been conducted on AI-assisted imaging technologies while recognizing their potential to reshape urological practices. This review paper outlines current AI techniques employed for image analysis to offer an overview of the latest technological trends and applications in the field of urology.
INTRODUCTION
In recent years, medical imaging has come to play a central and essential role in the diagnosis and treatment of various diseases, in particular, within the field of urology. The development of urologic imaging can be traced back to Roentgen’s discovery of the x-ray in 1895 [1], and since then, numerous innovations in imaging technology have brought about significant changes in how urological patients are diagnosed and managed. As a result, a technology that previously demanded specialized expertise to be proficient has now evolved into an essential component of urological practice. Various imaging techniques including microscopy, ultrasound (US), x-rays, computed tomography (CT), and magnetic resonance imaging (MRI) are now employed to accurately diagnose and treat patients with urologic conditions.
Furthermore, with the integration of artificial intelligence (AI) into medical imaging, its importance is on the rise [2-4]. In the field of urology, AI provides a wide range of applications from precise diagnosis achieved through image segmentation and anomaly detection, to procedures such as biopsies and surgical interventions where assisted navigation and robotic guidance tools are employed. For example, by employing AI for urology image analysis, the identification of abnormalities can be achieved with higher accuracy compared to traditional methods that involve analyzing extensive image datasets [5]. This leads to more precise diagnoses and facilitates early intervention for treatment planning. AI can also enhance the detection, segmentation, classification, and diagnosis of abnormalities. For instance, AI algorithms can effectively recognize and categorize various disease sites, such as prostate cancer in MRI images, which has significant implications that the potential of AI in identifying predictive and prognostic biomarkers in prostate cancer is high. Thus, AI-aided imaging technology can significantly enhance urologists’ decision-making by improving accuracy while saving both time and resources.
Notwithstanding these advantages, several challenges must be addressed before full integration into clinical practice is feasible. Firstly, substantial quantities of high-quality data are necessary to adequately train AI algorithms. However, the use of personalized data raises important ethical concerns, such as data security and liability. In addition, identifying which specific AI algorithm produced a particular diagnosis or treatment poses risks in terms of interpretation and its transparency. Moreover, the widespread adoption of AI in urology demands the integration of AI technology into existing clinical workflows, which involves additional costs and training to modify established practices.
There is considerable potential for AI in analyzing urologic images, and continual efforts are directed towards implementing AI in the field of urology. In this regard, this review paper presents the current AI techniques employed for image analysis and provides an overview of the latest technological trends and applications in the field of urology.
AI TECHNOLOGIES FOR IMAGE ANALYSIS IN UROLOGY
Machine Learning
Machine learning is a field of study that is dedicated to the development of algorithms capable of learning patterns from data and making predictions. This field encompasses various mathematical disciplines including probability theory, statistics, approximation theory, convex analysis, and algorithms. Within the domain of medical image diagnostics, machine learning involves the collection of extensive medical image data sources and the preprocessing of lesions to prepare them for labeling and analysis [6-8]. Then, it is employed to analyze medical images such as MRIs, CT scans, and ultrasounds for the detection of urinary diseases. For instance, it can be utilized in the identification and classification of tumors, such as those associated with prostate or bladder cancer. Thus, machine learning excels in rapidly and accurately scrutinizing radiological images to identify issues and uncover potential risks.
Support vector machine learning
In the realm of medical image recognition and classification, among machine learning algorithms to be employed, support vector machines (SVMs) are utilized to identify cancer by leveraging AI data processing for the recognition and detection of urine cell images [6]. In addition, SVMs are used to predict Polybromo-1 mutation status in renal cell carcinoma through CT texture analysis [9]. Moreover, research has been conducted to compare radiology classifiers and classifier ensembles for the detection of peripheral large prostate tumors in T2-weighted MRI using machine learning algorithms [10]. These studies also explore the differentiation between angiomyolipoma without visible fat and renal cell carcinoma (RCC) of the kidney [11]. Thus, SVM has been used to present a pipeline method for predicting cell mutational status and distinguishing between benign tumors, abnormal tissue growth, and early-stage kidney cancers preoperatively.
Convolutional neural network
In urology, convolutional neural networks (CNNs) are capable of precisely recognizing and analyzing color changes that are often challenging for humans to detect. Moreover, they can be trained to reliably judge color alterations under varying lighting conditions. Research utilizing CNN models in urology includes a spectrum of applications such as AI-assisted colorimetry to enhance the detection of glucose in urine [12], improved interpretation of MRI for diagnosing and staging prostate cancer [13]. For instance, in July 2023, the AI model that was created by researchers at University of California, Los Angeles (UCLA) Jonsson Comprehensive Cancer Center and department of UCLA Urology demonstrated a more precise prediction of tumor extent compared to MRI. This advancement holds the potential to enhance the effectiveness of local treatments, standardize treatment scope definition, and decrease the probability of cancer recurrence [14]. In addition, this CNN model helps to identify urinary tract extent and detect the presence of stones in CT images for urolithiasis treatment [15].
In the realm of bladder cancer research, active investigations involve assessing bladder cancer treatment response using tomography [16], predicting bladder tumor identification and grading via the RGB (red, green, blue) colors [17], and exploring the potential use of CNNs to more objectively evaluate cystoscopic images for classifying tumor lesions as normal or abnormal. This exploration aims to enhance the quality of bladder cancer diagnosis [18], or to improve tumor resection in transurethral resection of bladder tumors with high sensitivity and to aid in the early detection of bladder cancer. The outcomes of studies that use CNN algorithms offer accurate identification of urinary lesion areas, measurements of their size and shape, and potentially aid radiologists in prioritizing the interpretation of medical images [19].
Computer Vision
Computer vision refers to the technology that enables computers to extract information from images or videos and it facilitates analysis to recognize and comprehend real-world objects. In the domain of urology, computer vision plays a crucial role in identifying specific types of tumors or inflammation, in particular, concerning color changes in tissues. In practical applications, the validation of kidney stone removal results necessitates continuous tracking of stones throughout the procedure. The AI technology enables real-time visualization of stone locations on preoperative CT scans to provide a means to monitor and verify the effectiveness of the removal process [20].
Color histograms
The analysis of color histograms, rooted in computer vision techniques, is frequently employed to examine the color distribution within an image and it enables the determination of the proportional representation of specific colors [21]. Some studies aim to quantify intratissue blood flow and vascularization extending from the bladder neck to the urethral orifice [22] and to extract and analyze texture from CT images to aid in differentiating between angiomyolipoma and RCC [12].
Semantic segmentation
In the domain of computer vision, semantic segmentation is a technique that discerns individual pixels within an image or video and then classifies them into distinct object categories or regions [23]. The escalation in both the quantity and quality of medical imaging has underscored the significance of employing semantic segmentation techniques. This is attributed to the fact that manual image segmentation demands specialized knowledge, is time-consuming, and often costly, while automated methods can alleviate these challenges.
A dedicated semantic segmentation system was devised for each plane (anterior, posterior, and axial) of a CT image to heighten diagnostic accuracy [24]. Furthermore, semantic segmentation was utilized to analyze transvaginal images in real time with the purpose of detecting areas that might be overlooked by conventional cystoscopy, such as small or flat lesions [25]. The ongoing advancement and exploration of these techniques hold substantial promise in contributing to the early identification and treatment of bladder cancer [26].
Recognition and detection
Computer vision techniques play a pivotal role in identifying and precisely localizing specific objects, such as stones or tumors, within medical images, greatly assisting in treatment. Accurate identification of lesions in multiparametric MRI to estimate the Gleason score, a significant factor in assessing the prognosis of men with prostate cancer [27]. Utilizing computer vision for AI-aided cystoscopy through object recognition enhances the diagnostic capabilities during the procedure [28]. Moreover, the application of local binary patterns, a well-established technique in computer vision, particularly for pattern recognition and object detection, has been beneficial for enhanced cystoscopic image analysis and the diagnosis of bladder cancer [29,30]. The detailed images that illustrates AI algorithms applied to medical imaging in urology and their related cases are listed in Table 1.
AI algorithms applied to medical imaging in urology
AI-ASSISTED UROLOGICAL IMAGE ANALYSIS APPLICATIONS
AI-assisted urologic image analysis technology has rapidly emerged as a practical and invaluable tool within the medical field. Its applications range from automated lesion detection to aiding in surgical planning in order to facilitate surgical navigation, and even address patient comfort.
Diagnosis
In prostate cancer screening, which stands as the second most prevalent cancer in men, AI is playing a critical role in evaluating and interpreting data derived from prostate MRI images based on the Prostate Imaging Reporting and Data System (PIRADS). AI systems apply computer vision and deep learning (DL) techniques to pinpoint areas of interest including details such as prostate boundaries, zones, and tumors. For example, AI algorithms analyze tumors by assessing gradient, color distribution, and tissue homogeneity within the images [31]. This method assists radiologists in objectively calculating the PIRADS score, thereby refining prostate biopsy guidelines. These advancements provide crucial insights into critical details like cancer size, location, and status to empower doctors to make more informed decisions about the most suitable treatment options for their patients.
Recently, the integration of Philips’ AI-powered MR Smart-Speed image reconstruction software with Quibim’s AI-powered image analysis software has resulted in a comprehensive solution designed to expedite and simplify the treatment of prostate cancer for clinicians. The amalgamation allows for automated real-time segmentation of the prostate from MR images [32]. It produces valuable quantitative insights and standardizes the reporting of MR prostate exams. This integration has the potential to decrease the necessity for unnecessary biopsies and facilitate precise, targeted treatment for prostate cancer cases that require intervention. Also, it can help mitigate staffing shortages and reduce the overall cost of care to optimize the treatment process for prostate cancer patients.
Researchers also have combined colorimetric methods with AI to measure glucose levels in urine. This hybrid approach, merging colorimetry with CNNs, is capable of discerning subtle color variations and accurately determining glucose levels in urine samples. The study applied the Faster R-CNN object detection algorithm and Google’s Inception-V3 architecture to develop an application within a smartphone environment. This app possesses high color resolution, capable of distinguishing the glucose concentration in urine samples that may be challenging for the human eye to discern. It retains a high level of accuracy and sensitivity across varying environmental conditions so it presents potentially significant implications for diabetes monitoring [12].
Similarly, another study introduced a system designed to automatically detect and categorize urine color in real-time using computer vision and machine learning techniques, specifically the random forest (RF) algorithm. Utilizing a webcam, images of urine are captured and analyzed by a MATLB program (MathWorks, Massachusetts, USA) to identify its color. This color information is then classified using the RF algorithm, and the results are transmitted as a message to a healthcare provider or directly to the patient via an Arduino (Arduino LLC & Arduino SRL, Ivrea, Italy) and GSM module (SIMCom, Shanghai, China). This system’s capability to detect changes in urine color in real-time provides early warnings and health feedback based on the gathered information. The proposed urine color detection system could assist healthcare professionals in enhancing treatment strategies through early diagnosis, while empowering patients to actively monitor their conditions [33].
This year, Avenda Health introduced Unfold AI, an innovative cancer mapping platform designed to enhance the accessibility of personalized prostate cancer diagnosis and treatment. Unfold AI harmonizes patient-specific data derived from prostate imaging and biopsies to produce a tailored cancer probability map [34]. This 3-dimensional AI-generated map enables physicians to visually pinpoint the precise location of the cancer and to facilitate informed intervention planning. Moreover, the platform assists patients in making decisions regarding their cancer treatment moving forward.
Surgical Support Tools
AI-powered imaging technology can improve the surgical experience and result in better outcomes for patients. Improved surgical techniques can reduce complications and surgical video analytics can reduce the cost of care. Primarily, AI technology aids in training both novice and proficient surgeons by scrutinizing and reviewing surgical images [35]. Through the analysis of previous procedure recordings, surgical teams can identify errors and refine their expertise, particularly in learning advanced endoscopic techniques like endoscopic retrograde cholangiopancreatography. AI plays a role in improving performance in intricate procedures such as accessing the common bile duct, navigating the duodenum, and accurately inserting stents. This technology acts as a valuable tool in the continuous improvement and skill development of surgeons across various levels of expertise.
In addition, AI-powered imaging techniques have become integral in the realm of robotic surgery within urology. One such instance involves the application of DL and ensemble machine learning to forecast continence recovery post-robotic-assisted radical prostatectomy (RARP). Diverse machine learning methods, including Naïve Bayes, Support Vector Machine, Random Forest, and Artificial Neural Network, were utilized with two distinct approaches: the single model and ensemble model. These models used image features and clinical information to predict the occurrence of early or delayed incontinence [36]. Likewise, another study delved into the analysis of preoperative MRI data from patients to construct a model to forecast the severity and recovery rate of incontinence subsequent to RARP. The objective was to establish a system where a DL model could discern significant patterns from MRI data to predict a patient’s likelihood of rapid recovery from incontinence postsurgery [37]. And the system can offer patients more precise prognostic insights by predicting the likelihood of swift recovery from incontinence post-RARP. This information is particularly valuable for patients seeking to optimize their treatment strategies to prevent long-term incontinence.
A research team from Nagoya City University conducted a study, led by assistant professors Kazumi Taguchi and Shuzo Hamamoto, along with chair and professor Takahiro Yasui, to explore the efficacy of an AI-assisted robotic device called ANT-X (Intuitive Surgical, Sunnyvale, CA, USA) in renal access during kidney stone surgery [38]. This research aimed to compare the novel robotic-assisted fluoroscopic-guided (RAF) method to conventional ultrasound-guided percutaneous nephrolithotomy (PCNL). The study revealed that the RAF method, utilizing the ANT-X, significantly reduced the mean number of needle punctures and puncture duration compared to ultrasound-guided procedures. The reduction in needle punctures, crucial in preventing postoperative complications, suggests the potential for improved patient outcomes. The research highlighted the ANT-X’s ability to simplify the complex PCNL procedure, potentially enabling more doctors to perform it effectively, reducing the burden on experienced surgeons and possibly decreasing complications.
Patient-Oriented Systems
AI-based image analysis techniques are evolving to create patient-centric systems designed to seamlessly integrate into individuals’ daily routines to aid in the continual monitoring and enhancement of their health status. Among the emerging technologies, a highly investigated area involves AI-enabled smart toilets [7,39]. These toilets are engineered to provide analytics, facilitate disease monitoring, and issue automatic alerts. For instance, a significant study focuses on the development of an AIToilet for an Integrated Health Monitoring System. This innovative system employs smart sensors including a three-voltage sensor and an image sensor for prolonged monitoring and recording of personal health data via smart toilets. The system not only allows user identification but also supports the extraction and analysis of personalized health data [40]. In 2021, Duke University devised a prototype incorporating AI features into a conventional toilet that can enable the analysis of a patient’s stool. This technology provides gastroenterologists with vital information necessary for administering suitable treatment. AI toilets present an opportunity to address chronic gastrointestinal conditions like inflammatory bowel disease and irritable bowel syndrome. By collecting extended-term information, these AI toilets support more precise and timely diagnoses, ultimately enhancing the management of such conditions [41]. Motoilets bidet (Motoilet International, Hong Kong, China) is another AI-driven toilet system available in the market [42]. Controlled via a mobile app, this bidet analyzes health conditions by detecting stool types and subsequently provides personalized dietary recommendations directly to the patient’s mobile device.
In a study aiming to enhance the early detection and management of kidney disease, researchers developed a user-friendly smartphone application for quantifying urine creatinine. Utilizing the modern programming language Kotlin, the app analyzes average RGB colors by adjusting smartphone camera properties and deactivating automatic exposure and white balance. This application can identify object positions and colors in photos automatically to detect and analyze their hues. It offers the advantage of high portability across diverse environments and can be used by individuals without specialized training [43]. Additionally, there is a method employing smartphone-based colorimetric analysis of urine test strips. This approach is applicable to off-the-shelf test strips and does not require extra hardware. The primary objective of this study is to transition care into the home setting and to enhance the accuracy of urine testing, particularly crucial in prenatal care [44].
Healthy.io was introduced as an innovative app that transforms a smartphone camera into a high-precision scanner capable of analyzing urine test strips at a clinical level. With the Healthy.io kit, users urinate into a provided cup, dip the test strip, and subsequently utilize their smartphone camera to scan and interpret the test results with accuracy and clinical-level precision [45]. There is also an AI app that utilizes the sound of urine to monitor urological health. The app is reported to perform nearly as effectively as specialized machines used in clinics for listening to a patient’s urine and identifying abnormal flows and related health issues [46].
Another example is the Olive WatchOS app that links with an AI-powered real-time urine analysis device that can be attached to any toilet [47]. This Internet of Things device has the capability to identify biomarkers during regular urination and provide alerts for potential risks such as dehydration and urinary tract infections. Clinical trials have already confirmed its precision in measuring concentrations of molecules including albumin, red blood cell, nitrite, pH, specific gravity, and volume, which are pertinent to conditions like kidney stones, renal failure, and other health issues.
The instances where AI applied to medical imaging in urology along with detailed descriptions are outlined in Table 2. The patient-oriented system offers a non-invasive and ongoing health monitoring approach and foster early detection and preventative management of health issues by analyzing and continuously tracking health data within the user’s daily life. However, for this technology to gain widespread acceptance, additional considerations such as ensuring patient privacy, maintaining accuracy, and enhancing user convenience are crucial.
AI-assisted urological image analysis applications
CONCLUSIONS
Image analysis technology coupled with AI has significantly influenced the entire medical process in urology. It has revolutionized diagnosis, treatment, and surgical procedures, and empowered both patients and doctors to make more informed decisions. Through the precise analysis of urological imaging data including extensive images and videos, automated and unbiased evaluations facilitate early detection of urological diseases. This early detection not only aids in treatment planning but also enables effective postoperative patient monitoring.
Moreover, AI-assisted analysis streamlines image interpretation and allows radiologists and urologists to focus on more intricate cases. In addition, AI in image analysis contributes to personalized medicine by enabling routine examination of stool and urine, facilitating swift analysis and simple collection of individualized health data. These advancements not only promise improved patient health but also foster the growth of telemedicine.
However, the effective integration of AI-based image analysis into clinical practice necessitates the accumulation of more clinical evidence and its alignment with established clinical practice guidelines. Thus, the future trajectory of AI-driven urological image analysis will require ongoing research, expanded clinical trials, and the development of user-friendly products to fully harness the technology’s potential.
Notes
Grant/Fund Support
This work was supported by the Gachon University research fund of 2023 (GCU-202303660001) and Basic Science Research Program through the National Research Foundation of Korea (NRF) by the Ministry of Education (NRF-2022R1F1A1066602).
Conflict of Interest
No potential conflict of interest relevant to this article was reported.
AUTHOR CONTRIBUTION STATEMENT
· Conceptualization: HSK, JYK
· Data curation: HSK, EJK
· Formal analysis: HSK, EJK
· Funding acquisition: JYK
· Methodology: HSK, EJK, JYK
· Project administration: JYK
· Visualization: HSK, EJK
· Writing - original draft: HSK, EJK
· Writing - review & editing: HSK, EJK, JYK