THIR: Topological Histopathological Image Retrieval
Tabatabaei Z and Sporring J
Published on: 2025-12-27
Abstract
According to the World Health Organization, breast cancer claimed the lives of approximately 685,000 women in 2020. Early diagnosis and accurate clinical decision-making are critical in reducing this global burden. In this study, we propose THIR, a novel Content-Based Medical Image Retrieval (CBMIR) framework that leverages topological data analysis and Betti numbers derived from persistent homology to characterize and retrieve histopathological images based on their intrinsic structural patterns. Unlike conventional deep learning approaches that rely on extensive training, annotated datasets, and powerful GPU resources, THIR operates entirely without supervision. It extracts topological finger-prints directly from RGB histopathological images using cubical persistence, encoding the evolution of loops as compact, interpretable feature vectors. The similarity retrieval is then performed by computing the distances between these topological descriptors, efficiently returning the top-K most relevant matches.
Extensive experiments on the BreaKHis dataset demonstrate that THIR outperforms state-of-the-art supervised and unsupervised methods. It processes the entire dataset in under 20 minutes on a standard CPU, offering a fast, scalable, and training-free solution for clinical image retrieval.
Keywords
Content-based medical image retrieval; Histopathological images; BreaKHis; THIRIntroduction
Cancer is a leading cause of death worldwide, with nearly 10 million deaths in 2020, or nearly one in six deaths [1]. Breast cancer accounts for 25% of all cancers in women worldwide, and about 685,000 women lost their lives due to breast cancer in 2020 [2]. Histopathology is the gold standard for cancer diagnosis [3], which involves extracting tissue specimens from suspicious areas to prepare a glass slide for a microscopic examination [4]. However, this examination might have some human errors or intraobserver variability. Singh, et al. [5] made an extensive re- view of the errors in cancer diagnosis. Accurate cancer diagnosis and grading rely on many factors, including the knowledge, experience, and skills of pathologists [6], which can increase the rate of human error in diagnosis. Diagnosis is a high-risk area of errors, including missed, inappropriately delayed, or wrong diagnoses [6]. Digital pathology, by employing Deep Learning (DL) and Machine Learning (ML) techniques, has a significant impact on decreasing human errors by providing a second opinion for pathologists [7]. An image retrieval tool that finds cases with similar morphological features can help diagnose rare diseases and unusual conditions that may not have enough cases available to develop accurate supervised classification models [8].
While Content-Based Image Retrieval (CBIR) has been under investigation for decades [9], only with the emergence of digital pathology and DL, the studies have begun to focus on image search and analysis in histopathology [10]. Content-Based Medical Image Retrieval (CBMIR) offers a new approach to computational pathology [11]. CBMIR provides the top K with patches similar to the query from the previously diagnosed and treated cases. This can assist pathologists in tackling the above-mentioned errors. The main base of CBMIR is similarity measurement, which considers features including texture, shape, intensity, etc., and compares them with the previous cases [12]. This retrieval helps pathologists receive not only the labels, but also patches similar to their query. This can increase the explain ability of the methods since pathologists can analyse the texture and compare it with similar cases based on their expertise.
Visual examination of the patterns of the tissue in a monitor is a task usually relegated to pathologists or computational biologists. CBMIR can assist pathologists in analysing and managing a large volume of images to enhance diagnosis, collaboration, education, research, and the decision-making processes [13]. By searching and retrieving visually similar images with their labels, pathologists have more information to detect the abnormalities. CBMIR not only increases the accuracy of cancer diagnosis but also speeds up the process of consulting peers. It can support remote consultation and collaboration between pathologists from all over the world [14]. In addition, CBMIR is a crucial tool in research for exploring large histopathological image archives to discover and identify new patterns, trends, graphs, and correlations between cancer grades [13]. For example, it can be a teaching tool in histopathological education, allowing trainers to study and compare various histopathological patterns.
Figure 1: Shows the main workflow of a cbmir. It contains two main phases, offline and online. The same fe applies in both phases to extract features. Then, the euclidean distance as a distance measurement function is applied to find the top-3 similar patches. On top of the query image and the retrieved images, their labels are mentioned.
CBMIR consists of ranking images concerning a query image based on visual similarities with a typical workflow, as illustrated in Figure 1. Following [15], it has two phases: offline and online. The offline phase includes extracting features of the previous cases and indexing them. In the online phase, a query image, which is an unseen image for the Feature Extractor (FE), is fed to the same FE as in the offline phase to extract its features. To identify those most relevant images, a similarity function is applied between the extracted features of the query and the previous cases. Then, the top-K most relevant images from the previous cases are retrieved. This is similar to the traditional workflow in hospitals using Atlas books [16].
DL-based methods, particularly Convolutional Neural Networks (CNNs), have shown great success in CBMIR tasks by automatically learning hierarchical feature representations. However, these models often require large amounts of labelled data, are prone to over fitting, suffer from limited GPU resources, and lack interpretability. In this paper, THIR focuses on the topological information of images and texture features extracted from Topological Data Analysis (TDA). TDA provides hand-crafted, mathematically interpretable features, such as Betti values, that describe the global structure of images. While DL focuses on local patterns and appearance, TDA emphasizes global connectivity and shape, offering complementary information [17].
To the best of the authors’ knowledge, this is the first study to apply TDA to CBMIR in digital pathology. The main contributions of this paper are as follows:
- THIR is a fully unsupervised method that eliminates the need for labelled datasets, addressing one of the key challenges in medical image analysis.
- Through cubical persistence, we capture unique topological signatures by tracking how topological structures develop across the color channels of the images.
- THIR is a fully unsupervised method that eliminates the need for labelled datasets, addressing one of the key challenges in medical image analysis.
- Through cubical persistence, we capture unique topological signatures by tracking how topological structures develop across the color channels of the images.
Related Work
The performance of CBMIR mainly depends on the choice and performance of the FE method. A high- quality FE algorithm can improve the precision of the search engine [18]. There are some common descriptors, such as Scale-invariant feature transform (SIFT) [19], Local Binary Patterns (LBP) [20], Histogram of Oriented Gradient (HOG) [21], edge histogram descriptor [22], and Gabor filter [23] for texture analysis, which explore the local features of the images. Local features refer to the general pattern of images, such as a point, edge, or small image patch.
Artificial intelligence (AI) enabled CBMIR for an effective diagnosis in [24]. Then, as CNNs show their power and high effectiveness in extracting features, they have become increasingly used for CBMIR. DL- based models yield high-performance search engines by extracting features of images for tasks related to CBMIR. Many recent studies [25-28] were dedicated to exploring the performance of different DL-based methods in CBMIR.
Among the various types of DL-based methods, Auto Encoders (AEs), GANs, and Siamese net- works [29] have a special place as FE in the CBMIR task. In [14], the author reported 9.33 and 6.59 hours of training time for training a Convolutional Auto Encoder (CAE) and Federated Learning (FL) CAE, respectively. [14] Claims that FedCBMIR is faster compared to traditional CBMIR using the same CAE structure and the same GPU type (NVIDIA GeForce RTX 3090). FedCBMIR provides 98% accuracy, and the UCBMIR in [30] yields 93% accuracy at the top-5 on the BreaKHis data set. The Siamese network in [31] obtains 94% an F1-score at the top 5 retrievals for breast cancer. Authors in [32] in Google AI Healthcare proposed an automatic high- level feature extraction on prostate cancer. The obtained results were reported at the top-5 similar patches with an accuracy of 73%. Yottixel [33] uses the Dense Net structure, which is trained on the Image Net data set for extracting patches with- out being trained specifically for the CBMIR task.
RetCCL [34] proposes a method based on clustering feature vectors of the patches. In this work, a ResNet50 was trained using contrastive learning. In [35], a Graph Neural Network (GNN) encodes Region of Interest (ROI) graphs into representations using a contrastive loss function in a self-supervised manner. The study in [36] proposes size-scalable CBMIR from databases that consist of whole-slide images (WSIs). This method has addressed scalable retrieval frameworks tailored to WSIs, focusing on efficient indexing and patch-level comparisons to manage the immense size and complexity of histopathological data.
The authors in [37] provide an overview of the TDA methods in biomedicine. After reviewing the recently published literature, this study aims to explore the potential of TDA with CBMIR applications. To the best of our knowledge, this combination has never been explored for CBMIR tasks in breast cancer.
Material And Methodology
Our methodology consists of two steps. First, we extract the topological feature vectors from the data set and the query. Then, we apply a similarity measurement function to these vectors to find the top-K similar patches to the query from the data set. This provides a search engine based on the images’ topological information, resulting in a fast and interpretable model called THIR. The implementation in this study focuses on breast cancer. Figure 2 and Figure 3 illustrate the whole workflow of the proposed methodology.
Data Set
BreaKHis data set [38] was created in the PD lab- oratory in Prana, Brazil, and consists of 7909 microscopic images of breast cancer. This collection contains four different magnifications (40×, 100×, 200×, and 400×) 1 as shown in Table 1. In this bi-nary data set, tissues were stained with Hematoxylinand Eosin (H&E), which is the most common color in histopathological images [39]. Following the previous studies [14, 30, 31] to be able to have comparable results, we resized the images into 240×240×3. Since the images are from the cancerous tissue, they are not affected by image transforming, inverting, zooming in, or rotation by 90 degrees [40].
Topological Data Analysis (TDA)
Topology studies properties of spaces that are in- variant under any continuous deformation. Over the past two decades, TDA has proven highly effective in identifying topological structures within images [41].
Table 1: The distribution of Break his data set.
|
Magnification |
Benign |
Malignant |
Total |
|
40× |
625 |
1370 |
1995 |
|
100× |
644 |
1437 |
2081 |
|
200× |
623 |
1390 |
2013 |
|
400× |
588 |
1232 |
1820 |
|
Total |
2480 |
5429 |
7909 |
In the context of histopathological image analysis, TDA has demonstrated strong potential in cancer detection [42]. TDA extracts meaningful patterns by analyzing the homological features of images. These features can quantify the complex topological shapes and geometric structures in the data. The advantage of TDA is that it can effectively process complex and high-dimensional data, capture the global topological structure of the data, and provide a deep understanding of the shape of the data [43]. This means that TDA offers a mathematically robust and interpretable way to capture topological features, especially in medical images where texture, shape, and structure matter.
We utilize Topological Data Analysis (TDA) to extract topological features from medical images, focusing on Persistent Homology (PH), one of TDA’s most prominent tools. PH quantifies how topological features-such as connected components, loops, and voids-evolve across scales through a filtration. A filtration is a sequence of nested spaces generated by progressively thresholding the data. Within this framework, a feature is born at the threshold where it first appears and dies when it merges with another feature or disappears. Long-lived features (with large persistence) are considered topologically meaningful, while short-lived ones are typically attributed to noise [44]. A comprehensive overview of TDA and PH is provided in [45].
Two main approaches are commonly used to compute homology: simplicial and cubical homology. While simplicial complexes offer greater generality, cubical complexes are more computationally efficient and better suited to image data, which naturally reside on regular pixel grids [46, 47]. In the cubical setting, images are modeled as 2D grids of square cells (pixels), and topological structures are constructed by thresholding pixel intensities. The resulting binary images are interpreted using 0D (points), 1D (edges), and 2D (squares) elements [48].
As the threshold increases, topological features appear and disappear, which can be quantified using Betti numbers: β0 (connected components), β1 (loops), and β2 (voids). These quantities provide compact and interpretable descriptors of image structure and texture. In medical imaging, they serve as shape-based features that are invariant to rotation and intensity changes and robust to spatial scaling, offering a complementary perspective to pixel-based and deep learning approaches. This interpretability is particularly valuable in clinical settings, where explain ability is essential.
Figure 2: THIR model. We first generate persistence diagrams for any input images, utilizing the cubical complex on each channel of the images. Next, we derive our topological feature vectors, represented as the Betti curves. The values of this curve are then input into the CBMIR workflow to produce the results of the search engine.
Figure 2 illustrates the cubical complex construction. The input image is overlaid with a grid, and each intensity value is used as a threshold (th). For 8-bit grayscale images, the filtration is defined as Krr ∈ T = K0…., K255. As th increases, connected components and loops appear, merge, or vanish. The persistence diagram summarizes this evolution: β0 features are shown in red and β1 in blue. The diagonal represents birth = death, and points farther from the diagonal correspond to more persistent and meaningful features [49].
In this study, we focus on β1, where Betti values represent the number of loops in the image. As shown in Figure 2, the Betti curve exhibits a bell-shaped pattern, peaking where the number of loops is maximal. At low thresholds, few features appear due to limited pixel activation. As the threshold increases, more loops emerge until a saturation point is reached. At high thresholds, the image be- comes nearly black, and topological features disappear. This behaviour is further illustrated in Figure 3, which demonstrates the effect of different thresholds on the R, G, and B channels. The variation of pixel intensities across channels explains the bell-shaped Betti curve. The dataset uses the default RGB color space, consistent with tissue staining. A comprehensive overview of color spaces in digital pathology is provided in [50].
Since the dataset consists of RGB images, the cubical complex pipeline is applied independently to each channel. As a result, each image produces three Betti curves, one per channel. These curves are concatenated and passed to the CBMIR pipeline for downstream analysis and similarity-based image retrieval.
Figure 3: Shows three channels of an RGB image under different th values. These values were defined as a sub- level filtration in [0-1] for the normalized images. Each channel goes through the same th values to illustrate the th impacts on channels.
Betti curves are computed on a non-uniform filtration scale. Given n loops represented as β1 = [(b1, d1), (b2, d2), . . . , (bn, dn)], we uniformly sample R = i filtration points between the mini- mum birth and maximum death values, denoted as X = [R1, R2, . . . , Ri]. A loop β1 (n) is considered active at Xi if bn ≤ Xi ≤ dn. This defines the Betti curve resolution (R), which controls the granularity of detected features. Higher resolutions preserve fine structures but may capture noise, while lower resolutions emphasize large-scale patterns at the expense of subtle details. Selecting an appropriate resolution is a modelling choice that balances sensitivity and robustness.
Experiments
In this paper, our method of choice is the Betti curves of the cubical complexes. The cubic complex from the Gudhi Library2 works with grayscale images. However, in digital pathology and working with WSIs, color plays a significant role that cannot be discarded from the experiments [40]. To address this, we apply cubical complex persistence separately to each RGB channel, treating them as independent grayscale images. The resulting Betti values from each channel are then concatenated to form a comprehensive topological descriptor for each image. With Betti curves sampled uniformly in Resolution, for each image we have 3 × R features. As mentioned above, finding the optimum value for the resolution is challenging and has a direct impact on the final results. As the resolution R increases, more fine-grained topological details are captured, though at the cost of longer computation times. In the case of Break His data set, R = 200 offers a strong balance between performance and efficiency, yielding the best accuracy with a much lower computational burden, followed by [45]. So, for the following experiments in this paper, we considered R = 200. Therefore, a descriptor representing each image is constructed from the features derived from β1 for the CBMIR framework. In the next step, the extracted features for the entire dataset are saved. Subsequently, we compute the features of the test set, which was previously separated from the training set. With 600 features extracted per image, we proceed to the comparison phase. Specifically, we use the Euclidean distance metric to identify the top-K most similar patches from the training set for each query image. After ranking the images based on their smallest distances, the top matching patches are retrieved and visualized along with their corresponding labels to assist pathologists in analysis.
Results And Discussion
When evaluating the performance of CBMIR methods, several important aspects must be considered, including training time, accuracy, and ease of training. In this section, we compare the performance of THIR with other methods across all these dimensions.
Accuracy Comparison
One challenge is the scarcity of comparable methods evaluated on the same dataset under identical conditions (i.e., same K value and magnification). To address this, we provide Table 2 and some more figures, such as Figure 4, to provide a comprehensive overview of the results in this study.
Summarizing recent studies on BreaKHis, indicating their magnification and the K value considered this enables clearer benchmarking of THIR across multiple settings. The evaluation is performed at multiple values of K, focusing on key performance metrics: accuracy, recall, precision, and F1-score.
In published studies on CBMIR, the value of K varies across works. Additionally, different magnifications of the BreaKHis dataset have been considered, and only a few studies have applied their methods to all magnifications (40×, 100×, 200×, and 400×). These variations make direct comparison challenging. To address this, we applied THIR across all magnifications of the data set using the most commonly used values of K. Following the evaluation protocol of [32,51,2], we conducted a fair comparison with several state-of-the-art methods.
At 400× magnification, THIR achieves an accuracy of 0.98 at the top-5, outperforming both super- vised and unsupervised methods. Notably, it delivers an accuracy of 0.98, which is 18% higher than Breast- twins (0.69), a fully supervised method utilizing a Siamese network for similarity learning. In [14], Fed- CBMIR was designed to generalize across all magnifications and was evaluated at 400×. Although FedCBMIR aims to enhance performance on unseen data, THIR consistently delivers better retrieval accuracy and precision, even without any learning phase or hyperparameter tuning. This trend continues at 200× magnification with K = 5, where THIR achieves a precision of 0.99, surpassing FedCBMIR by 10% (FedCBMIR precision = 0.89). Addition- ally, THIR demonstrates stronger performance than other baselines like CNN-based AE (0.93 precision) and MCCH (0.89 precision).
In Table 2, we include three methods from [53], for which the value of K used in top-K retrieval is not explicitly defined in the original paper. The only instance where a specific K value is mentioned appears in the caption of a qualitative figure, where they state K = 5. However, this information is not provided for the quantitative results reported in the tables. Therefore, due to the lack of clarity regarding the retrieval threshold, we marked the K value as”Not-defined” in our comparison table. The paper [53] focuses on hashing-based methods and reports retrieval performance at various code lengths: 16, 32, and 64 bits. To include these results in our comparison, we selected the highest performance across all bit lengths. For example, the DCMMH method achieves its best performance (0.95) at 32 bits for 40× magnification, while DPSH reaches its highest at 16 bits for the same magnification. To ensure a fair comparison with our proposed method, we report only the top-performing results from each approach, regardless of the bit length used.
Across all magnifications and for both K = 3, K = 5, and K = Not defined, THIR maintains high and stable performance. It consistently outperforms CNN-based AE, MCCH, and several hashing- based methods (e.g., HashNet, IDHN, DTQ), as well as state-of-the-art frameworks such as FedCBMIR and VTHC. These results highlight the robustness and effectiveness of TDA in CBMIR, especially com- pared to DL-based methods that require substantial training time and parameter optimization.

Figure 4: Shows four random queries and their similar patches. Each row represents a query image (leftmost) followed by its top-3 retrieved images. The true class labels are shown as Query Label and Retrieved Label. Misclassified retrievals are outlined in red.
Figure 4 shows four random examples of retrieval results with corresponding labels. Incorrect retrievals (where the retrieved label differs from the query label) are outlined in red. This visual analysis enables qualitative evaluation of system performance and highlights the capability of topological features to capture structural similarities in histopathological breast cancer images. Beyond label checking, pathologists can also examine structural patterns between queries and retrieved results based on their expertise.
Figure 5: shows four panels, each with four concatenated Betti curves for four random images. The label of each image is represented in a small bar on top of each panel. L = 0 and L = 1 mean”Benign and”Malignant” cases, respectively. This explains how images with the same cancer grade have similar Betti curves. The y-axis illustrates the number of loops at each filtration step, and the x-axis represents the filtration step for each channel. R = 200 yields 200 filter steps for each channel, which means 600 filter steps in total for an RGB image.
Figure 5 contains four panels, each showing the Betti curves of four randomly selected images. Different colored lines represent different images. A guide bar on top of each panel indicates the corresponding class label of each image related to its curve. The x-axis displays the filtration steps, while the y-axis shows the Betti values. The x-axis ranges from 0 to 600, illustrating that 600 features were extracted from the Betti values of each RGB image. As mentioned earlier, we computed the Betti values for each channel separately using the cubical complex and then concatenated them to obtain a representative feature vector for the entire image. Thus, the x-axis can be interpreted in intervals of 200: the interval [0-200] represents the Betti values for the red channel, [200-400] for green, and [400-600] for blue. This figure demonstrates how the trend of Betti curves behaves across channels for images from the same or different classes. For instance, in the top-left panel, all images belong to the same class (Benign), and their Betti curves follow a similar trend, indicating topological similarity. In contrast, in the top-right panel, the red line deviates noticeably, suggesting a different topological pattern compared to the other three images. The remaining three (blue, orange, and green lines) follow a similar trend, which reflects their similarity in class labels. In the bottom panels, two additional examples show that two images have similar Betti curves and share the same labels, while the other two follow distinct trends and have different labels. These characteristic patterns suggest that Betti curves encode discriminative information suitable for unsupervised CBMIR.
Furthermore, Figure 6 demonstrates the retrieval of four random queries based on Betti values. In each retrieval panel, four Betti curves are shown: the red line represents the query, while blue, orange, and green lines represent the top-3 retrieved images. The class labels are indicated above each image, and a guide bar links the curve colors to the images. In some cases, such as the top-left panel, the retrieved image shares similar topological features with the query but has a different label, suggesting intraobserver variability. In other cases, such as the top-right panel, all retrieved images share the same label with the query, reinforcing the robustness of Betti features. Such visualization suggests that im- ages with similar Betti curves may share underlying histopathological characteristics, even when labeled differently. This highlights the potential of CBMIR for digital pathology applications and demonstrates advantages over traditional Computer-Aided Diagnosis (CAD) tools.
As an indirect comparison between the THIR result and state-of-the-art classifiers on the same images, Table 3 provides comprehensive information regarding the accuracy of classifiers on the BreaKHis data set at 40X. TopOC-1 and TopOC-CNN in [45] obtained 89% and 93% accuracy, while THIR was successful in retrieving images with the same label with 98% accuracy. However, TopOC-1 and TopOC- CNN needed some training time and hyperparameter tuning.
How Fast, How Simple: Training, Searching, and Using the Model
Several recent CBMIR studies based on CAE have reported training times of approximately 9.33 hours and 6.59 hours on the BreaKHis dataset [14] at 40× magnification, using an NVIDIA Tesla T4 GPU. Al- though FedCBMIR reduces training time compared to traditional CBMIR, it still requires a substantial training duration. In DL-based methods, once the models are trained, features are subsequently extracted for CBMIR tasks; however, the feature extraction time is often not reported in these studies. In contrast, our method bypasses the lengthy train-ing phase entirely. THIR efficiently processes the entire dataset by performing the filtration, calculating the Betti values β1, and extracting 600 features per image. This comprehensive feature extraction for all images in the dataset requires merely about 20 minutes, which is significantly faster than the combined training and feature extraction time of existing deep learning approaches.
Table 2: Performance comparison of THIR with recent methods at various K values on BreaKHis across all magnifications. The best results per metric are bolded.
|
Magnification |
Method |
K |
Accuracy |
Recall |
Precision |
F1-score |
|
40× |
THIR |
3 |
0.95 |
0.97 |
0.96 |
0.97 |
|
THIR |
5 |
0.98 |
0.97 |
0.96 |
0.95 |
|
|
FedCBMIR [14] |
5 |
0.97 |
- |
0.96 |
0.98 |
|
|
CBMIR [14] |
0.95 |
- |
0.93 |
0.96 |
||
|
MCCH [54] |
- |
- |
0.94 |
- |
||
|
CNN-based AE [52] |
- |
0.77 |
0.95 |
- |
||
|
HSDH [55] |
400 |
- |
- |
0.99 |
- |
|
|
DTQ [55] |
- |
- |
0.91 |
- |
||
|
ATH [55] |
- |
- |
0.89 |
- |
||
|
IDHN [55] |
- |
- |
0.95 |
- |
||
|
HashNet [55] |
- |
- |
0.91 |
- |
||
|
VTHC [56] |
6 |
- |
- |
0.98 |
- |
|
|
DCMMH [53] |
Not defined |
- |
- |
0.96 |
- |
|
|
DPSH [53] |
- |
- |
0.95 |
- |
||
|
ADSH [53] |
- |
- |
0.95 |
- |
||
|
100× |
THIR |
3 |
0.97 |
0.98 |
0.98 |
0.98 |
|
THIR |
5 |
0.99 |
0.99 |
0.99 |
0.99 |
|
|
FedCBMIR [14] |
5 |
0.94 |
- |
0.92 |
0.96 |
|
|
CBMIR [14] |
0.9 |
- |
0.88 |
0.94 |
||
|
MCCH [54] |
- |
-0.49 |
0.92 |
- |
||
|
CNN-based AE [52] |
- |
|
0.77 |
- |
||
|
VTHC [56] |
6 |
- |
- |
0.99 |
- |
|
|
DCMMH [53] |
Not defined |
- |
- |
0.95 |
- |
|
|
DPSH [53] |
- |
- |
0.95 |
- |
||
|
ADSH [53] |
- |
- |
0.94 |
- |
||
|
200× |
THIR |
3 |
0.97 |
0.98 |
0.98 |
0.98 |
|
THIR |
5 |
0.99 |
0.99 |
0.99 |
0.99 |
|
|
FedCBMIR [14] |
5 |
0.92 |
- |
0.89 |
0.94 |
|
|
CBMIR [14] |
0.89 |
- |
0.87 |
0.93 |
||
|
MCCH [54] |
- |
-0.76 |
0.91 |
- |
||
|
CNN-based AE [52] |
- |
|
0.92 |
- |
||
|
VTHC [56] |
6 |
- |
- |
0.98 |
- |
|
|
DCMMH [53] |
Not defined |
- |
- |
0.97 |
- |
|
|
DPSH [53] |
- |
- |
0.96 |
- |
||
|
ADSH [53] |
- |
- |
0.95 |
- |
||
|
400× |
THIR |
3 |
0.95 |
0.97 |
0.95 |
0.97 |
|
THIR |
5 |
0.98 |
0.99 |
0.98 |
0.99 |
|
|
FedCBMIR [14] |
5 |
0.96 |
- |
0.94 |
0.97 |
|
|
UCBMIR [30] |
- |
0.79 |
0.91 |
- |
||
|
Breast-twins [54] |
0.69 |
0.82 |
0.91 |
0.81 |
||
|
MCCH [31] |
- |
- |
0.89 |
- |
||
|
CNN-based AE [52] |
- |
0.69 |
0.93 |
- |
||
|
VTHC [56] |
6 |
- |
- |
0.99 |
- |
|
|
DCMMH [53] |
Not defined |
- |
- |
0.96 |
- |
|
|
DPSH [53] |
- |
- |
0.95 |
- |
||
|
ADSH [53] |
- |
- |
0.95 |
- |
Figure 6: Demonstrates the retrieval of four random queries using Betti values. There are four panels of Betti curves with the corresponding queries and returned images. For each panel of Betti curves, a query image (top left) is compared to a set of retrieved images using the Euclidean distance function. Each curve in the panel reflects topological features over the filtration steps. The alignment of Betti curve patterns with the query supports the effectiveness of PH for topology-aware image retrieval. The label of each image is represented in a small bar on top of each panel. L = 0 and L = 1 mean”Benign and “Malignant” images, respectively.
Conclusion And Future Work
This paper introduces THIR, an unsupervised, training-free, and interpretable framework for Content-Based Medical Image Retrieval (CBMIR) based on topological data analysis. Using cubical persistence and Betti values for loops (β1), THIR extracts robust topological fingerprints from raw RGB histopathological images without annotations, GPU acceleration, or hyperparameter tuning.
Table 3: Classification accuracy of THIR compared to state-of-the-art methods on the BreaKHis dataset at 400× magnification, with K = 5 for THIR.
|
Method |
Accuracy |
|
THIR |
0.98 |
|
DenseNet201 [57] |
0.95 |
|
IDSNet [58] |
0.94 |
|
VGG16 [59] |
0.94 |
|
Resnet [60] |
0.94 |
|
DenseNet201 [61] |
0.89 |
|
BkCapsNet [30] |
0.88 |
|
CapsNet [62] |
0.88 |
|
BkNet [63] |
0.84 |
|
MobileNet [64] |
0.84 |
|
AlexNet [65] |
0.81 |
Experiments on the BreaKHis dataset show that THIR consistently outperforms supervised and un- supervised baselines across all magnifications. At 400×, it improves accuracy by 31% and precision by 18% over Breast-twins, a Siamese-network-based model. At 200×, it achieves a 10% gain in precision over FedCBMIR. Similar margins are observed at lower magnifications, with precision up to 0.99 at 100×, surpassing all compared methods. These results confirm THIR as a strong alternative for CB- MIR in label-scarce or resource-constrained settings.
THIR extracts 600 topological features per image in under 20 minutes on a standard CPU, offering an efficient and scalable solution compared to deep learning models requiring extensive training. Its performance, efficiency, and interpretability make it suitable for clinical applications and diagnostic support.
Future work will explore higher-dimensional features (e.g., persistence images, landscapes), extension to multi-channel and multi-organ datasets, alternative color spaces beyond RGB, and generalization to multi-class and whole-slide image retrieval tasks.
References
- Bray F, Laversanne M, Sung H, Ferlay J, Siegel RL, Soerjomataram I, et al. “Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries”. In: CA: a cancer journal for clinicians. 2024; 74: 229-263.
- Arnold M, Morgan E, Rumgay H, Mafra A, Singh D, Laversanne M, et al. “Current and future burden of breast cancer: Global statistics for 2020 and 2040”. In: The Breast. 2022; 66: 15-23.
- Kanwal N, Khoraminia F, Kiraz U, Zamudio AM, Monteagudo C, Janssen EA, et al. “Equipping computational pathology systems with artifact processing pipelines: a showcase for computation and performance trade-offs”. In: BMC Medical Informatics and Decision Making. 2024; 24: 288.
- Khan AM, Sirinukunwattana K, Rajpoot N. “A global covariance descriptor for nuclear atypia scoring in breast histopathology images”. In: IEEE J biomedical and health informatics. 2015; 19: 1637-1647.
- Singh H, Sethi S, Raber M, Petersen LA. “Errors in cancer diagnosis: current understanding and future directions”. In: J clinical oncology. 2007; 25: 5009-5018.
- Kostopoulou O, Delaney BC, Munro CW. “Diagnostic difficulty and error in primary care-a systematic review”. In: Family practice. 2008; 25: 400-413.
- Abbasniya MR, Sheikholeslamzadeh SA, Nasiri H, Emami S. “Classification of breast tumors based on histopathology images using deep features and ensemble of gradient boosting methods”. In: Computers and Electrical Engineering. 2022; 103: 108382.
- Chen C, Lu MY, Williamson DF, Chen TY, Schaumberg AJ, Mahmood F. “Fast and scalable search of whole-slide images via self-supervised deep learning”. In: Nature Biomedical Engineering. 2022; 6: 1420-1434.
- Veltkamp RC, Tanase M. Content-based image retrieval systems: A survey. Tech. rep. Technical Report UU-CS-2000-34, Dept. of Computing Science, Utrecht University. 2000.
- Komura D, Ishikawa S. “Machine learning methods for histopathological image analysis”. In: Computational and structural biotechnology journal. 2018; 16: 34-42.
- Kalra S, Tizhoosh HR, Shah S, Choi C, Damaskinos S, Safarpoor A, et al. “Pan-cancer diagnostic consensus through searching archival histopathology images using artificial intelligence”. In: NPJ digital medicine. 2020; 3: 31.
- Kumar A, Nette F, Klein K, Fulham M, Kim J. “A visual analytics approach using the exploration of multidimensional feature spaces for content-based medical image retrieval”. In: IEEE J biomedical and health informatics. 2014; 19: 1734-1746.
- Tizhoosh HR, Pantanowitz L. “On image search in histopathology”. In: J Pathology Informatics. 2024; 15: 100375.
- Tabatabaei Z, Wang Y, Colomer A, Moll JO, Zhao Z, Naranjo V. “Wwfedcbmir: World-wide federated content-based medical image retrieval”. In: Bioengineering. 2023; 10: 1144.
- Ma Y, Jiang Z, Zhang H, Xie F, Zheng Y, Shi H, et al. “Breast histopathological image retrieval based on latent dirichlet allocation”. In: IEEE J biomedical and health informatics. 2016; 21: 1114-1123.
- Bhaskar T, Ramadevi Y, Kavitha PN, Sravan P. “MCBIR: Deep Learning based Framework for Efficient Content Based Image Retrieval System of Medical Images”. 12; 1364-1373.
- Liu C, Ma X, Zhang H, Xie S, Yu D. “Dynamic Neuropsychological Approach for Multi-Quality Image Assessment Using Grey- Topological Data Analysis”. In: IEEE Access. 2024; 12: 139609 -139619.
- Tian M, Su M, Xiao X, Yi S, Hua Z, Zhang Y. “High-precision privacy-protected image retrieval based on multi-feature fusion”. In: Knowledge-Based Systems. 2025; 315: 113243.
- Sharif U, Mehmood Z, Mahmood T, Javid MA, Rehman A, Saba T. “Scene analysis and search using local features and support vector machine for effective content-based image retrieval”. In: Artificial Intelligence Review. 2019; 52: 901-925.
- Sarwar A, Mehmood Z, Saba T, Qazi KA, Adnan, Jamal H. “A novel method for content-based image retrieval to improve the effectiveness of the bag-of-words model using a support vector machine”. In: J Information Science. 2018; 45: 117-135.
- Mehmood Z, Abbas F, Mahmood T, Javid MA, Rehman A, Nawaz T. “Content- based image retrieval based on visual words fusion versus features fusion of local and global features”. In: Arabian J for Science and Engineering. 2018; 43: 7265-7284.
- Won CS, Park DK, Park SJ. “Efficient use of MPEG-7 edge histogram descriptor”. In: ETRI journal. 2002; 24: 23-30.
- Manjunath BS, Ohm JR, Vasude-van VV, Yamada A. “Color and texture descriptors”. In: IEEE Transactions on circuits and systems for video technology. 2001; 11: 703-715.
- Owais M, Arsalan M, Choi J, Park KR. “Effective diagnosis and treatment through content-based medical image retrieval (CB- MIR) by using artificial intelligence”. In: Journal of clinical medicine. 2019; 8: 462.
- Widmer A, Schaer R, Markonis D, Muller H. “Gesture interaction for content-based medical image retrieval”. In: Proceedings of international conference on multimedia retrieval. 2014; 503-506.
- Karthik K, Kamath SS. “A deep neural network model for content-based medical image retrieval with multi-view classification”. In: The Visual Computer. 2021; 37: 1837-1850.
- Tuyet VTH, Binh NT, Quoc NK, Khare A. “Content based medical image retrieval based on salient regions combined with deep learning”. In: Mobile Networks and Applications. 2021; 26: 1300-1310.
- Agrawal S, Chowdhary A, Agarwala S, Mayya V, Kamath SS. “Content-based medical image retrieval system for lung dis- eases using deep CNNs”. In: Int J Information Technology. 2022; 14: 3619-3627.
- Dubey SR. “A decade survey of content based image retrieval using deep learning”. In: IEEE Transactions on Circuits and Systems for Video Technology. 2021; 32: 2687-2704.
- Tabatabaei Z, Colomer A, Moll JO, Naranjo V. “Toward More Transparent and Accurate Cancer Diagnosis with an Unsupervised CAE Approach”. In: IEEE Access. 2023; 11: 143387-143401.
- Tabatabaei Z, Colomer A, Moll JO, Naranjo V. “Siamese Content-based Search Engine for a More Transparent Skin and Breast Cancer Diagnosis through Histological Imaging”. In: arXiv preprint arXiv: 2024; 2401: 08272.
- Hegde N, Hipp JD, Liu Y, Buck ME, Reif E, Smilkov D, et al. “Similar image search for histopathology: SMILY”. In: NPJ digital medicine. 2019; 2: 56.
- Kalra S, Tizhoosh HR, Choi C, Shah S, Diamandis P, Campbell CJ, et al. “Yottixelan image search engine for large archives of histopathology whole slide images”. In: Medical Image Analysis. 2020; 65: 101757.
- Wang X, Du Y, Yang S, Zhang J, Wang M, Zhang J, et al. “RetCCL: Clustering-guided contrastive learning for whole-slide image retrieval”. In: Medical image analysis. 2023; 83: 102645.
- Ozen Y, Aksoy S, Kosemehmetoglu K, Onder S, Uner A. “Self-supervised learning with graph neural networks for region of interest retrieval in histopathology”. In: 2020 25th International conference on pattern recognition (ICPR). IEEE. 2021; 6329-6334.
- Zheng Y, Jiang Z, Zhang H, Xie F, Ma Y, Shi H, et al. “Size-scalable content- based histopathological image retrieval from database that consists of WSIs”. In: IEEE J biomedical and health informatics. 2017; 22: 1278-1287.
- Skaf Y, Laubenbacher R. “Topological data analysis in biomedicine: A review”. In: J Biomedical Informatics. 2022; 130: 104082.
- Abdulaal AH, Valizadeh M, Yassin RA, Amirani MC, Shah AS, Albaker BM, et al. “Hybrid CNN and RNN Model for Histopathological Sub-Image Classification in Breast Cancer Analysis Using Self- Learning”. In: J Engineering and Sustainable Development. 2025; 29: 310-320.
- Kanwal N, Bueno FP, Schmidt A, Engan K, Molina R. “The devil is in the details: Whole slide image acquisition and processing for artifacts detection, color variation, and data augmentation: A review”. In: Ieee Access. 2022; 10: 58821-58844.
- Al Noumah W, Jafar A, Al Joumaa K. “Using parallel pre-trained types of DCNN model to predict breast cancer with color normalization”. In: BMC research notes. 2022; 15: 1-6.
- Wang F, Kapse S, Liu S, Prasanna P, Chen C. “TopoTxR: a topological biomarker for predicting treatment response in breast cancer”. In: International Conference on Information Processing in Medical Imaging. Springer. 2021; 386-397.
- Crawford L, Monod A, Chen AX, Mukherjee S, Rabadan R. “Predicting clinical outcomes in glioblastoma: an application of topological and functional data analysis”. In: J the American Statistical Association. 2020; 115: 1139-1150.
- Cui J. “Extended persistence and duality in cubical complexes”. In. 2024.
- Strombom D. Persistent homology in the cubical setting: theory, implementations and applications. 2007.
- Fatema S, Nuwagira B, Chakraborty S, Gedik R, Coskunuzer B. “TopOC: Topo- logical Deep Learning for Ovarian and Breast Cancer Diagnosis”. In: International Work-shop on Topology-and Graph-Informed Imaging Informatics. Springer. 2024; 22-32.
- Otter N, Porter MA, Tillmann U, Grindrod P, Harrington HA. “A roadmap for the computation of persistent homology”. In: EPJ Data Science. 2017; 6: 1-38.
- Rabadan R, Blumberg AJ. Topological data analysis for genomics and evolution: topology in biology. Cambridge University Press. 2019.
- Dugast M, Bouleux G, Mory O, Marcon E. “Improving health care management through persistent homology of time-varying variability of emergency department patient flow”. In: IEEE J biomedical and health informatics. 2018; 23: 2174-2181.
- Lawson P, Schupbach J, Fasy BT, Sheppard JW. “Persistent homology for the automatic classification of prostate cancer aggres- siveness in histopathology images”. In: Medical Imaging 2019: Digital Pathology. SPIE. 2019; 10956: 72-85.
- Velastegui R, Pedersen M. “The impact of using different color spaces in histological image classification using convolutional neural networks”. In: 2021 9th European Workshop on Visual Information Processing (EUVIP). IEEE. 2021; 1-6.
- Murala S, Wu QJ. “Local mesh patterns versus local binary patterns: biomedical image indexing and retrieval”. In: IEEE J biomedical and health informatics. 2013; 18: 929-938.
- Minarno AE, Ghufron KM, Sabrila TS, Husniah L, Sumadi FDS. “Cnn based autoencoder application in breast cancer image retrieval”. In: 2021 International Seminar on Intelligent Technology and Its Applications (ISITIA). IEEE. 2021; 29-34.
- Gu Y, Yang J. “Densely-connected multi- magnification hashing for histopathological image retrieval”. In: IEEE J biomedical and health informatics. 2018; 23: 1683-1691.
- Gu Y, Yang J. “Multi-level magnification correlation hashing for scalable histopathological image retrieval”. In: Neurocomputing. 2019; 351: 134-145.
- Alizadeh SM, Helfroush MS, Muller H. “A novel Siamese deep hashing model for histopathology image retrieval”. In: Ex- pert Systems with Applications. 2023; 225: 120169.
- Kumar M, Singh R, Mukherjee P. “VTHSC-MIR: Vision Transformer Hashing with Supervised Contrastive learning based medical image retrieval”. In: Pattern Recognition Letters. 2024; 184: 28-36.
- Taheri F, Rahbar K. “Enhancing breast cancer diagnosis: transfer learning on DenseNet with neural hashing for histopathology fine-grained image classification”. In: Medical and Biological Engineering and Computing. 2025; 63: 2717-2731.
- Li X, Shen X, Zhou Y, Wang X, Li TQ. “Classification of breast cancer histopathological images using interleaved DenseNet with SENet (IDSNet)”. In: PloS one. 2020; 15: e0232127.
- Liang Y, Meng Z. “Brea-net: An in- terpretable dual-attention network for imbalanced breast cancer classification”. In: IEEE Access. 2023; 11: 100508-100517.
- Ashraf FB, Alam SM, Sakib SM. “Enhancing breast cancer classification via histopathological image analysis: Leveraging self-supervised contrastive learning and transfer learning”. In: Heliyon. 2024; 10: e24094.
- Maleki, M. Raahemi, and H. Nasiri. “Breast cancer diagnosis from histopathology images using deep neural network and XGBoost”. In: Biomedical Signal Processing and Control. 2023; 86: 105152.
- Sabour, N. Frosst, and G. E. Hinton. “Dynamic routing between capsules”. In: Advances in neural information processing systems 30 (2017).
- Wang P, Wang J, Li Y, Li P, Li L, Jiang M. “Automatic classification of breast cancer histopathological images based on deep feature fusion and enhanced routing”. In: Biomedical Signal Processing and Control. 2021; 65: 102341.
- Simonyan EO, Badejo JA, Weijin JS. “Histopathological breast cancer classification using CNN”. In: Materials Today: Proceedings. 2024; 105: 268-275.
- Spanhol FA, Oliveira LS, Petitjean C, Heutte L. “Breast cancer histopathological image classification using convolutional neural networks”. In: 2016 international joint conference on neural networks (IJCNN). IEEE. 2016; 2560-2567.