Insights | Chenglu Zhu

Research on Robustness and Generalization Learning Theory in Pathological Image Analysis

Thu, 29 Jan 2026 00:00:00 +0000

Clinical translation of pathological image analysis faces three fundamental theoretical hurdles: mixed data biases (co-occurring noise and long-tail distributions), cross-center domain shifts, and the need for unsupervised annotation.

This project addresses these issues through gradient optimization dynamics, test-time adaptation mechanisms, and self-supervised representation mining. Below, we detail our latest findings in gradient-aware decoupling (DAR), stable test-time training (Stable TTT), unsupervised cell recognition (PSM), and robustness benchmarking for pathological foundation models.

1. Gradient-Aware Decoupling for Mixed Data Bias (DAR)

Original Paper: Gradient-aware learning for joint biases: Label noise and class imbalance (Neural Networks 2024) Authors: Shichuan Zhang, Chenglu Zhu, Honglin Li, Jiatong Cai, Lin Yang

The Challenge

Real-world pathological data rarely comes clean; it often suffers from simultaneous label noise and class imbalance (long-tail distribution). This “mixed bias” creates a conflict for traditional handling strategies. For instance, reweighting methods designed for class imbalance can inadvertently amplify the influence of noisy samples, causing the model to overfit incorrect labels and degrading performance.

Our Approach

We observed that the feature extractor (Encoder) and the Classifier react differently to these biases: the Encoder is more sensitive to noise, while the Classifier is more sensitive to imbalance. Based on this, we developed the Gradient-aware Decoupling and Regulation (DAR) framework:

Probe Data Guidance: Leveraging the “early-learning” phenomenon, we use loss distribution during the initial training phase to automatically identify a small set of high-confidence, class-balanced samples (Probe Data).
Gradient Rectification: We train an auxiliary network on this probe data to generate reference gradients. By calculating the cosine similarity between the main network’s gradients and these reference gradients, we generate a direction matrix $\Omega$.
Decoupled Updates: Using $\Omega$, we intervene in the parameter updates for the Encoder and Classifier separately.

Results

In a controlled CIFAR-10 environment with extreme mixed bias (40% label noise + 0.02 imbalance factor), DAR demonstrated strong resilience:

Accuracy: Improved from a baseline (ERM) of 52.34% to 74.48%, significantly outperforming existing methods like L2R and HAR.
Real-world Validation: On the Clothing1M dataset, DAR achieved an accuracy of 76.37%.

Method	C10 (N=0.2, I=0.1)	C10 (N=0.2, I=0.05)	C10 (N=0.2, I=0.02)	C10 (N=0.4, I=0.1)	C10 (N=0.4, I=0.05)	C10 (N=0.4, I=0.02)	C100 (N=0.2, I=0.1)	C100 (N=0.2, I=0.05)	C100 (N=0.2, I=0.02)
ERM	72.18 ± 0.27	67.68 ± 0.61	61.81 ± 0.63	62.21 ± 1.73	59.21 ± 2.32	52.34 ± 1.04	31.16 ± 0.29	27.94 ± 1.14	25.58 ± 0.72
FL	69.40 ± 0.56	66.17 ± 0.60	56.37 ± 2.53	61.55 ± 0.40	55.89 ± 2.17	48.64 ± 1.66	31.17 ± 0.91	27.40 ± 1.15	24.36 ± 0.82
GCE	73.76 ± 0.26	59.06 ± 0.78	55.19 ± 0.23	69.22 ± 0.33	59.79 ± 0.80	52.10 ± 0.37	26.97 ± 1.08	22.68 ± 0.26	17.57 ± 0.39
L2R(w)	82.95 ± 0.19	78.57 ± 0.27	68.54 ± 0.96	78.10 ± 0.21	70.43 ± 0.64	57.25 ± 0.91	37.01 ± 0.29	33.33 ± 0.17	13.60 ± 0.37
WN-Net(w)	76.83 ± 0.22	72.57 ± 0.50	65.00 ± 0.60	70.42 ± 0.18	61.68 ± 1.22	53.23 ± 0.91	38.81 ± 0.58	31.73 ± 2.30	26.88 ± 0.40
HAR	82.36 ± 0.64	78.63 ± 0.86	70.76 ± 1.49	76.80 ± 1.32	67.70 ± 2.03	54.55 ± 2.39	43.24 ± 1.59	36.38 ± 0.5	28.68 ± 0.71
AutoDO	78.36 ± 0.24	73.42 ± 0.64	65.44 ± 0.50	71.25 ± 0.42	66.14 ± 1.46	53.31 ± 2.02	39.43 ± 1.63	32.33 ± 0.58	23.01 ± 0.57
DAR(w/o)	82.34 ± 0.30	78.73 ± 0.64	72.26 ± 0.63	76.84 ± 0.26	72.31 ± 0.46	62.18 ± 0.70	46.01 ± 0.54	39.49 ± 0.64	31.42 ± 0.97
DAR(w)	82.79 ± 0.13	79.50 ± 0.31	74.83 ± 0.79	77.39 ± 0.33	74.48 ± 0.94	67.63 ± 0.46	45.03 ± 0.53	38.02 ± 0.39	31.17 ± 0.77

Table 1: The accuracy results on CIFAR10 and CIFAR100. The training sets are with various noise ratios (N) and imbalance factors (I).

2. Stable Test-Time Training for Cross-Center Generalization (Stable TTT)

Original Paper: Stable Test-Time Training for Semantic Segmentation with Output Contrastive Loss (ICASSP 2025) Authors: Yunlong Zhang, Zhongyi Shui, Honglin Li, Yuxuan Sun, Chenglu Zhu, Lin Yang

The Challenge

Pathology models often fail when deployed across different centers due to variations in stain styles and scanners (domain drift). Existing Test-Time Training (TTT) methods for segmentation are computationally heavy and prone to “model collapse”—where the model starts predicting the background class for everything.

Our Approach

We propose a lightweight strategy using Output Contrastive Loss (OCL):

Output Space Contrast: Instead of expensive feature space comparisons, OCL forces the model to pull prediction distributions of similar pixels closer while pushing dissimilar ones apart directly in the output space.
High Temperature & Stochastic Restoration: We use a high temperature coefficient to prevent overly sharp predictions and a “Stochastic Restoration” mechanism to randomly reset parameters, preventing catastrophic forgetting.

Figure 1: A visual comparison showing how OCL acts directly and efficiently on the output layer to separate classes.

Results

On the GTA5 $\to$ Cityscapes semantic segmentation task:

Performance: mIoU increased by 7.5% over the baseline (from 37.5% to 45.0%).
Efficiency: Reduced VRAM usage by 2.1GB and cut adaptation time by 23 seconds, making deployment on clinical edge devices feasible.

Method	Setting	Road	Side.	Build.	Wall	Fence	Pole	Light	Sign	Veget.	Terr.	Sky	Pers.	Rider	Car	Truck	Bus	Train	Motor.	Bike	mIoU
GTA→CS (Val.)
AdaptSegNet	DA	86.5	36.0	79.9	23.4	23.3	23.9	35.2	14.8	83.4	33.3	75.6	58.5	27.6	73.7	32.5	35.4	3.9	30.1	28.1	42.4
CLAN	DA	87.0	27.1	79.6	27.3	23.3	28.3	35.5	24.2	83.6	27.4	74.2	58.6	28.0	76.2	33.1	36.7	6.7	31.9	31.4	43.2
AdvEnt	DA	89.4	33.1	81.0	26.6	26.8	27.2	33.5	24.7	83.9	36.7	78.8	58.7	30.5	84.8	38.5	44.5	1.7	31.6	32.4	45.5
CBST	DA	91.8	53.5	80.5	32.7	21.0	34.0	28.9	20.4	83.9	34.2	80.9	53.1	24.0	82.7	30.3	35.9	16.0	25.9	42.8	45.9
DACS	DA	89.9	39.7	87.9	30.7	39.5	38.5	46.4	52.8	88.0	44.0	88.8	67.2	35.8	84.5	45.7	50.2	0.0	27.3	34.0	52.1
Source only	-	71.9	15.6	74.4	22.4	14.8	22.9	35.4	18.4	81.1	22.0	68.3	57.3	27.9	68.1	33.1	5.8	6.5	30.5	35.3	37.5
+TENT	TTT	71.5	22.6	76.9	20.0	17.1	21.6	29.2	15.3	78.4	33.9	75.3	50.8	3.5	80.9	29.5	31.7	4.3	13.7	2.1	35.7
+MEMO	TTT	82.6	0.1	68.0	0.0	0.2	1.3	1.7	0.2	78.3	0.3	82.3	1.3	0.3	77.9	6.2	1.8	0.0	0.8	0.1	21.3
+CoTTA	TTT	74.4	13.5	75.3	24.1	14.0	22.9	31.1	16.1	81.8	22.6	69.7	57.3	26.7	71.9	33.4	6.2	8.1	27.2	31.7	37.3
+OCL	TTT	87.1	42.1	81.6	29.7	20.2	27.5	37.8	18.3	83.8	33.8	74.7	60.5	24.8	85.3	36.3	46.7	4.4	29.6	31.7	45.0
FDA	DA	92.1	52.3	80.7	23.6	26.4	35.5	37.7	38.6	81.2	32.4	73.2	61.2	34.0	84.0	32.2	51.2	8.0	26.8	44.1	48.2
+OCL	+TTT	93.2	57.0	83.5	31.5	31.5	38.6	41.3	39.4	85.0	42.6	76.8	63.1	34.2	85.5	34.2	51.5	9.0	26.6	46.1	51.1
DAFormer	DA	96.5	74.0	89.5	53.4	47.7	50.6	54.7	63.6	90.0	44.4	92.6	71.8	44.8	92.6	77.8	80.6	63.6	56.7	63.4	68.8
+OCL	+TTT	96.6	74.7	89.6	53.5	48.1	51.3	55.3	64.0	90.0	44.5	92.5	72.3	45.4	92.8	78.6	81.4	66.8	59.0	64.0	69.5
Synthia→CS (Val.)
AdvEnt	DA	85.6	42.2	79.7	8.7	0.4	25.9	5.4	8.1	80.4	-	84.1	57.9	23.8	73.3	-	36.4	-	14.2	33.0	41.2
CBST	DA	68.0	29.9	76.3	10.8	1.4	33.9	22.8	29.5	77.6	-	78.3	60.6	28.3	81.6	-	23.5	-	18.8	39.8	42.6
MRKLD	DA	67.7	32.2	73.9	10.7	1.6	37.4	22.2	31.2	80.8	-	80.5	60.8	29.1	82.8	-	25.0	-	19.4	45.3	43.8
DACS	DA	80.6	25.1	81.9	21.5	2.9	37.2	33.7	24.0	83.7	-	90.8	67.6	38.3	82.9	-	38.9	-	28.5	47.6	48.3
Source only	-	45.2	19.6	72.0	6.7	0.1	25.4	5.5	7.8	75.3	-	81.9	57.3	17.3	39.0	-	19.5	-	7.0	25.7	31.5
+TENT	TTT	38.1	18.9	57.5	1.1	0.2	24.7	7.1	9.0	74.5	-	81.4	47.0	17.0	67.7	-	8.6	-	5.9	29.7	30.5
+MEMO	TTT	63.9	0.7	65.4	0.0	0.0	2.1	0.3	0.3	66.4	-	78.1	6.7	0.5	15.5	-	0.8	-	0.5	0.1	19.0
+CoTTA	TTT	48.5	20.8	73.1	8.4	0.2	24.3	12.6	11.0	76.0	-	82.2	56.6	17.3	40.2	-	21.1	-	9.2	27.7	33.0
+OCL	TTT	66.6	27.5	78.8	8.0	0.2	29.0	8.1	11.3	80.1	-	82.4	55.9	16.5	58.9	-	28.3	-	11.8	28.4	36.9
FDA	DA	76.2	33.3	74.8	8.3	0.3	32.2	19.8	24.5	62.6	-	83.8	58.2	27.3	82.2	-	40.3	-	31.5	45.1	43.8
+OCL	+TTT	78.0	33.8	78.9	10.9	0.3	34.1	21.9	26.1	75.7	-	84.8	60.8	28.6	84.3	-	43.1	-	32.5	45.3	46.2
DAFormer	DA	82.2	37.2	88.6	42.9	8.5	50.1	55.1	54.3	85.7	-	88.0	73.6	48.6	87.6	-	62.8	-	53.1	62.4	61.3
+OCL	+TTT	81.6	36.5	88.7	43.1	8.4	50.8	55.8	55.1	86.2	-	88.4	74.2	49.5	87.8	-	63.2	-	54.5	62.8	61.7

3. Unsupervised Cell Recognition with Prior Self-Activation Maps (PSM)

Original Paper: Exploring Unsupervised Cell Recognition with Prior Self-activation Maps (MICCAI 2023) Authors: Pingyi Chen, Chenglu Zhu, Zhongyi Shui, Jiatong Cai, Sunyi Zheng, Shichuan Zhang, Lin Yang

The Challenge

Pathology images contain dense cell structures, making pixel-level manual annotation prohibitively expensive. We investigated a core question: Can deep networks spontaneously locate and segment cells without any human labels?

Our Approach

We found that shallow layers of self-supervised pre-trained networks inherently contain rich morphological cues. We developed the Prior Self-activation Maps (PSM) framework:

Gradient Aggregation: Extracts and aggregates gradient information from shallow layers to generate an initial Class Activation Map (CAM).
Semantic Clustering Module (SCM): Uses K-Means clustering to refine these activation maps into high-quality “Pseudo Masks” for training downstream networks.

Figure 1: The PSM pipeline: Gradient extraction -> Activation map generation -> Semantic clustering -> Training downstream networks.

Results

Achieved performance close to supervised methods without using a single manual annotation:

Methods	Loc	Cnt	Pixel-level IoU	Pixel-level F1	Object-level Dice	Object-level AJI
Unet* [20]	✔	✔	0.606	0.745	0.715	0.511
MedT [24]	✔	✔	0.662	0.795	-	-
CDNet [8]	✔	✔	-	-	0.832	0.633
Competition Winner [13]	✔	✔	-	-	-	0.691
Qu et al. [19]	✔	✘	0.579	0.732	0.702	0.496
Tian et al. [22]	✔	✘	0.624	0.764	0.713	0.493
CellProfiler [1]	✘	✘	-	0.404	0.597	0.123
Fiji [21]	✘	✘	-	0.665	0.649	0.273
CyCADA [9]	✘	✘	-	0.705	-	0.472
Hou et al. [10]	✘	✘	-	0.750	-	0.498
Ours	✘	✘	0.610	0.762	0.724	0.542

Table 3: Results on MoNuSeg. Loc: Localization, Cnt: Contour. * indicates the model is trained from scratch with the same hyperparameter as ours.

Detection: 0.811 F1-score on the BCData dataset.
Segmentation: AJI score of 0.542 on the MoNuSeg dataset.

4. Robustness Benchmarking for Pathological Foundation Models

Original Paper: Benchmarking PathCLIP for Pathology Image Analysis (JIIM 2025) Authors: Sunyi Zheng, Xiaonan Cui, Yuxuan Sun, Jingxiong Li, Honglin Li, Yunlong Zhang, Pingyi Chen, Xueping Jing, Zhaoxiang Ye, Lin Yang

The Challenge

With the rise of foundation models like PathCLIP, their resilience to real-world interference (color casts, blur, marker pen strokes) remains largely untested.

Our Work & Key Findings

We established a standardized benchmark covering 11 types of clinical image corruptions. Stress-testing multiple models (OpenAI-CLIP, PLIP, PathCLIP) revealed critical vulnerabilities.

This finding emphasizes the necessity of rigorous pre-processing steps, such as stain normalization and digital marker removal, before clinical deployment.

Summary

This phase of our research tackles the “imperfect data” and “inconsistent environment” problems in Pathological AI at a theoretical level.

DAR resolves training conflicts in mixed-bias scenarios.
Stable TTT enables low-cost cross-center adaptation.
PSM proves the potential of unsupervised learning for cell analysis.
Robustness Benchmark defines the safety boundaries for foundation models.

Together, these works form the theoretical cornerstone for the next generation of robust pathological AI systems.

Reconstructing Computational Paradigms for Pathological Image Analysis

Tue, 21 May 2024 00:00:00 +0000

The gigapixel scale of Whole Slide Images (WSI), the chronic absence of clinical multimodal data, and the “compute wall” for fine-tuning large models constitute the “Three Major Hurdles” restricting the development of high-precision pathological AI.

This project reconstructs the computational paradigm of pathological image analysis across three dimensions: low-rank architectural breakthroughs, robust fusion mechanisms for missing modalities, and task-specific efficient fine-tuning.

1. Breaking the “Low-Rank” Bottleneck in Long Sequences (LongMIL)

Original Paper: Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis (NeurIPS 2024) Authors: Honglin Li, Yunlong Zhang, Pingyi Chen, Zhongyi Shui, Chenglu Zhu, Lin Yang

The Scientific Question: The Transformer’s “Achilles’ Heel” in WSI

When processing WSIs containing tens of thousands of patches, traditional Transformers face two critical challenges:

Explosive Complexity: The $O(N^2)$ complexity of standard Self-Attention makes memory usage unsustainable.
Low-Rank Bottleneck: We theoretically revealed that when sequence length $N$ far exceeds embedding dimension $D$, the attention matrix exhibits mathematical “low-rank” properties. This means attention maps become homogenized, failing to capture fine-grained local microenvironmental differences.

Core Method: Local-Global Hybrid Attention

To break the rank limit and reduce computation, we propose the LongMIL architecture:

Local Attention Mask: By introducing local window constraints, we force the model to focus on interactions within local neighborhoods. Theory proves this sparsification significantly increases the Rank of the attention matrix.
Linear Complexity: Utilizing a Chunked Computation strategy reduces complexity from quadratic $O(N^2)$ to linear $O(N \times w)$ (where $w$ is window size).
Dual-Stream Architecture: A “Local-First, Global-Second” design captures cell community features before aggregating slide-level information.

Figure 3: The LongMIL framework: Stage-1 prepares features; Stage-2 uses Local Masks for accelerated attention; Overall-stage models the hierarchy from local to global.

Results

Table1

Method,ViT-S Lunit [36] F1,ViT-S Lunit [36] AUC,ViT-S DINO (our pre-train) F1,ViT-S DINO (our pre-train) AUC KNN (Mean),0.503 ± 0.011,0.691 ± 0.007,0.430 ± 0.029,0.649 ± 0.008 KNN (Max),0.472 ± 0.009,0.771 ± 0.018,0.416 ± 0.019,0.645 ± 0.007 Mean-pooling,0.534 ± 0.026,0.741 ± 0.017,0.487 ± 0.034,0.717 ± 0.020 Max-pooling,0.649 ± 0.032,0.843 ± 0.018,0.598 ± 0.032,0.818 ± 0.006 AB-MIL [32],0.668 ± 0.032,0.866 ± 0.016,0.621 ± 0.048,0.837 ± 0.035 DS-MIL [40],0.607 ± 0.044,0.824 ± 0.028,0.622 ± 0.063,0.808 ± 0.033 CLAM-SB [50],0.647 ± 0.020,0.836 ± 0.021,0.627 ± 0.032,0.836 ± 0.009 DTFD-MIL MaxS [89],0.597 ± 0.025,0.874 ± 0.026,0.521 ± 0.059,0.807 ± 0.016 DTFD-MIL AFS [89],0.608 ± 0.083,0.869 ± 0.018,0.538 ± 0.053,0.824 ± 0.011 TransMIL [65],0.648 ± 0.054,0.835 ± 0.031,0.591 ± 0.049,0.798 ± 0.029 Full Attention,0.689 ± 0.036,0.870 ± 0.010,0.648 ± 0.028,0.839 ± 0.018 LongMIL (ours),0.706 ± 0.025,0.888 ± 0.019,0.657 ± 0.026,0.848 ± 0.004

Table 1: Slide-Level Survival Prediction based on HIPT [9] pre-trained embedding with variousWSI-MIL architectures including vanilla attention, GCN, TransllL, self-attention (HIPT with regionslicing and absolute embedding), full self-attention and our LongMl.

Table 2

Method,COADREAD,STAD,BRCA AB-MIL [32],0.566 ± 0.075,0.562 ± 0.049,0.549 ± 0.057 AMISL [86],0.561 ± 0.088,0.563 ± 0.067,0.545 ± 0.071 DS-MIL [40],0.470 ± 0.053,0.546 ± 0.047,0.548 ± 0.058 GCN-MIL [43],0.538 ± 0.049,0.513 ± 0.069,- HIPT [9],0.608 ± 0.088,0.570 ± 0.081,- TransMIL [65],0.597 ± 0.134,0.564 ± 0.080,0.587 ± 0.063 Full Attention,0.603 ± 0.048,0.568 ± 0.074,0.601 ± 0.047 LongMIL (ours),0.624 ± 0.057,0.589 ± 0.066,0.619 ± 0.053

Table 2: Slide-Level Tumor Subtyping on BRACS by using two pre-trained embeddings. Top Rows.Various WSI-MI architectures with vanilla attention (no interaction among different instances)Bottom Rows. TransMlL, (using Nyströmformer and learnable absolute position embedding), fullattention (+RoPE) and our LongMIL.

On BRACS and TCGA-BRCA datasets:

Performance: F1-score reached 0.657 on BRACS tumor typing, significantly outperforming SOTA methods like TransMIL.
Extrapolation: In “train small, test large” experiments, LongMIL showed strong robustness (p-value $\approx$ 0.1), proving its adaptability to varying WSI sizes.

2. Feature Mining under Weak Supervision: Attention-Challenging MIL (ACMIL)

Original Paper: Attention-Challenging Multiple Instance Learning for Whole Slide Image Classification (ECCV 2024) Authors: Yunlong Zhang, Honglin Li, Yunxuan Sun, Sunyi Zheng, Chenglu Zhu, Lin Yang

The Scientific Question: Attention “Laziness”

In Weakly Supervised Multiple Instance Learning (MIL), models tend to focus only on the most obvious discriminative regions (e.g., tumor cores), ignoring edges or atypical key features. This “Attention Laziness” leads to poor generalization on heterogeneous tumors.

Figure 3: Motivation of MBA (left) & Figure 4: Motivation of STKIM.

Core Method: Adversarial Attention Enhancement

We propose the ACMIL framework to “manufacture difficulty” for the model:

Multi-Branch Attention (MBA): Parallel attention branches capture distinct clustering patterns in the feature space (verified via UMAP), covering more diverse pathological features.
Stochastic Top-K Instance Masking (STKIM): During training, we randomly “mask” the Top-K instances with the highest attention scores.

Figure 6: Heatmap comparison showing ACMIL (right) covering broader tumor regions than the baseline (left).

Results

Camelyon16: Achieved an AUC of 0.954, outperforming methods like DTFD-MIL.
TCGA-LBC: AUC increased to 0.901 on liquid-based cytology data, proving effectiveness in sparse feature mining.

3. Addressing Missing Clinical Data: Bidirectional Distillation

Original Paper: Multi-modal Learning with Missing Modality in Predicting Axillary Lymph Node Metastasis (BIBM 2023) Authors: Shichuan Zhang, Sunyi Zheng, Zhongyi Shui, Honglin Li, Lin Yang

The Scientific Question: The Multimodal “Bucket Effect”

In clinical practice, WSI and tabular data (genomics, clinical markers) are often asynchronous. Existing multimodal models often suffer a severe performance drop—sometimes below single-modal baselines—when clinical data is missing.

Core Method: Bidirectional Distillation & Learnable Prompts

We propose a Bidirectional Distillation (BD) framework to teach the model how to handle missingness:

Decoupling: Parallel “Single-Modal Branch” (WSI only) and “Multi-Modal Branch” (WSI + Clinical).
Learnable Prompt: A learnable vector acts as a placeholder for missing modalities in the single-modal branch.
Bidirectional Distillation: We distill fused knowledge from Multi $\to$ Single ($\mathcal{M} \to \mathcal{S}$) and distill pure image features back from Single $\to$ Multi ($\mathcal{S} \to \mathcal{M}$) to prevent noise interference.

Figure 2: The BD structure showing parallel branches, the Learnable Prompt, and the bidirectional distillation loss paths.

Results

In BCNB Breast Cancer Lymph Node Metastasis prediction:

Resilience: With 80%-100% clinical data missing, BD maintained an F1-score of ~74.9%, while direct filling methods crashed to below 68%.

4. Low-Cost WSI Adaptation: Variational Information Bottleneck Fine-tuning

Original Paper: Task-specific Fine-tuning via Variational Information Bottleneck for Weakly-supervised Pathology Whole Slide Image Classification (CVPR 2023) Authors: Honglin Li, Chenglu Zhu, Yunlong Zhang, Yuxuan Sun, Zhongyi Shui, Wenwei Kuang, Sunyi Zheng, Lin Yang

The Scientific Question: The WSI “Compute Wall”

Pathology models typically use ImageNet pre-trained backbones, which suffer from a domain gap. However, end-to-end full fine-tuning on WSIs (thousands of patches) requires VRAM far beyond standard GPU capabilities.

Core Method: Sparse Critical Instance Selection

Based on Variational Information Bottleneck (VIB) theory, we screen for the “minimal sufficient statistics”:

IB Module Screening: A lightweight module selects the Top-K diagnostic instances (usually <1000) based on mutual information maximization.
Sparse Backpropagation: Gradients are back-propagated only through selected instances during fine-tuning, reducing computational overhead by >10x.

Figure 3: The three-stage VIB process: Learning the Bottleneck -> Sparse representation fine-tuning -> Retraining the WSI head.

Results

Method,Camelyon-16 F1,Camelyon-16 AUC,TCGA-BRCA F1,TCGA-BRCA AUC,LBP-CECA F1,LBP-CECA AUC Full Supervision,0.967±0.005,0.992±0.003,-,-,0.741±0.006,0.942±0.002 RNN-MIL [7],0.834±0.017,0.861±0.021,0.776±0.035,0.871±0.033,-,- AB-MIL [19],0.828±0.013,0.851±0.025,0.771±0.040,0.869±0.037,0.525±0.017,0.845±0.002 DS-MIL [25],0.857±0.023,0.892±0.012,0.775±0.044,0.875±0.041,-,- CLAM-SB [30],0.839±0.018,0.875±0.028,0.797±0.046,0.879±0.019,0.587±0.014,0.860±0.005 TransMIL [38],0.846±0.013,0.883±0.009,0.806±0.046,0.889±0.036,0.533±0.006,0.850±0.007 DTFD-MIL [45],0.882±0.008,0.932±0.016,0.816±0.045,0.895±0.042,0.569±0.026,0.847±0.003 FT+ CLAM-SB,0.911±0.017,0.956±0.013,0.845±0.032,0.935±0.027,0.718±0.010,0.907±0.005 FT+ TransMIL,0.923±0.012,0.967±0.003,0.848±0.044,0.945±0.020,0.720±0.024,0.918±0.004 FT+ DTFD-MIL,0.921±0.007,0.962±0.006,0.849±0.027,0.951±0.016,0.723±0.008,0.922±0.005 Mean-pooling,0.629±0.029,0.591±0.012,0.818±0.022,0.910±0.032,0.350±0.017,0.735±0.006 Max-pooling,0.805±0.012,0.824±0.016,0.644±0.179,0.826±0.096,0.636±0.064,0.893±0.019 KNN (Mean),0.468±0.000,0.506±0.000,0.633±0.066,0.749±0.055,0.393±0.000,0.650±0.000 KNN (Max),0.559±0.000,0.535±0.000,0.524±0.032,0.639±0.063,0.477±0.000,0.743±0.000 FT+ Mean-pooling,0.842±0.006,0.831±0.007,0.866±0.035,0.952±0.018,0.685±0.014,0.900±0.002 FT+ Max-pooling,0.927±0.011,0.969±0.004,0.852±0.043,0.948±0.019,0.695±0.013,0.912±0.004 FT+ KNN (Mean),0.505±0.000,0.526±0.000,0.784±0.044,0.907±0.034,0.529±0.000,0.737±0.000 FT+ KNN (Max),0.905±0.000,0.916±0.000,0.802±0.063,0.882±0.036,0.676±0.000,0.875±0.000

Table 3. Slide-Level Classification by using the IN-lK pre-trained backbone or the proposed fine-tuned (FT) in three datasets. Top RowsDifierent Mll, architectures are compared to select the top 3 $OTA methods to validate the transfer learning performance using the IN-lKbre-trained backbone or the FT, Bottom Rows. The competition of various traditional aggrcgalion and feature evaluation methods by usingore-trained IN-lK or the FT.

Figure 6. T-SNE visualization of different representations onpatches. Our method converts chaotic ImageNet-lK and SSL fea-tures into a more task-specifc and separable distribution. Thecluster evaluation measurement, v-scores, show weakly super-vised fine-tuned features are more close to full supervision com-pared to others. a. ImageNet-1k pretraining. b. Full patch supervi-sion.c.Self-supervised Learning. d. Fine-tuning with WSI labels.

Performance Leap: On Camelyon16, a VIB fine-tuned ResNet-50 with simple Max-pooling achieved an AUC of 0.969, a 14.5% jump over the ImageNet baseline (0.824).
Feature Space: t-SNE visualization confirms significantly improved inter-class separation.

Summary

This research directly targets the “compute” and “data” bottlenecks in pathological AI deployment.

LongMIL & ACMIL reconstruct WSI attention mechanisms.
BD Framework solves the pain point of missing clinical data.
VIB Fine-tuning breaks the compute barrier for large-scale model optimization.

Together, these provide the core algorithmic support for building high-precision, low-cost, and robust pathological AI systems.