bioMoR: Biology-Guided Mixture-of-Recursions for Effective Genomic Learning

Koushik Howlader1, Tirtho Roy1, Md Tauhidul Islam2, Wei Le1,†

1 Iowa State University, USA

2 Stanford University, USA

†Corresponding author: weile@iastate.edu

Under Review Manuscript under review — not yet accepted or published

Paper arXiv Code (soon)

Overview of bioMoR

bioMoR overview: genomic tokenization and three biology-injection sites in a weight-shared recursive Transformer

Single-cell and multi-omics inputs are represented as gene or pathway tokens. Fixed biological knowledge is incorporated at three sites: (1) embedding smoothing refines related tokens, (2) biology-guided routing selects tokens for further computation, and (3) attention bias promotes interactions between biologically related tokens in the weight-shared recursive Transformer block. Because the block fθ is reused, recursion depth adds computation but no parameters.

Abstract

Transformer models for high-dimensional omics analysis process thousands of genes or pathways, although only a subset requires deep computation. Mixture-of-Recursions (MoR) improves efficiency through adaptive token-choice or expert-choice routing. We propose bioMoR, which, to the best of our knowledge, is the first framework to apply MoR to gene- and pathway-level learning. Our contributions include identifying three locations for integrating structured biological knowledge within an MoR backbone: graph-based information sharing refines token embeddings, a structural bias guides self-attention toward biologically related tokens, and a graph-aware router uses neighborhood information to determine each token's recursion depth. These techniques are centered on our insight that additional knowledge of token interaction can effectively help models construct embeddings and select which tokens should be learned more deeply. Across eight benchmarks spanning diverse omics data types and evaluated under a unified five-fold cross-validation protocol, bioMoR improves average macro-F1 by 8.2 percentage points and balanced accuracy by 7.1 percentage points over the strongest biology-agnostic MoR baseline while using 75 percent fewer parameters and up to 58 percent fewer FLOPs than a non-recursive Transformer. The selected marker genes or pathways provide biological interpretability, while their token-specific recursion depths reveal how computation is allocated.

Guiding assumption

We should perform computation and learning in a selective and structured manner, based on the semantics of omics data. The model should retain biologically meaningful marker gene / pathway tokens and spend more recursive computation on the tokens whose biological neighborhoods are most predictive for the task.

Genomic signal is sparse: a cell type, tumor state, or metastatic phenotype is usually driven by a small set of marker genes, regulatory programs, or pathways. Efficient-attention and adaptive-computation methods route tokens dynamically, but they are biology-agnostic — they do not know whether two genes co-express, share a pathway, or contribute to the same biological process. Biology-aware resources and models supply exactly that structure, but use it for feature engineering, architectural constraints, or post-hoc interpretation rather than to decide which tokens deserve deeper computation. bioMoR connects the two: biological structure becomes a control signal for adaptive computation, not only a static prior.

Contributions

  1. First Mixture-of-Recursions framework for omics — bioMoR applies MoR to gene- and pathway-level learning, compressing high-dimensional inputs into marker-gene or Reactome pathway tokens and routing each token to an adaptive number of recursive steps through one weight-shared Transformer block.
  2. Three sites for biological knowledge — gene co-expression and pathway-pathway relations are injected into representation learning (embedding smoothing), attention learning (structural bias), and adaptive routing (a zero-initialized, graph-aware router correction) on top of the MoR backbone.
  3. Extensive evaluation — across eight single-cell and multi-omics benchmarks under one unified five-fold protocol, the three sites significantly improve macro-F1 and balanced accuracy while reducing both parameter count and FLOP cost, and the gains hold across routing policies, recursion depths, marker budgets, and model widths.

Main results: eight benchmarks, one protocol

Single-cell Pathway-based
ModelRoutingParam SegerstolpeLungT-cellMuraroSpleen BLCAPAN-2MPAN-3MAvg.
Macro-F1
Vanilla—300K 62.5 ± 11.272.5 ± 2.552.1 ± 3.580.2 ± 4.348.3 ± 3.7 40.6 ± 5.875.6 ± 3.595.2 ± 1.565.9
Recursive—75K 61.4 ± 7.372.9 ± 2.950.5 ± 3.575.8 ± 4.149.4 ± 3.4 40.2 ± 1.172.3 ± 1.894.3 ± 1.164.6
MoRExpert75K 64.3 ± 11.672.3 ± 2.750.9 ± 4.774.8 ± 4.753.2 ± 1.3 43.7 ± 2.875.5 ± 3.094.3 ± 1.366.1
MoRToken75K 60.7 ± 5.873.3 ± 2.450.4 ± 2.281.6 ± 2.751.1 ± 3.1 40.6 ± 6.978.0 ± 4.695.3 ± 1.666.4
bioMoR (ours)Expert75K 72.2 ± 4.580.0 ± 1.270.0 ± 1.983.7 ± 4.959.7 ± 0.8 47.1 ± 7.185.8 ± 1.598.0 ± 0.474.6
bioMoR (ours)Token75K 75.8 ± 2.879.7 ± 0.969.8 ± 1.281.9 ± 6.861.0 ± 0.6 39.5 ± 5.986.0 ± 1.996.4 ± 1.673.8
Balanced accuracy
Vanilla—300K 71.7 ± 10.375.2 ± 1.555.2 ± 5.779.1 ± 3.355.3 ± 1.4 48.3 ± 2.874.1 ± 4.094.9 ± 1.969.2
Recursive—75K 69.6 ± 9.175.3 ± 1.956.5 ± 4.677.8 ± 2.456.9 ± 1.5 48.5 ± 2.874.4 ± 1.693.6 ± 1.769.1
MoRExpert75K 69.2 ± 6.176.5 ± 1.856.6 ± 6.175.5 ± 2.357.8 ± 2.5 49.8 ± 5.074.6 ± 3.193.7 ± 3.169.2
MoRToken75K 66.4 ± 8.076.1 ± 2.256.6 ± 4.180.6 ± 8.556.5 ± 1.4 46.7 ± 7.873.5 ± 2.293.7 ± 4.668.8
bioMoR (ours)Expert75K 78.8 ± 4.980.4 ± 0.670.4 ± 1.688.9 ± 4.064.7 ± 1.6 48.7 ± 7.582.9 ± 3.395.4 ± 1.376.3
bioMoR (ours)Token75K 73.3 ± 7.082.0 ± 1.571.3 ± 1.483.4 ± 3.764.1 ± 1.1 48.3 ± 2.283.4 ± 2.696.1 ± 1.975.2

Macro-F1 and balanced accuracy (mean ± SD) from five-fold stratified cross-validation on eight benchmark datasets. All rows use the same folds and operating point; Recursive, MoR, and bioMoR use K = 4. Avg. is the arithmetic mean across the eight displayed datasets; underlining marks the top two results per dataset. The best bioMoR configuration improves average macro-F1 by 8.2 points and balanced accuracy by 7.1 points over the strongest non-bioMoR baseline — at a quarter of the parameters of the vanilla Transformer.

Comparison with external baselines

Mean macro-F1 for bioMoR and external baselines on PANCANCER and Lung

Mean macro-F1 ± SD for bioMoR and dataset-appropriate baselines on (a) pathway-based PANCANCER and (b) single-cell Lung under the matched five-fold evaluation setting. bioMoR obtains the highest mean macro-F1 in both panels: 0.86 ± 0.01 on PANCANCER versus 0.84 ± 0.01 for scBiGNN, and 0.80 ± 0.01 on Lung versus 0.78 ± 0.03 for TabNet. Baselines span Random Forest, Nearest-Centroid, scBiGNN, MOGONET, Pathformer, Geneformer, scGPT, and scTransformer-style models.

Beyond classification: survival prediction

Fold-level C-index distributions on four TCGA cohorts

Fold-level C-index distributions for the HNSC, UCEC, COAD, and KIRP TCGA cohorts; higher values indicate better concordance between predicted risk and observed survival outcomes. bioMoR-Expert and bioMoR-Token stay competitive with the Vanilla, Recursive, MoR-Expert, and MoR-Token baselines across all four cohorts, showing that biology-guided adaptive recursion transfers beyond classification to a time-to-event endpoint.

Which injection site matters?

Injection-site ablation: macro-F1 with biological knowledge at embedding, attention, router, or all three sites

Macro-F1 across Spleen, Segerstolpe, Xin, and Pancancer-3M with biological-knowledge injection at the embedding, attention-bias, router, or all three sites; all runs use expert-choice routing with K = 4, and the inset shows average rank across datasets (1 is best). Injecting at all three sites achieves the highest macro-F1. Among single-site variants, embedding injection performs best, followed by router-only and then attention-bias-only — the three sites carry complementary information rather than duplicating one signal.

Training and computational efficiency

T-cell training curves, checkpoint-selected accuracy, and macro-F1 versus FLOPs

T-cell with expert-choice routing at K = 4. (a) Training loss and (b) validation macro-F1 over epochs; bioMoR drops the loss fastest and reaches nearly 70% validation macro-F1 while the baselines stall near 40–50%. (c) Test macro-F1 and training epoch at checkpoints selected by lowest validation loss (○) or highest validation macro-F1 (★) — bioMoR wins under both rules (69.9%). (d) Test macro-F1 versus FLOPs for K = 2, 3, 4 with bubble size proportional to parameter count; bioMoR holds 69–70% macro-F1 at every depth while using fewer FLOPs than Vanilla — 58% fewer at K = 2.

The gain does not depend on the configuration

ModelRouting Normalized FLOPs / macro-F1
K = 2 K = 3 K = 4
Vanilla———1.00 / 52.1 ± 3.5
Recursive—0.50 / 53.0 ± 2.00.75 / 49.8 ± 4.41.00 / 50.5 ± 3.5
MoRExpert0.42 / 50.6 ± 4.90.59 / 52.3 ± 2.40.76 / 50.9 ± 4.7
MoRToken0.42 / 50.7 ± 2.20.52 / 55.8 ± 3.00.56 / 50.4 ± 2.2
bioMoRExpert0.42 / 69.2 ± 0.70.59 / 69.6 ± 0.60.76 / 70.0 ± 1.9
bioMoRToken0.42 / 68.6 ± 1.70.52 / 68.6 ± 0.90.56 / 69.8 ± 1.2

T-cell macro-F1 across routing policies and recursion depths; each cell reports normalized FLOPs / macro-F1 (mean ± SD). Vanilla, Recursive, and biology-agnostic MoR stay far lower everywhere, while bioMoR stays high under both routing policies and at every depth. The main effect is the biological knowledge; routing policy and depth tune the accuracy–efficiency trade-off around it. The same holds as the marker budget grows from M = 128 to 2048 (mean macro-F1 71.8–73.5) and as model width grows from d = 96 to 352 (mean macro-F1 70.5–72.7).

Better representations, not just a better head

UMAP of frozen penultimate embeddings on Segerstolpe for Vanilla, Recursive, MoR, and bioMoR

UMAP visualization of frozen penultimate per-cell embeddings on Segerstolpe for Vanilla, Recursive, MoR, and bioMoR, colored by cell type. Probe accuracy comes from freezing each trained model and fitting a logistic-regression classifier on the training cells. bioMoR forms cleaner cell-type groups and the linear probe performs best on its embeddings (0.92 linear-probe accuracy), indicating that the gains come from the learned representation rather than the final prediction layer alone.

Which pathways get the deepest computation?

Biological case study: pathway tokens kept for further recursion versus frozen tokens

On PAN-2M — localized versus metastatic cancer, one token per named Reactome pathway — the router scores active tokens at each recursion; the top 75% continue (green) and the remaining 25% stop and keep their current representations (yellow). The pathways reaching the greatest depth include Wnt signaling, epithelial-to-mesenchymal transition (EMT), and PI3K–Akt signaling — all independently linked in the literature to colonization of distant organs, invasiveness, and outgrowth of micrometastases. Because tokens carry pathway names, the depth assignment is directly readable. This agreement with published studies is encouraging, but it does not establish a causal relationship.

BibTeX

Status: preprint available on arXiv; the manuscript is under review — not yet accepted or published.

@misc{howlader2026biomor,
  title         = {bioMoR: Biology-Guided Mixture-of-Recursions for Effective Genomic Learning},
  author        = {Howlader, Koushik and Roy, Tirtho and Islam, Md Tauhidul and Le, Wei},
  year          = {2026},
  eprint        = {2608.06727},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  url           = {https://arxiv.org/abs/2608.06727}
}

Team

Koushik Howlader
Tirtho Roy
Md Tauhidul Islam
Wei Le