Continual learning that drops the disjoint-task assumption.
Continual learning frameworks for VLMs and LLMs that detect semantic overlap across tasks, consolidate redundant experts via on-policy self-distillation, and route inputs through zero-parameter or GMM-based routers. Outperforms strongest baselines by +7–15 points across disjoint and overlapping benchmarks while reducing deployed adapters by up to 3×.
I created the first VLM continual-learning benchmark with controlled inter-task overlap, because the standard disjoint-task setup hides the failure mode that matters most in production: tasks that share concepts but differ in distribution.
This is the model-side complement to the production agent system: the agent answers how to ship updates fast, this work answers how to ship them without forgetting. The thread connects to earlier domain-adaptation work (Xu et al., 2019; Zhou et al., 2020) that established stochastic neighborhood embedding for cross-domain transfer, and to recent advances in instruction grounding (Zong et al., 2025) and salient-concept-aware data augmentation (Zhao et al., 2025).
@inproceedings{zong2025groundv,title={Ground-V: Teaching VLMs to Ground Complex Instructions in Pixels},author={Zong, Yongshuo and Zhang, Qin and An, Dongsheng and Li, Zhihua and Xu, Xiang and Xu, Linghan and Tu, Zhuowen and Xing, Yifan and Dabeer, Onkar},booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},year={2025},}
@inproceedings{zhao2025salient,title={Salient Concept-Aware Generative Data Augmentation},author={Zhao, Tianchen and Chen, Xuanbai and Li, Zhihua and Fang, Jun and An, Dongsheng and Xu, Xiang and Tu, Zhuowen and Xing, Yifan},booktitle={Advances in Neural Information Processing Systems (NeurIPS)},year={2025},}
2020
Book
d-SNE: Domain Adaptation Using Stochastic Neighborhood Embedding
Xiong Zhou, Xiang Xu, Ragav Venkatesan, and 2 more authors
In Domain Adaptation in Computer Vision with Deep Learning, 2020
@incollection{zhou2020dsne,title={d-SNE: Domain Adaptation Using Stochastic Neighborhood Embedding},author={Zhou, Xiong and Xu, Xiang and Venkatesan, Ragav and Swaminathan, Gurumurthy and Majumder, Orchid},booktitle={Domain Adaptation in Computer Vision with Deep Learning},pages={43--56},year={2020},publisher={Springer International Publishing Cham},}
@inproceedings{xu2019dsne,title={d-SNE: Domain Adaptation Using Stochastic Neighborhood Embedding},author={Xu, Xiang and Zhou, Xiong and Venkatesan, Ragav and Swaminathan, Gurumurthy and Majumder, Orchid},booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},pages={2497--2506},year={2019},note={Oral}}