1. 2022.10至今  中国科学院自动化研究所 紫东太初大模型研究中心 副研究员
2. 2022.3-2022.9 中国科学院自动化研究所 模式识别国家重点实验室 副研究员
3. 2019.7-2022.2 中国科学院自动化研究所 模式识别国家重点实验室 助理研究员
4. 2013.9-2019.6 北京邮电大学 博士
5. 2009.9-2013.6 河北师范大学 本科

多模态大模型及其应用

1. 面向图像语义描述的网络结构搜索方法研究 国家自然青年基金 2022.01-2024.12 主持
2. 多模态预训练技术项目 荣耀终端有限公司 2023.04-2024.12 主持
3. 基于Mindspore的千亿级图文音三模态大模型 中国人工智能学会-华为MindSpore学术奖励基金研究课题 2021.11-2022.11 主持

1. Sun, Mingzhen; Wang, Weining; Zhu, Xinxin; Liu, Jing;  Reparameterizing and dynamically quantizing image features for image generation.  Pattern Recognition 2024
2. Chen, Sihan; Li, Handong; Wang, Qunbo; Zhao, Zijia; Sun, Mingzhen; Zhu, Xinxin; Liu, Jing;  Vast: A vision-audio-subtitle-text omni-modality foundation model and dataset.  Advances in Neural Information Processing Systems 2024
3. Zhao, Zijia; Guo, Longteng; Yue, Tongtian; Chen, Sihan; Shao, Shuai; Zhu, Xinxin; Yuan, Zehuan; Liu, Jing;  Chatbridge: Bridging modalities with large language model as a language catalyst.  arXiv preprint arXiv:2305.16103 2023
4. Sun, Mingzhen; Wang, Weining; Zhu, Xinxin; Liu, Jing;  MOSO: Decomposing motion, scene and object for video prediction.  Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023
5. Sun, Mingzhen; Wang, Weining; Qiao, Yanyuan; Guo, Longteng; Sun, Jiahui; Zhu, Xinxin; Liu, Jing;  MM-LDM: Multi-Modal Latent Diffusion Model for Sounding Video Generation.   2023
6. Lu, Shichen; Guo, Longteng; He, Xingjian; Zhu, Xinxin; Liu, Jing; Liu, Si;  CSDNet: Contrastive Similarity Distillation Network for Multi-lingual Image-Text Retrieval.  International Conference on Image and Graphics 2023
7. Liu, Jiawei; Wang, Weining; Chen, Sihan; Zhu, Xinxin; Liu, Jing;  Sounding video generator: A unified framework for text-guided sounding video generation.  IEEE Transactions on Multimedia 2023
8. Chen, Sihan; He, Xingjian; Guo, Longteng; Zhu, Xinxin; Wang, Weining; Tang, Jinhui; Liu, Jing;  Valor: Vision-audio-language omni-perception pretraining model and dataset. arXiv preprint arXiv:2304.08345 2023
9. Liu, Wei; Chen, Sihan; Guo, Longteng; Zhu, Xinxin; Liu, Jing;  Cptr: Full transformer network for image captioning. arXiv preprint arXiv:2101.10804 2021
10. Liu, Jing; Zhu, Xinxin; Liu, Fei; Guo, Longteng; Zhao, Zijia; Sun, Mingzhen; Wang, Weining; Lu, Hanqing; Zhou, Shiyu; Zhang, Jiajun;  OPT: Omni-perception pre-trainer for cross-modal understanding and generation.  arXiv preprint arXiv:2107.00249 2021
11. Li, Jiangyun; Zhao, Yikai; He, Xingjian; Zhu, Xinxin; Liu, Jing;  Dynamic warping network for semantic video segmentation.  Complexity 2021
12. He, Xingjian; Liu, Jing; Fu, Jun; Zhu, Xinxin; Wang, Jinqiao; Lu, Hanqing;  Consistent-separable feature representation for semantic segmentation.  Proceedings of the AAAI Conference on Artificial Intelligence 2021
13. Guo, Longteng; Liu, Jing; Zhu, Xinxin; Lu, Hanqing;  Fast sequence generation with multi-agent reinforcement learning.  arXiv preprint arXiv:2101.09698 2021
14. Chen, Sihan; Zhu, Xinxin; Liu, Wei; He, Xingjian; Liu, Jing;  Global-local propagation network for RGB-D semantic segmentation.  arXiv preprint arXiv:2101.10801 2021
15. Chen, Sihan; Zhu, Xinxin; Hao, Dongze; Liu, Wei; Liu, Jiawei; Zhao, Zijia; Guo, Longteng; Liu, Jing;  Mm21 pre-training for video understanding challenge: Video captioning with pretraining techniques.  Proceedings of the 29th ACM International Conference on Multimedia 2021
16. Zhu, Xinxin; Wang, Weining; Guo, Longteng; Liu, Jing;  AutoCaption: Image captioning with neural architecture search.  arXiv preprint arXiv:2012.09742 2020
17. Liu, Fei; Liu, Jing; Zhu, Xinxin; Hong, Richang; Lu, Hanqing;  Dual hierarchical temporal convolutional network with QA-aware dynamic normalization for video story question answering.  Proceedings of the 28th ACM International Conference on Multimedia 2020
18. Jiang, Jie; Liu, Jing; Fu, Jun; Zhu, Xinxin; Lu, Hanqing;  Point Set Attention Network For Semantic Segmentation.  2020 IEEE International Conference on Image Processing (ICIP) 2020
19. Jiang, Jie; Liu, Jing; Fu, Jun; Zhu, Xinxin; Li, Zechao; Lu, Hanqing;  Global-guided selective context network for scene parsing.  IEEE Transactions on Neural Networks and Learning Systems 2020
20. Guo, Longteng; Liu, Jing; Zhu, Xinxin; Yao, Peng; Lu, Shichen; Lu, Hanqing;  Normalized and geometry-aware self-attention network for image captioning.  Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2020
21. Guo, Longteng; Liu, Jing; Zhu, Xinxin; He, Xingjian; Jiang, Jie; Lu, Hanqing;  Non-autoregressive image captioning with counterfactuals-critical multi-agent learning.  arXiv preprint arXiv:2005.04690 2020
22. Zhu, Xinxin; Guo, Longteng; Yao, Peng; Lu, Shichen; Liu, Wei; Liu, Jing;  Vatex video captioning challenge 2020: Multi-view features and hybrid reward strategies for video captioning.  arXiv preprint arXiv:1910.11102 2019
23. Zhu, Xinxin; Guo, Longteng; Yao, Peng; Liu, Jing; Lu, Hanqing; Yu, Zheng; Liu, Wei; Lu, Hanqing;  Multi-view features and hybrid reward strategies for vatex video captioning challenge 2019.  arXiv preprint arXiv:1910.11102 2019
24. Zhu, Xinxin; Li, Lixiang; Liu, Jing; Peng, Haipeng; Niu, Xinxin;  Captioning transformer with stacked attention modules.  Applied Sciences 2018
25. Zhu, Xinxin; Li, Lixiang; Liu, Jing; Li, Ziyi; Peng, Haipeng; Niu, Xinxin;  Image captioning with triple-attention and stack parallel LSTM.  Neurocomputing 2018
26. Zhu, Xinxin; Li, Lixiang; Liu, Jing; Guo, Longteng; Fang, Zhiwei; Peng, Haipeng; Niu, Xinxin;  Image captioning with word gate and adaptive self-critical learning.  Applied Sciences 2018

1. 2022年度世界人工智能大会最高奖项SAIL奖
2. ACM MM 2021视频理解竞赛冠军
3. CVPR 2020视频描述竞赛VATEX中英文赛道冠军
4. ICCV 2019视频描述竞赛VATEX中英文赛道分获亚季军
5. AI Challenge 2017全球AI挑战赛中文图像语义描述比赛双周赛冠军

1. CN113591902B，基于多模态预训练模型的跨模态理解与生成方法和装置，刘静,朱欣鑫,刘飞,郭龙腾
2. CN114598926B，一种视频生成方法、装置、电子设备及存储介质,王卫宁,朱欣鑫,刘静,孙铭真,刘佳伟
3. CN 202310369935.6，图文音多模态预训练模型方法、装置、电子设备和介质,刘静,何兴建,陈思涵,王卫宁,朱欣鑫