Everything about Transfer Learning. 迁移学习.

Papers • Tutorials • Research areas • Theory • Survey • Code • Dataset & benchmark

Thesis • Scholars • Contests • Journal/conference • Applications • Others • Contributing

Widely used by top conferences and journals:

• Conferences: [CVPR'22] [NeurIPS'21] [IJCAI'21] [ESEC/FSE'20] [IJCNN'20] [ACMMM'18] [ICME'19]
• Journals: [IEEE TKDE] [ACM TIST] [Information sciences] [Neurocomputing] [IEEE Transactions on Cognitive and Developmental Systems]

@Misc{transferlearning.xyz,
howpublished = {\url{http://transferlearning.xyz}},   
title = {Everything about Transfer Learning and Domain Adapation},  
author = {Wang, Jindong and others}  
}  

Related Codes:

• Large language model evaluation: [llm-eval]
• Large language model enhancement: [llm-enhance]
• Robust machine learning: [robustlearn: robust machine learning]
• Semi-supervised learning: [USB: unified semi-supervised learning benchmark] | [TorchSSL: a unified SSL library]
• LLM benchmark: [PromptBench: adverarial robustness of prompts of LLMs]
• Federated learning: [PersonalizedFL: library for personalized federated learning]
• Activity recognition and machine learning [Activity recognition]｜[Machine learning]

---

NOTE: You can directly open the code in Gihub Codespaces on the web to run them without downloading! Also, try github.dev.

0.Papers (论文)

Awesome transfer learning papers (迁移学习文章汇总)

• Paperweekly: A website to recommend and read paper notes

Latest papers:

• By topic: doc/awesome_papers.md
• By date: doc/awesome_paper_date.md

Updated at 2024-03-12:

• Attention Prompt Tuning: Parameter-efficient Adaptation of Pre-trained Models for Spatiotemporal Modeling [arxiv]

  • Parameter-efficient adaptation for spatiotemporal modeling

• ICASSP'24 Test-time Distribution Learning Adapter for Cross-modal Visual Reasoning [arxiv]

  • Test-time distribution learning adapter

• A Study on Domain Generalization for Failure Detection through Human Reactions in HRI [arxiv]

  • Domain generalization for failure detection through human reactions in HRI

• ICLR'24 Towards Robust Out-of-Distribution Generalization Bounds via Sharpness [arxiv]

  • Robust OOD generalization bounds

• Learning with Noisy Foundation Models [arxiv]

  • Learning with noisy foundation models

Updated at 2024-02-26:

• Unsupervised Domain Adaptation within Deep Foundation Latent Spaces [arxiv]
  • Domain adaptation using foundation models

Updated at 2024-01-26:

• Facing the Elephant in the Room: Visual Prompt Tuning or Full Finetuning? [arxiv]

  • A comparison between visual prompt tuning and full finetuning 比较prompt tuning和全finetune

• Out-of-Distribution Detection & Applications With Ablated Learned Temperature Energy [arxiv]

  • OOD detection for ablated learned temperature energy

• LanDA: Language-Guided Multi-Source Domain Adaptation [arxiv]

  • Language guided multi-source DA 在多源域自适应中使用语言指导

• AdaEmbed: Semi-supervised Domain Adaptation in the Embedding Space [arxiv]

  • Semi-spuervised domain adaptation in the embedding space 在嵌入空间中进行半监督域自适应

• Inter-Domain Mixup for Semi-Supervised Domain Adaptation [arxiv]

  • Inter-domain mixup for semi-supervised domain adaptation 跨领域mixup用于半监督域自适应

• Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimation [arxiv]

  • Source-free and image-only unsupervised domain adaptation

Updated at 2024-01-17:

• ICLR'24 spotlight Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks [arxiv]

  • A new research direction of transfer learning in the era of foundation models 大模型时代一个新研究方向：研究预训练数据的噪声对下游任务影响

• ICLR'24 Supervised Knowledge Makes Large Language Models Better In-context Learners [arxiv]

  • Small models help large language models for better OOD 用小模型帮助大模型进行更好的OOD

Updated at 2024-01-16:

• NeurIPS'23 Geodesic Multi-Modal Mixup for Robust Fine-Tuning [paper]

  • Geodesic mixup for robust fine-tuning

• NeurIPS'23 Parameter and Computation Efficient Transfer Learning for Vision-Language Pre-trained Models [paper]

  • Parameter and computation efficient transfer learning by reinforcement learning

• NeurIPS'23 Test-Time Distribution Normalization for Contrastively Learned Visual-language Models [paper]

  • Test-time distribution normalization for contrastively learned VLM

• NeurIPS'23 A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP) [paper]

  • A fine-gained analysis of CLIP robustness

• NeurIPS'23 When Visual Prompt Tuning Meets Source-Free Domain Adaptive Semantic Segmentation [paper]

  • Source-free domain adaptation using visual prompt tuning

Updated at 2024-01-08:

• NeurIPS'23 CODA: Generalizing to Open and Unseen Domains with Compaction and Disambiguation [arxiv]
  • Open set domain generalization using extra classes

Updated at 2024-01-05:

• CPAL'24 FIXED: Frustratingly Easy Domain Generalization with Mixup [arxiv]

  • Easy domain generalization with mixup

• SDM'24 Towards Optimization and Model Selection for Domain Generalization: A Mixup-guided Solution [arxiv]

  • Optimization and model selection for domain generalization

• Leveraging SAM for Single-Source Domain Generalization in Medical Image Segmentation [arxiv]

  • SAM for single-source domain generalization

• Multi-Source Domain Adaptation with Transformer-based Feature Generation for Subject-Independent EEG-based Emotion Recognition [arxiv]

  • Multi-source DA with Transformer-based feature generation

---

1.Introduction and Tutorials (简介与教程)

Want to quickly learn transfer learning？想尽快入门迁移学习？看下面的教程。

• Books 书籍

  • Introduction to Transfer Learning: Algorithms and Practice [Buy or read]
  • 《迁移学习》（杨强） [Buy] [English version]
  • 《迁移学习导论》(王晋东、陈益强著) [Homepage] [Buy]

• Blogs 博客

  • Zhihu blogs - 知乎专栏《小王爱迁移》系列文章

• Video tutorials 视频教程

  • Transfer learning 迁移学习:
    • Recent advance of transfer learning - 2022年最新迁移学习发展现状探讨
    • Definitions of transfer learning area - 迁移学习领域名词解释 [Article]
    • Transfer learning by Hung-yi Lee @ NTU - 台湾大学李宏毅的视频讲解(中文视频)
  • Domain generalization 领域泛化：
    • IJCAI-ECAI'22 tutorial on domain generalization - 领域泛化tutorial
    • Domain generalization - 迁移学习新兴研究方向领域泛化
  • Domain adaptation 领域自适应：
    • Domain adaptation - 迁移学习中的领域自适应方法(中文)

• Brief introduction and slides 简介与ppt资料

  • Recent advance of transfer learning
  • Domain generalization survey
  • Brief introduction in Chinese
    • PPT (English) | PPT (中文)
  • 迁移学习中的领域自适应方法 Domain adaptation: PDF ｜ Video on Bilibili | Video on Youtube
  • Tutorial on transfer learning by Qiang Yang: IJCAI'13 | 2016 version

• Talk is cheap, show me the code 动手教程、代码、数据

  • Pytorch tutorial on transfer learning
    • Pytorch finetune
    • DeepDA: a unified deep domain adaptation toolbox
    • DeepDG: a unified deep domain generalization toolbox
    • 更多 More...

• Transfer Learning Scholars and Labs - 迁移学习领域的著名学者、代表工作及实验室介绍

• Negative transfer - 负迁移

---

2.Transfer Learning Areas and Papers (研究领域与相关论文)

• Survey
• Theory
• Per-training/Finetuning
• Knowledge distillation
• Traditional domain adaptation
• Deep domain adaptation
• Domain generalization
• Source-free domain adaptation
• Multi-source domain adaptation
• Heterogeneous transfer learning
• Online transfer learning
• Zero-shot / few-shot learning
• Multi-task learning
• Transfer reinforcement learning
• Transfer metric learning
• Federated transfer learning
• Lifelong transfer learning
• Safe transfer learning
• Transfer learning applications

---

3.Theory and Survey (理论与综述)

Here are some articles on transfer learning theory and survey.

Survey (综述文章)：

• 2023 Source-Free Unsupervised Domain Adaptation: A Survey [arxiv]
• 2022 Transfer Learning for Future Wireless Networks: A Comprehensive Survey
• 2022 A Review of Deep Transfer Learning and Recent Advancements
• 2022 Transferability in Deep Learning: A Survey, from Mingsheng Long in THU.
• 2021 Domain generalization: IJCAI-21 Generalizing to Unseen Domains: A Survey on Domain Generalization | 知乎文章 | 微信公众号
  • First survey on domain generalization
  • 第一篇对Domain generalization (领域泛化)的综述
• 2021 Vision-based activity recognition: A Survey of Vision-Based Transfer Learning in Human Activity Recognition
• 2021 ICSAI A State-of-the-Art Survey of Transfer Learning in Structural Health Monitoring
• 2020 Transfer learning: survey and classification, Advances in Intelligent Systems and Computing.
• 2020 迁移学习最新survey，来自中科院计算所庄福振团队，发表在Proceedings of the IEEE: A Comprehensive Survey on Transfer Learning
• 2020 负迁移的综述：Overcoming Negative Transfer: A Survey
• 2020 知识蒸馏的综述: Knowledge Distillation: A Survey
• 用transfer learning进行sentiment classification的综述：A Survey of Sentiment Analysis Based on Transfer Learning
• 2019 一篇新survey：Transfer Adaptation Learning: A Decade Survey
• 2018 一篇迁移度量学习的综述: Transfer Metric Learning: Algorithms, Applications and Outlooks
• 2018 一篇最近的非对称情况下的异构迁移学习综述：Asymmetric Heterogeneous Transfer Learning: A Survey
• 2018 Neural style transfer的一个survey：Neural Style Transfer: A Review
• 2018 深度domain adaptation的一个综述：Deep Visual Domain Adaptation: A Survey
• 2017 多任务学习的综述，来自香港科技大学杨强团队：A survey on multi-task learning
• 2017 异构迁移学习的综述：A survey on heterogeneous transfer learning
• 2017 跨领域数据识别的综述：Cross-dataset recognition: a survey
• 2016 A survey of transfer learning。其中交代了一些比较经典的如同构、异构等学习方法代表性文章。
• 2015 中文综述：迁移学习研究进展
• 2010 A survey on transfer learning
• Survey on applications - 应用导向的综述：
  • 视觉domain adaptation综述：Visual Domain Adaptation: A Survey of Recent Advances
  • 迁移学习应用于行为识别综述：Transfer Learning for Activity Recognition: A Survey
  • 迁移学习与增强学习：Transfer Learning for Reinforcement Learning Domains: A Survey
  • 多个源域进行迁移的综述：A Survey of Multi-source Domain Adaptation。

Theory （理论文章）:

• ICML-20 Few-shot domain adaptation by causal mechanism transfer
  • The first work on causal transfer learning
  • 日本理论组大佬Sugiyama的工作，causal transfer learning
• CVPR-19 Characterizing and Avoiding Negative Transfer
  • Characterizing and avoid negative transfer
  • 形式化并提出如何避免负迁移
• ICML-20 On Learning Language-Invariant Representations for Universal Machine Translation
  • Theory for universal machine translation
  • 对统一机器翻译模型进行了理论论证
• NIPS-06 Analysis of Representations for Domain Adaptation
• ML-10 A Theory of Learning from Different Domains
• NIPS-08 Learning Bounds for Domain Adaptation
• COLT-09 Domain adaptation: Learning bounds and algorithms
• MMD paper：A Hilbert Space Embedding for Distributions and A Kernel Two-Sample Test
• Multi-kernel MMD paper: Optimal kernel choice for large-scale two-sample tests

---

4.Code (代码)

Unified codebases for:

• Deep domain adaptation
• Deep domain generalization
• See all codes here: https://github.com/jindongwang/transferlearning/tree/master/code.

More: see HERE and HERE for an instant run using Google's Colab.

---

5.Transfer Learning Scholars (著名学者)

Here are some transfer learning scholars and labs.

全部列表以及代表工作性见这里

Please note that this list is far not complete. A full list can be seen in here. Transfer learning is an active field. If you are aware of some scholars, please add them here.

---

6.Transfer Learning Thesis (硕博士论文)

Here are some popular thesis on transfer learning.

这里, 提取码：txyz。

---

7.Datasets and Benchmarks (数据集与评测结果)

Please see HERE for the popular transfer learning datasets and benchmark results.

这里整理了常用的公开数据集和一些已发表的文章在这些数据集上的实验结果。

---

8.Transfer Learning Challenges (迁移学习比赛)

• Visual Domain Adaptation Challenge (VisDA)

---

Journals and Conferences

See here for a full list of related journals and conferences.

---

Applications (迁移学习应用)

• Computer vision
• Medical and healthcare
• Natural language processing
• Time series
• Speech
• Multimedia
• Recommendation
• Human activity recognition
• Autonomous driving
• Others

See HERE for transfer learning applications.

迁移学习应用请见这里。

---

Other Resources (其他资源)

• Call for papers:

  • Advances in Transfer Learning: Theory, Algorithms, and Applications, DDL: October 2021

• Related projects:

  • Salad: A semi-supervised domain adaptation library

---

Contributing (欢迎参与贡献)

If you are interested in contributing, please refer to HERE for instructions in contribution.

---

Copyright notice

[Notes]This Github repo can be used by following the corresponding licenses. I want to emphasis that it may contain some PDFs or thesis, which were downloaded by me and can only be used for academic purposes. The copyrights of these materials are owned by corresponding publishers or organizations. All this are for better adademic research. If any of the authors or publishers have concerns, please contact me to delete or replace them.