What is transfer learning and when is fine-tuning better than feature extraction?
mediumAnswer
- Transfer learning reuses a model pretrained on a large source task on a new target task.
- Feature extraction freezes the backbone and trains a new head — fast, works when the target is small and similar to pretraining data.
- Full fine-tuning updates all weights — better when you have enough target data or when target differs significantly.
- Modern LLMs often use parameter-efficient fine-tuning (LoRA, adapters) to combine both benefits.
Check yourself — multiple choice
- Feature extraction updates all weights
- Fine-tuning is preferred when target data is large or differs from pretraining
- LoRA is a full fine-tuning method
- Transfer learning requires target labels equal in count to source labels
Larger, more distinct target data justifies full or LoRA fine-tuning.
#transfer-learning#fine-tuning
Practise Deep Learning
214 interview questions in this topic.