Deep learning interviews check whether you understand why models work, not just whether you can call a library. Expect questions on neural network basics, how training behaves, common architectures, and how you’d handle a real project. Here are 25 common deep learning interview questions with concise answers. For broader topics, see our machine learning interview questions.
Neural Network Basics
1. What is deep learning, and how does it differ from traditional machine learning?
Deep learning uses neural networks with many layers that learn features directly from raw data. Traditional machine learning usually relies more on hand-built features. Deep learning tends to need more data and computing power, but it does especially well with images, audio and text.
2. Why do neural networks need activation functions?
Without a non-linear activation, stacked layers collapse into a single linear transformation, however deep the network is. Non-linearity lets the network model complex relationships.
3. Compare common activation functions.
ReLU is simple and works well in hidden layers but can produce “dead” neurons. Leaky ReLU and GELU reduce that problem. Sigmoid suits binary outputs and softmax suits multi-class outputs. Both saturate, so they’re rarely used in deep hidden layers.
4. What is backpropagation?
It applies the chain rule to compute the gradient of the loss with respect to every weight, working backward from the output. An optimizer then uses those gradients to update the weights.
5. What’s the difference between batch, mini-batch and stochastic gradient descent?
Batch uses all the data for each update, stochastic uses one example, and mini-batch uses a small group. Mini-batch is the usual choice because it balances stable updates with speed and works well on GPUs.
6. Which loss functions would you use, and when?
Cross-entropy for classification, mean squared error or mean absolute error for regression (MAE is less sensitive to outliers), and specialized losses such as focal loss when classes are very imbalanced.
Training and Optimization
7. What are vanishing and exploding gradients?
In deep networks, gradients can shrink toward zero or grow out of control as they pass through layers. ReLU-style activations, good weight initialization, normalization and residual connections help with vanishing gradients. Gradient clipping helps with exploding ones.
8. Compare SGD with momentum and Adam.
Adam adapts the learning rate for each parameter and often converges quickly with little tuning. SGD with momentum can take more tuning, but on some problems, such as image classification, it can generalize very well. AdamW, which handles weight decay properly, is a common default.
9. Why does the learning rate matter so much?
Too high and training diverges or oscillates. Too low and it’s slow or gets stuck. Warmup and decay schedules, such as cosine decay, are standard. A learning rate finder can help you choose a starting value.
10. What does batch normalization do? How is it different from layer normalization?
Both normalize activations to make training more stable and faster. Batch norm normalizes across the batch and behaves differently during training and inference. Layer norm normalizes across features for each example, so it doesn’t depend on batch size, which is why transformers use it.
11. How do you prevent overfitting?
Get more data or use augmentation, add dropout or weight decay, use early stopping, simplify the model, or use transfer learning. Always compare against a proper validation set.
12. How does dropout work?
During training it randomly switches off a share of the units, which stops the network from relying too heavily on any one of them. It’s turned off at inference, and frameworks handle the scaling so outputs stay consistent.
13. Your training loss isn’t going down. What do you check?
Try to overfit one small batch first. If that fails, look for bugs: data and labels that don’t match, wrong preprocessing, the wrong loss for the output, a poor learning rate, or weights that aren’t updating. Check the gradients as well.
14. Training accuracy is high but validation accuracy is low. What now?
That’s overfitting, or a difference between the two datasets. Check for data leakage and whether validation data matches real use, then try regularization, augmentation, more data or a smaller model.
Architectures
15. What makes convolutional networks good for images?
Convolutions share weights across positions and look at local patterns, so they need far fewer parameters than fully connected layers and can detect a feature wherever it appears. Stacked layers build from edges up to shapes and objects.
16. What do pooling and stride do?
Both reduce spatial resolution, which lowers computation and widens the area each later layer can see. Pooling summarizes a region, often using the maximum or the average, while stride sets how far the filter moves at each step.
17. Why do residual connections help?
Skip connections let gradients flow directly through the network and make it easier for layers to learn small changes on top of an identity mapping. That’s what made very deep networks like ResNet practical to train.
18. What’s the difference between an RNN and an LSTM?
RNNs process sequences step by step but struggle with long-range dependencies because of vanishing gradients. LSTMs add gates that control what to keep and forget, which helps them remember for longer. GRUs are a simpler variant.
19. What is attention?
It lets a model weigh which parts of the input matter most for the current output. Each element is compared with the others using queries, keys and values, and the result is a weighted combination.
20. How does a transformer work, and why did it replace RNNs for many tasks?
Transformers use self-attention, multiple attention heads, feed-forward layers and positional encodings, without processing the sequence step by step. They train in parallel and handle long-range relationships well, though they can take a lot of data and compute.
21. What is transfer learning, and when would you use it?
You start from a model trained on a related task, then fine-tune it or train only the last layers. It’s especially useful with limited data. With very little data, freeze most of the layers. With more, unfreeze more and use a lower learning rate.
22. What’s the difference between a VAE and a GAN?
A VAE learns a smooth latent space by optimizing reconstruction plus a regularization term, and it’s stable but can produce blurrier samples. A GAN trains a generator against a discriminator and can create sharp samples, but it’s harder to train and can suffer from mode collapse. Diffusion models are now common for high-quality image generation.
Real-World Projects
23. How do you handle imbalanced classes?
Use weighted losses or focal loss, resampling and targeted augmentation. Evaluate with precision, recall, F1 or PR-AUC rather than accuracy, and set your threshold based on the cost of each type of error.
24. How do you make a model faster for production?
Options include quantization, pruning, knowledge distillation, a smaller architecture, batching requests, and exporting to an optimized runtime such as ONNX Runtime. Measure latency and accuracy together, on the hardware you’ll really use.
25. Your model’s performance drops after deployment. What do you do?
Check for data or concept drift and for pipeline bugs, such as features computed differently in training and production. Monitor inputs and predictions, compare with a recent labeled sample, retrain if needed, and keep a way to roll back safely.
How to Prepare
- Be able to explain one project in depth: the data, your baseline, why you chose the model, what went wrong and how you measured success.
- Practice the math behind the basics: gradients, backpropagation on a small network, and tensor shapes through a layer.
- Write simple code from scratch: a training loop in PyTorch, and be able to debug it out loud.
- Prepare for general questions too: the system design interview questions and behavioral answers often matter as much as the technical round.
Get Help in the Interview Itself
Preparation gets you most of the way. For the interview itself, ApplyQuick Interview Copilot listens to each question and suggests a structured answer based on your resume and the job description, on Zoom, Microsoft Teams, Google Meet or a phone call. Your first session is free, no card needed.
