Deep Learning
Interview Questions with Answers

41. What is model overfitting and how can you detect it from training curves?

Answer:
If training accuracy is high and validation accuracy is low, the model is overfitting.
A gap between training and validation loss curves is a strong indicator.

42. What is model fine-tuning and how is it done in Keras?

Answer:
Fine-tuning involves unfreezing some layers of a pre-trained model and retraining it on a new dataset.

for layer in model.layers[-5:]:

    layer.trainable = True

43. What is the difference between functional and sequential API in Keras?

Answer:

• Sequential API: For linear stack of layers.
• Functional API: For building non-linear, multi-input/output models.

# Functional

input = Input(shape=(32,))

x = Dense(64)(input)

output = Dense(1)(x)

model = Model(inputs=input, outputs=output)

44. What is a custom loss function in Keras and how do you define it?

Answer:

import tensorflow.keras.backend as K

def custom_loss(y_true, y_pred):

    return K.mean(K.square(y_true – y_pred), axis=-1)

model.compile(loss=custom_loss, optimizer=’adam’)

Used when built-in loss functions aren’t sufficient.

45. What is data augmentation and how do you use it for training image models?

Answer:
Data augmentation artificially expands your dataset with transformations (rotation, flip, zoom).

from tensorflow.keras.preprocessing.image import ImageDataGenerator
datagen = ImageDataGenerator(rotation_range=20, zoom_range=0.2, horizontal_flip=True)

Helps reduce overfitting and improve generalization.

46. What is the Attention Mechanism in deep learning, and how is it implemented?

Answer:
The attention mechanism allows a model to focus on relevant parts of the input sequence during output generation. It computes weighted importance (attention scores) for each input token.

Implementation:

import tensorflow as tf

def scaled_dot_product_attention(q, k, v):
    matmul_qk = tf.matmul(q, k, transpose_b=True)
    d_k = tf.cast(tf.shape(k)[-1], tf.float32)
    scaled_attention_logits = matmul_qk / tf.math.sqrt(d_k)
    attention_weights = tf.nn.softmax(scaled_attention_logits, axis=-1)
    return tf.matmul(attention_weights, v)

47. What are Transformers and why are they important?

Answer:
Transformers are sequence models that replace RNNs and CNNs using self-attention and positional encoding. They enable parallelization and outperform in NLP and Vision tasks.

Popular models: BERT, GPT, ViT (Vision Transformer)

48. What are Generative Adversarial Networks (GANs)?

Answer:
GANs consist of two networks:

• Generator: Produces fake data
• Discriminator: Distinguishes real from fake

They are trained adversarially:

# Pseudo-code

for step in training_steps:

    train_discriminator()

    train_generator()

Used in image synthesis, super-resolution, and deepfake generation.

49. What is Layer Normalization and how does it differ from Batch Normalization?

Answer:

• BatchNorm normalizes across a batch
• LayerNorm normalizes across features for a single sample
  Useful in Transformer models

from tensorflow.keras.layers import LayerNormalization
model.add(LayerNormalization())

50. How does BERT work and how is it fine-tuned for a classification task?

Answer:
BERT is a Transformer-based language model pre-trained on masked language modeling and next sentence prediction.

Implementation to fine-tune:

from transformers import BertTokenizer, TFBertForSequenceClassification
model = TFBertForSequenceClassification.from_pretrained('bert-base-uncased')
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')

51. What is multi-head attention and why is it useful?

Answer:
It allows the model to learn different attention patterns in parallel, capturing diverse relationships.

from tensorflow.keras.layers import MultiHeadAttention
attention = MultiHeadAttention(num_heads=4, key_dim=64)
output = attention(query, value, key)

52. How do you implement a custom callback in Keras for monitoring metrics?

Answer:

from tensorflow.keras.callbacks import Callback
class CustomLogger(Callback):
    def on_epoch_end(self, epoch, logs=None):
        print(f"Epoch {epoch}: Val Loss = {logs['val_loss']:.4f}")

Used for early stopping, custom logging, or model behavior control.

53. What is label smoothing and how is it implemented?

Answer:
Label smoothing replaces 1-hot encoded labels (0s and 1s) with soft values (e.g., 0.9 and 0.1) to prevent overconfidence. 

model.compile(loss=’categorical_crossentropy’, label_smoothing=0.1) 

54. How can you deploy a deep learning model using TensorFlow Serving?

Answer:

1. Export the model:

model.save(‘my_model’)

1. Start TensorFlow Serving:

tensorflow_model_server --rest_api_port=8501 --model_name=my_model --model_base_path=/models/my_model

1. Make predictions via REST API using curl or requests.

55. What is model quantization and when is it used?

Answer:
Model quantization reduces model size and speeds up inference by converting weights from float32 to int8.

converter = tf.lite.TFLiteConverter.from_saved_model("model/")
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()

Used in mobile and edge deployments.

56. Explain knowledge distillation in deep learning.

Answer:
A student model learns to mimic a teacher model’s soft labels (probability distribution).
Improves smaller model accuracy with fewer parameters.

57. What is a Vision Transformer (ViT) and how does it differ from CNNs?

Answer:
ViTs treat images as patch sequences, like tokens in NLP.
They do not use convolution but apply self-attention to image patches.

ViTs outperform CNNs with sufficient data and compute.

58. How do you implement gradient clipping in Keras?

Answer:

from tensorflow.keras.optimizers import Adam
opt = Adam(clipnorm=1.0)  # or clipvalue
model.compile(optimizer=opt, loss='mse')

Prevents exploding gradients during backpropagation.

59. What is class imbalance and how do you handle it in loss function?

Answer:
Class imbalance occurs when one class has significantly more samples than others. This can cause the model to become biased toward the majority class and perform poorly on the minority class.

Solution 1: Using class_weight in model training

You can assign higher weights to the minority class during training using the class_weight argument:

model.fit(X_train, y_train, class_weight={0: 1.0, 1: 3.0})  # Class 1 is weighted more

This tells the model to pay more attention to class 1 during optimization.

Solution 2: Using Focal Loss

Focal Loss is especially useful when class imbalance is severe. It reduces the loss contribution from well-classified examples and focuses more on hard, misclassified examples.

Formula:

Where:

• αt​ balances importance of classes
• γ focuses more on hard-to-classify examples

Focal Loss Implementation (TensorFlow/Keras)

import tensorflow as tf

from tensorflow.keras import backend as K
def focal_loss(gamma=2., alpha=0.25):
    def loss(y_true, y_pred):
        y_pred = K.clip(y_pred, K.epsilon(), 1. - K.epsilon())
        pt = tf.where(K.equal(y_true, 1), y_pred, 1 - y_pred)
        return -K.mean(alpha * K.pow(1. - pt, gamma) * K.log(pt))

    return loss

# Usage

model.compile(optimizer=’adam’, loss=focal_loss(gamma=2., alpha=0.25), metrics=[‘accuracy’])

60. What is the difference between static and dynamic computation graphs?

Answer:

• Static (TensorFlow 1.x): Graph is compiled before execution.
• Dynamic (PyTorch): Graph is created on the fly during runtime.

Dynamic graphs are easier to debug and write using native Python control flows.