Generative AI using Python (Advanced)
Interview Questions with Answers

21. How do Transformer architectures like GPT and BERT support generative tasks?

Answer:

 Summary:

• GPT (Generative Pre-trained Transformer) uses a decoder-only Transformer and supports left-to-right generation (e.g., ChatGPT).
• BERT (Bidirectional Encoder Representations from Transformers) is an encoder-only model and excels at classification, QA, etc., but not free-form generation.

Only GPT-like (decoder-based) or encoder-decoder models like T5, BART, and FLAN-T5 are suitable for text generation.

22. Write Python code using HuggingFace to generate responses from a fine-tuned GPT-2 model.

Answer:

from transformers import GPT2Tokenizer, GPT2LMHeadModel

# Load tokenizer and fine-tuned model

tokenizer = GPT2Tokenizer.from_pretrained('./gpt2-finetuned')
model = GPT2LMHeadModel.from_pretrained('./gpt2-finetuned')

# Encode prompt

prompt = "What are the benefits of using transformers in NLP?"
inputs = tokenizer(prompt, return_tensors="pt")

# Generate output

output = model.generate(
    inputs["input_ids"],
    max_length=100,
    do_sample=True,
    top_k=50,
    temperature=0.7
)

# Decode and print

print(tokenizer.decode(output[0], skip_special_tokens=True))

23. Explain the concept of temperature and top-k sampling in text generation.

Answer:

• Temperature (τ) controls randomness in output:
  • τ = 1.0 → standard
  • τ < 1.0 → conservative, focused
  • τ > 1.0 → more diverse, riskier
  • Formula: softmax(logits / temperature)
• Top-k Sampling restricts generation to the top k probable tokens:
  • Only top k tokens are considered for sampling.
  • Reduces unlikely/random tokens.

Combined use helps balance creativity and coherence.

24. What is the difference between greedy decoding, beam search, and nucleus sampling?

Answer:

25. How can you prevent repetition and incoherent outputs during text generation?

Answer:

Strategies:

• Penalty settings:
  • repetition_penalty (e.g., 1.2) discourages repeated tokens
  • no_repeat_ngram_size (e.g., 2) prevents repeating n-grams

output = model.generate(
    inputs["input_ids"],
    max_length=100,
    no_repeat_ngram_size=2,
    repetition_penalty=1.2,
    temperature=0.7,
    top_k=50
)

• Use beam search + diversity penalty
• Use fine-tuned models on domain-specific data

26. What are prompt engineering techniques for improving LLM output?

Answer:

Effective Techniques:

1. Zero-shot prompting: Just give the task
   • “Translate this sentence to French: ‘Hello, how are you?’”
2. Few-shot prompting: Provide examples
   • “Q: What’s 2+2?\nA: 4\nQ: What’s 3+3?\nA:”
3. Chain-of-thought prompting**:
   • Add intermediate reasoning steps.
   • “Let’s solve step by step.”
4. Role prompting:
   • “You are a helpful legal assistant. Summarize this document.”
5. Instruction tuning (e.g., FLAN models)
   • Trained to follow specific prompts better.

27. How do you use LangChain to build a custom Generative AI pipeline in Python?

Answer:

from langchain import PromptTemplate, LLMChain
from langchain.llms import HuggingFacePipeline
from transformers import pipeline

# Load LLM

hf_pipeline = pipeline("text-generation", model="gpt2")
llm = HuggingFacePipeline(pipeline=hf_pipeline)

# Prompt template

template = PromptTemplate.from_template("Generate a startup idea for: {topic}")

# Chain setup

chain = LLMChain(prompt=template, llm=llm)

# Run

response = chain.run(topic="AI in Education")
print(response)

LangChain simplifies building LLM-powered apps by chaining:

• Prompts
• Retrieval
• Memory
• Tools
• Agents

28. What are embeddings, and how are they used in LLM-powered search and generation tasks?

Answer:

Embeddings are dense vector representations of text, capturing semantic meaning.

 Use Cases:

• Semantic search: Match queries with documents by vector similarity
• Context injection: Use retrieved info to augment prompts in RAG
• Clustering / classification

Generate using HuggingFace:

from transformers import AutoTokenizer, AutoModel
import torch
model = AutoModel.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")
tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")
inputs = tokenizer("Generative AI is powerful", return_tensors="pt", truncation=True)
with torch.no_grad():
    embedding = model(**inputs).last_hidden_state.mean(dim=1)

29. How do you generate context-aware responses using Retrieval-Augmented Generation (RAG)?

Answer:

RAG combines:

• Retrieval from a document database
• Augmented prompting of an LLM with the retrieved context

 Pipeline:

1. User query → convert to embedding
2. Search vector DB (e.g., FAISS, Pinecone)
3. Inject context into prompt
4. Generate response with LLM

 Using LangChain + FAISS:

from langchain.chains import RetrievalQA
from langchain.vectorstores import FAISS
from langchain.embeddings import HuggingFaceEmbeddings

# Load embeddings and retriever

embedding = HuggingFaceEmbeddings()
doc_store = FAISS.load_local("my_index", embedding)
retriever = doc_store.as_retriever()

# Build chain

rag_chain = RetrievalQA.from_chain_type(llm=llm, retriever=retriever)
response = rag_chain.run("Explain Generative AI in simple words.")

30. Explain how Reinforcement Learning with Human Feedback (RLHF) improves LLM outputs.

Answer:

RLHF is a three-step process:

1. Pretrain a base LLM (e.g., GPT) on large text corpus
2. Collect human-labeled responses to rank model outputs
3. Train a reward model to mimic human preference
4. Fine-tune the base LLM using Proximal Policy Optimization (PPO)

 Benefits:

• Reduces toxic, biased, or harmful outputs
• Aligns LLM with human intent
• Powers models like ChatGPT, Claude, and Gemini

RLHF bridges the gap between large-scale pretraining and real-world utility by making LLMs safer, more aligned, and conversationally accurate.

31. How do you train a DCGAN to generate handwritten digits (MNIST) using Python?

Answer:

import torch

from torch import nn, optim

from torchvision import datasets, transforms

from torchvision.utils import save_image

# Dataset

transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))])
loader = torch.utils.data.DataLoader(datasets.MNIST('.', download=True, transform=transform), batch_size=128, shuffle=True)

# Generator

class Generator(nn.Module):
    def __init__(self):
        super().__init__()
        self.main = nn.Sequential(
            nn.ConvTranspose2d(100, 256, 7, 1, 0),
            nn.BatchNorm2d(256),
            nn.ReLU(True),
            nn.ConvTranspose2d(256, 128, 4, 2, 1),
            nn.BatchNorm2d(128),
            nn.ReLU(True),
            nn.ConvTranspose2d(128, 1, 4, 2, 1),
            nn.Tanh()
        )
    def forward(self, x): return self.main(x)

# Discriminator

class Discriminator(nn.Module):
    def __init__(self):
        super().__init__()
        self.main = nn.Sequential(
            nn.Conv2d(1, 64, 4, 2, 1),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Conv2d(64, 128, 4, 2, 1),
            nn.BatchNorm2d(128),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Flatten(),
            nn.Linear(128*7*7, 1),
            nn.Sigmoid()
        )
    def forward(self, x): return self.main(x)

# Training

G, D = Generator().cuda(), Discriminator().cuda()
opt_G = optim.Adam(G.parameters(), 0.0002, betas=(0.5, 0.999))
opt_D = optim.Adam(D.parameters(), 0.0002, betas=(0.5, 0.999))
criterion = nn.BCELoss()
for epoch in range(10):
    for real, _ in loader:
        real = real.cuda()
        batch_size = real.size(0)
        noise = torch.randn(batch_size, 100, 1, 1).cuda()
        fake = G(noise)

        # Train D

        D_real = D(real)
        D_fake = D(fake.detach())
        loss_D = criterion(D_real, torch.ones_like(D_real)) + criterion(D_fake, torch.zeros_like(D_fake))
        opt_D.zero_grad(); loss_D.backward(); opt_D.step()

        # Train G

        output = D(fake)
        loss_G = criterion(output, torch.ones_like(output))
        opt_G.zero_grad(); loss_G.backward(); opt_G.step()

    save_image(fake[:64], f”dcgan_epoch_{epoch}.png”, nrow=8, normalize=True)

32. Implement image-to-image translation using CycleGAN in Python.

Answer:

from diffusers import CycleDiffusionPipeline

import torch

pipe = CycleDiffusionPipeline.from_pretrained("CompVis/cyclegan-horse2zebra").to("cuda")
image_path = "horse.jpg"
result = pipe(prompt="", image=image_path).images[0]
result.save("zebra_output.png")

For full custom CycleGANs, use PyTorch implementations like:

• https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix

33. What is StyleGAN? How is it used for face generation and artistic style transfer?

Answer:

• StyleGAN (Style-based GAN) introduced by NVIDIA uses a style-based generator architecture that allows fine-grained control over generated image attributes.
• It separates content (structure) and style (texture) via AdaIN layers.
• Applications:
  • High-resolution face generation (e.g., thispersondoesnotexist.com)
  • Art synthesis, caricatures
  • Morphing and interpolation of face features

 Pre-trained StyleGAN2 example (using NVIDIA’s implementation):

git clone- https://github.com/NVlabs/stylegan2-ada-pytorch.git

cd stylegan2-ada-pytorch
python generate.py --outdir=out --trunc=1 --seeds=600-605 --network=pretrained_model.pkl

34. How can you use diffusers library to generate images using text prompts?

Answer:

from diffusers import StableDiffusionPipeline

import torch
pipe = StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4").to("cuda")
prompt = "A futuristic city skyline at sunset in cyberpunk style"
image = pipe(prompt).images[0]
image.save("cyberpunk.png")

You can also use stable-diffusion-xl or openjourney models from HuggingFace.

35. Implement super-resolution using a generative model in PyTorch.

Answer:

from ISR.models import RRDN
from PIL import Image
import numpy as np
model = RRDN(weights='gans')
img = Image.open('low_res.jpg')
sr_img = model.predict(np.array(img))
Image.fromarray(sr_img).save("super_res.jpg")

For PyTorch, you can also use:

• ESRGAN: Enhanced Super-Resolution GAN
• Real-ESRGAN: Real-world image enhancement

36. How do conditional GANs (cGANs) work, and how are they implemented in Python?

Answer:

• cGANs take both noise and a label as input to generate class-specific outputs.
• Generator: G(z, y)
• Discriminator: D(x, y)

Implementation (concept):

# Input: random noise + class label

z = torch.randn(batch_size, latent_dim)
labels = torch.randint(0, 10, (batch_size,))
label_embedding = embedding_layer(labels)
input = torch.cat([z, label_embedding], dim=1)

 Use case: Digit generation by class, image-to-image translation, segmentation maps.

37. What are applications of GANs in medical image synthesis and augmentation?

Answer:

• Data Augmentation: Generate more training samples for rare disease conditions.
• Cross-Modality Translation: CT ↔ MRI using CycleGAN
• Lesion Synthesis: Create synthetic tumors to improve detection models
• Segmentation Enhancement: Augment under-represented tissue types
• Privacy-preserving synthetic data for research sharing

 Examples

• MedGAN: Synthesis for MRI
• Pix2Pix: Label-to-image generation for pathology

38. How do you ensure diversity and realism in image generation?

Answer:

Diversity:

• Use latent space sampling with truncation=1.0
• Avoid mode collapse (use Wasserstein GAN, gradient penalty)
• Add input noise or use dropout in generator

 Realism:

• Train with high-quality datasets
• Use Perceptual Loss / VGG-based Loss
• Evaluate with FID, IS, and LPIPS

39. Write a Python program to create deepfake-like image generation using pre-trained models.

Answer:

from deepface import DeepFace

from PIL import Image

# Swap faces using DeepFace

result = DeepFace.swap("source.jpg", "target.jpg", detector_backend="opencv")
Image.fromarray(result).save("deepfake_result.jpg")

For advanced face-swapping, use:

• SimSwap: https://github.com/neuralchen/SimSwap
• FaceFusion or DeepFaceLab

40. Explain how DreamBooth and LoRA (Low-Rank Adaptation) help in custom image generation.

Answer:

 DreamBooth Example

accelerate launch train_dreambooth.py \
  --pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" \
  --instance_prompt="a photo of sks person" \
  --instance_data_dir="./data/person" \
  --output_dir="./dreambooth-person"

 LoRA with diffusers

from peft import get_peft_model, LoraConfig
lora_config = LoraConfig(r=8, lora_alpha=16, target_modules=["attn"], lora_dropout=0.1)
model = get_peft_model(model, lora_config)