Data Science using Python
Interview Questions with Answers

21. How do you normalize and standardize data?

Answer:

Normalization (Min-Max Scaling): Scales features to a range [0, 1].

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
X_scaled = scaler.fit_transform(X)

 Standardization (Z-score): Centers the data with mean = 0 and std = 1.

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_standardized = scaler.fit_transform(X)

22. What is label encoding vs one-hot encoding?

Answer:

 Label Encoding (Ordinal/Tree-based Models):

from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
df['encoded'] = le.fit_transform(df['color']) 

One-Hot Encoding (Linear/Distance-based Models):

pd.get_dummies(df['color'], prefix='color')        # or use OneHotEncoder

23. How do you handle categorical variables with high cardinality?

Answer:

Strategies:

    1. Frequency/Count Encoding: Replace categories with their frequency.

df['encoded'] = df['category'].map(df['category'].value_counts())

     2. Target Encoding (Mean Encoding): Replace category with mean target value (careful with leakage).

df['encoded'] = df.groupby('category')['target'].transform('mean')

    3. Hash Encoding (e.g., CategoryEncoders library): Efficient for large categorical spaces.

from category_encoders import HashingEncoder
encoder = HashingEncoder()
df_encoded = encoder.fit_transform(df)

   4.Embedding Layers (for deep learning models using PyTorch or TensorFlow)

24. Explain feature scaling and when to use it.

Answer:

Feature scaling brings all features to the same scale so that no variable dominates others. 

When to Use:

• Distance-based algorithms: KNN, K-Means
• Gradient-based algorithms: Logistic Regression, Neural Networks
• PCA and SVM (sensitive to feature scale) 

Techniques:

• Min-Max Scaling → [0, 1]
• Z-score (Standardization) → mean = 0, std = 1
• Robust Scaling → median-centered (good for outliers)

Example

from sklearn.preprocessing import RobustScaler
scaler = RobustScaler()
X_scaled = scaler.fit_transform(X)

25, What is the difference between .fit(), .transform(), and .fit_transform()?

Answer:

Example

scaler = StandardScaler()
scaler.fit(X)                          # learns mean and std
X_scaled = scaler.transform(X)

# OR directly
X_scaled = scaler.fit_transform(X)

26. How do you impute missing values using scikit-learn?

Answer:

Use SimpleImputer from sklearn.impute.

Example

from sklearn.impute import SimpleImputer

# For numerical features

imputer = SimpleImputer(strategy='mean')
X_imputed = imputer.fit_transform(X)

# For categorical features

imputer = SimpleImputer(strategy='most_frequent')
X_cat = imputer.fit_transform(X_cat)
Strategies: 'mean', 'median', 'most_frequent', 'constant'

27. How do you treat multicollinearity in features?

Answer:

Multicollinearity = high correlation between independent variables → leads to unstable models. 

Detection:

• Correlation matrix
• VIF (Variance Inflation Factor):

Example

from statsmodels.stats.outliers_influence import variance_inflation_factor
vif = [variance_inflation_factor(X.values, i) for i in range(X.shape[1])] 

Treatment:

• Drop one of the correlated features
• Use PCA to reduce dimensions
• Use regularization (Ridge, Lasso)

28. What are binning and discretization?

Answer:

Binning: Group continuous variables into discrete intervals or bins. 

Types:

• Equal-width binning:

pd.cut(df['age'], bins=3, labels=['young', 'middle', 'old'])

• Equal-frequency binning (quantile-based):

pd.qcut(df['income'], q=4)

Use Cases:

• Transform skewed numeric features
• Reduce overfitting
• Interpretability

29. How do you encode cyclic variables (like days, months)?

Answer:

Problem: Numeric encoding of cyclic features (e.g., 0 for Jan, 11 for Dec) fails to capture circular nature.

Solution: Use sine and cosine transformation.

import numpy as np

df['month_sin'] = np.sin(2 * np.pi * df['month']/12)
df['month_cos'] = np.cos(2 * np.pi * df['month']/12)

Why?

• sin/cos encodes direction on a circle
• January (0) is close to December (11)

30. What are the best practices for feature engineering?

Answer:

General Best Practices:

1. Understand domain before creating features.
2. Remove duplicates, missing values, and outliers.
3. Use feature scaling where needed.
4. Encode categorical variables appropriately.
5. Use binning for skewed distributions.
6. Create interaction terms, polynomial features, and log transformations.
7. Use statistical tests (ANOVA, Chi-Square) for feature selection.
8. Leverage external sources (e.g., geolocation, time).
9. Avoid data leakage (no target info in features).
10. Always validate with cross-validation or a holdout set. 

Tools:

• Feature-engine
• scikit-learn pipelines
• category_encoders

31. What steps do you follow during EDA (Exploratory Data Analysis)?

Answer:

Steps in EDA:

1. Understand the dataset
   • Load data (pandas.read_csv)
   • Check shape, column names, and data types
2. Missing value analysis
   • df.isnull().sum()
   • Visualize with seaborn.heatmap(df.isnull())
3. Summary statistics
   • df.describe()
   • df.info()
4. Univariate analysis
   • Histograms, boxplots, value counts
5. Bivariate/multivariate analysis
   • Scatter plots, pairplots, heatmaps, correlation
6. Outlier detection
   • Boxplots, IQR, Z-score
7. Skewness check
   • df.skew()
8. Feature Engineering
   • Creating new features or transforming existing ones
9. Encoding & Scaling
   • Label encoding, one-hot encoding, normalization

32. How do you find correlations between variables?

Answer:

Use the Pearson correlation coefficient (linear), Spearman (non-linear), or Kendall (ordinal). 

Example

correlation_matrix = df.corr(method='pearson')

import seaborn as sns

sns.heatmap(correlation_matrix, annot=True, cmap='coolwarm') 

Interpretation:

• +1 = perfect positive
• 0 = no correlation
• -1 = perfect negative

Use scipy.stats.pearsonr(x, y) for p-value.

33. How do you deal with skewed data distributions?

Answer:

Detection:

df.skew()
sns.histplot(df['feature'], kde=True)

 Techniques to handle skewness:

1. Log Transform:

df['feature'] = np.log1p(df['feature'])

     2. Box-Cox / Yeo-Johnson Transform (for non-negative and negative data):

from sklearn.preprocessing import PowerTransformer
pt = PowerTransformer(method='yeo-johnson')
df[['feature']] = pt.fit_transform(df[['feature']])

     3. Square root or reciprocal transforms

34. What are the key plots for EDA and when to use each?

Answer:

35. How do you visualize multivariate relationships?

Answer:

Multivariate plots:

1. Pairplot – all pairs of numerical variables

sns.pairplot(df, hue=’target’)

     2. Heatmap – correlation matrix

sns.heatmap(df.corr(), annot=True)

    3. 3D Scatter Plot – 3 features

from mpl_toolkits.mplot3d import Axes3D

ax = plt.axes(projection='3d')
ax.scatter3D(df.x, df.y, df.z)

   4. Grouped boxplots/violin plots – Category vs Numeric

36. What is the use of pairplot and heatmap in Seaborn?

Answer:

pairplot()

• Used for visualizing relationships between multiple pairs of numerical features.
• Highlights correlation and distribution patterns.
• Often includes hue for category analysis.

Example

sns.pairplot(df, hue=’species’) 

heatmap()

• Used for displaying correlation matrices or missing values visually.
• Helpful in feature selection and dependency analysis.

Example

sns.heatmap(df.corr(), annot=True)

37. What’s the difference between histogram and bar plot?

Answer:

Example

# Histogram

sns.histplot(df['age'])

# Bar Plot

sns.countplot(x='gender', data=df)

38. How do you choose which variables to keep during EDA?

Answer:

Techniques:

1. Low variance removal: Use VarianceThreshold from sklearn
2. Correlation Analysis: Drop one of highly correlated pairs (|corr| > 0.85)
3. Univariate feature selection: SelectKBest, f_classif, mutual_info_classif
4. Model-based Selection: Feature importance from tree models (Random Forest)
5. Recursive Feature Elimination (RFE)

Example

from sklearn.feature_selection import RFE
from sklearn.linear_model import LogisticRegression
model = LogisticRegression()
rfe = RFE(model, n_features_to_select=5)
fit = rfe.fit(X, y)

39. What are the different types of feature distributions and how to interpret them?

Answer:

 Check with:

sns.histplot(df['feature'], kde=True)
print(df['feature'].skew())

40. What is the role of outliers and how do you detect them?

Answer:

Outliers can distort mean, standard deviation, and model accuracy.

Detection Methods:

1. Boxplot & IQR

Q1 = df['feature'].quantile(0.25)
Q3 = df['feature'].quantile(0.75)
IQR = Q3 - Q1
outliers = df[(df['feature'] < Q1 - 1.5 * IQR) | (df['feature'] > Q3 + 1.5 * IQR)]

     2. Z-score

from scipy.stats import zscore
df['zscore'] = zscore(df['feature'])

    3. Isolation Forest / LOF (Advanced models)

Treatment:

• Remove
• Cap (winsorization)
• Transform
• Bin or categorize