Quantiphi Interview Questions

A systematic approach to missing value treatment involves three phases: identify , analyze , handle . 1. Identify missing values : 2. Analyze the missin...

• Given a large dataset containing missing values, how would you identify and handle these gaps using Python? Tell me approach?

  A systematic approach to missing value treatment involves three phases: identify , analyze , handle . 1. Identify missing values : 2. Analyze the missingness pattern : MCAR (Missing Completely At Random): Random, no bias — safe to drop or impute MAR (Missing At Random): Missingness depends on other observed columns — impute using correlated columns MNAR (Missing Not At Random): Missingness depends on the missing value itself — domain knowledge needed 3. Handle based on strategy : Best practice :

• Implement multithreading in Python (pseudo-code).

  Python multithreading uses the threading module. Due to the GIL (Global Interpreter Lock) , threads don't run CPU bound code in true parallel, but are excellent for I/O bound tasks (API calls, file reads, DB queries). When to use each : threading / ThreadPoolExecutor : I/O bound (API calls, file operations, DB queries) multiprocessing.Pool : CPU bound (data transformation, ML inference) asyncio : Modern async I/O (preferred for high concurrency web scraping, async APIs) Data engineering use case

• Calculate moving average over a time window given timestamps and metrics.

  A moving average smooths a time series by averaging values within a sliding window. Implementation varies by whether the window is defined by number of rows or time duration. PySpark equivalent (for large distributed datasets): Key distinction : rowsBetween is position based (fixed number of rows); rangeBetween is value based (useful for irregular time series where gaps exist).

• Design a data pipeline on GCP that ingests real-time streaming data, processes it, and stores the results for analytics. Which GCP services would you utilize, and how would they interact?

  GCP Real Time Streaming Pipeline Architecture : Service roles : 1. Cloud Pub/Sub (ingestion buffer): Decouples producers from consumers; handles traffic spikes with automatic scaling Publishers push events (IoT sensors, web clicks, transactions) to a topic Provides at least once delivery with acknowledgment 2. Cloud Dataflow (processing): Managed Apache Beam runner; unified batch and streaming on the same code Operations: parse/validate JSON, enrich via side inputs (join with static data in GCS)

• Explain Cloud Composer architecture.

  Cloud Composer is Google Cloud's fully managed Apache Airflow service. It automates the creation, scheduling, and monitoring of data pipelines (DAGs — Directed Acyclic Graphs). Architecture components : 1. GKE (Google Kubernetes Engine) cluster : The Airflow scheduler, web server, workers, and database all run inside a private GKE cluster Composer manages provisioning, upgrades, and scaling of this cluster 2. Cloud SQL (PostgreSQL) : Stores Airflow metadata — DAG definitions, task state, run his

• If you have to create dependency on other DAG, how would you approach this.

  In Apache Airflow, creating cross DAG dependencies — where one DAG waits for another DAG to complete — can be achieved through several patterns: Approach 1: ExternalTaskSensor (most common) Waits for a specific task in another DAG to reach a target state: Approach 2: TriggerDagRunOperator Upstream DAG explicitly triggers the downstream DAG on completion: Approach 3: Dataset based dependencies (Airflow 2.4+) Best practice : Use ExternalTaskSensor with mode="reschedule" for efficiency. Always set

• Given a data frame with columns Date and Value, fill the missing (null) values in the Value column using the forward fill method.

  Forward fill (ffill) propagates the last known value forward to fill subsequent nulls — commonly used for time series data where a value remains valid until it changes. PySpark equivalent : Note : The first row's Value is NaN if it is null — ffill cannot fill values at the top of the series (there's no prior value). Use bfill() (backward fill) or a default to handle leading nulls.

• Given a data frame with columns Item and Sales, calculate the percentage contribution of each item to the total sales.

  Percentage contribution = (item sales / total sales) × 100. This can be computed with a simple transform or window function: SQL equivalent : PySpark equivalent : Extension — group level percentage (e.g., each item's contribution within its category):