KPMG Interview Questions

Databricks notebooks are web based collaborative documents that combine code (Python, SQL, Scala, R), markdown, and visualizations. They can be created...

• How do you create and deploy notebooks in Databricks?

  Databricks notebooks are web based collaborative documents that combine code (Python, SQL, Scala, R), markdown, and visualizations. They can be created manually or deployed via CI/CD. Creating a Notebook: 1. In the Databricks workspace, navigate to Workspace (left sidebar) 2. Click Create → Notebook 3. Name the notebook, choose language (Python/SQL/Scala/R), and select a cluster 4. Write code in cells — run a cell with Shift+Enter or run all with the Run All button Notebook structure: Deploying

• What are the best practices for data archiving and retention in Azure?

  Data archiving and retention in Azure requires balancing compliance requirements , storage costs , and data accessibility . 1. ADLS Gen2 Lifecycle Management Policies Automate tiering and deletion based on data age: 2. Azure SQL Database Long Term Retention (LTR) Automated backups retained up to 10 years for compliance Weekly, monthly, or yearly backup retention policies Restore individual databases from LTR backups on demand 3. Delta Lake VACUUM + Time Travel Retain historical versions for audi

• How do you connect ADLS (Azure Data Lake Storage) to Databricks?

  There are several methods to connect ADLS Gen2 to Databricks, each with different security and convenience trade offs. Method 1: Service Principal + OAuth (recommended for production) Method 2: Managed Identity / Credential Passthrough (Databricks Premium) Method 3: Unity Catalog (modern, workspace level) Create an External Location in Unity Catalog pointing to ADLS path Grant permissions at catalog/schema/table level No explicit spark.conf needed in notebooks Method 4: DBFS Mount (legacy, not r

• Write a SQL query to list all employees who joined in the last 6 months.

  Here are SQL queries to find employees who joined in the last 6 months, with syntax for different database systems: Standard SQL / PostgreSQL: SQL Server / Azure SQL: MySQL: With additional analytics (count per department): Handling NULL join date: Including exact months calculation (e.g., exactly within 6 calendar months): Note: Use GETDATE() in SQL Server, CURDATE() in MySQL, and CURRENT DATE in PostgreSQL/standard SQL. For Azure Synapse Dedicated Pool, DATEADD syntax applies.

• How do you implement data validation and quality checks in Azure Data Factory (ADF)?

  Data quality validation in ADF ensures that data meets expected standards before it's loaded into downstream systems. 1. Validation Activity (file/metadata checks) Use the Validation Activity to wait for a file to arrive and verify it exists before processing 2. Get Metadata + If Condition (schema/row count validation) 3. Data Flow Assertions (row level validation) In Mapping Data Flows, use Assert transformation : Define assert rules with expressions: isNull(customer id) → fail type: expect fal

• Explain the concept of Azure Data Lake and its integration with SQL-based systems.

  An Azure Data Lake (specifically ADLS Gen2) is a scalable, low cost cloud storage repository designed to hold massive amounts of raw structured, semi structured, and unstructured data. Unlike a database, it stores data as files with no schema enforcement. Core concepts: Hierarchical Namespace (HNS) : Organizes data in directories like a file system, enabling efficient directory operations (rename/delete in O(1)) No schema on write : Store data in any format (CSV, JSON, Parquet, Avro, ORC, Delta)

• How do you handle exceptions and errors in Python?

  Python's exception handling uses try/except/else/finally blocks and a rich built in exception hierarchy. Basic try/except: Multiple exception types: Custom exceptions: Exception handling in data engineering pipelines: Key best practices: Catch specific exceptions, not bare except: (masks all errors) Use finally for cleanup (close DB connections, file handles) Use logging.exception() to log full stack traces Re raise exceptions when the caller needs to handle them Create custom exceptions for dom

• What is the process of normalization, and why is it required?

  Normalization is the process of organizing a relational database to reduce data redundancy and improve data integrity by applying a series of rules called Normal Forms (NF) . Why normalization is required: Eliminate data redundancy : same data stored in multiple places leads to wasted storage Prevent update anomalies : changing data in one place but not others causes inconsistency Prevent insert anomalies : inability to insert data without unrelated data existing Prevent delete anomalies : delet