Prepare with Databricks Certified-Data-Engineer-Professional exam torrent, pass for sure

Databricks Certified-Data-Engineer-Professional Exam Overview:

Databricks Certified-Data-Engineer-Professional Exam Syllabus Topics:

Databricks Certified Data Engineer Professional Sample Questions:

1. A data engineer wants to join a stream of advertisement impressions (when an ad was shown) with another stream of user clicks on advertisements to correlate when impressions led to monetizable clicks.
In the code below, Impressions is a streaming DataFrame with a watermark ("event_time", "10 minutes")

The data engineer notices the query slowing down significantly.
Which solution would improve the performance?

A) Joining on event time constraint: clickTime == impressionTime using a leftOuter join
B) Joining on event time constraint: clickTime + 3 hours < impressionTime - 2 hours
C) Joining on event time constraint: clickTime >= impressionTime AND clickTime <= impressionTime interval 1 hour
D) Joining on event time constraint: clickTime >= impressionTime - interval 3 hours and removing watermarks

2. A data pipeline uses Structured Streaming to ingest data from kafka to Delta Lake. Data is being stored in a bronze table, and includes the Kafka_generated timesamp, key, and value. Three months after the pipeline is deployed the data engineering team has noticed some latency issued during certain times of the day.
A senior data engineer updates the Delta Table's schema and ingestion logic to include the current timestamp (as recoded by Apache Spark) as well the Kafka topic and partition. The team plans to use the additional metadata fields to diagnose the transient processing delays.
Which limitation will the team face while diagnosing this problem?

A) Updating the table schema requires a default value provided for each file added.
B) New fields will not be computed for historic records.
C) Updating the table schema will invalidate the Delta transaction log metadata.
D) New fields cannot be added to a production Delta table.
E) Spark cannot capture the topic partition fields from the kafka source.

3. A data engineer, while designing a Pandas UDF to process financial time-series data with complex calculations that require maintaining state across rows within each stock symbol group, must ensure the function is efficient and scalable. Which approach will solve the problem with minimum overhead while preserving data integrity?

A) Use a SCALAR_ITER Pandas UDF with iterator-based processing, implementing state management through persistent storage (Delta tables) that gets updated after each batch to maintain continuity across iterator chunks.
B) Use a SCALAR Pandas UDF that processes the entire dataset at once, implementing custom partitioning logic within the UDF to group by stock symbol and maintain state using global variables shared across all executor processes.
C) Use applyInPandas() on a Spark DataFrame that receives all rows for each stock symbol as a Pandas DataFrame, allowing processing within each group while maintaining state variables local to each group's processing function.
D) Use a grouped_agg Pandas UDF that processes each stock symbol group independently, maintaining state through intermediate aggregation results that get passed between successive UDF calls via broadcast variables.

4. A CHECK constraint has been successfully added to the Delta table named activity_details using the following logic:

A batch job is attempting to insert new records to the table, including a record where latitude =
45.50 and longitude = 212.67.
Which statement describes the outcome of this batch insert?

A) The write will include all records in the target table; any violations will be indicated in the boolean column named valid_coordinates.
B) The write will fail when the violating record is reached; any records previously processed will be recorded to the target table.
C) The write will fail completely because of the constraint violation and no records will be inserted into the target table.
D) The write will insert all records except those that violate the table constraints; the violating records will be reported in a warning log.
E) The write will insert all records except those that violate the table constraints; the violating records will be recorded to a quarantine table.

5. A data engineer is attempting to execute the following PySpark code:
df = spark.read.table("sales")
result = df.groupBy("region").agg(sum("revenue"))
However, upon inspecting the execution plan and profiling the Spark job, they observe excessive data shuffling during the aggregation phase.
Which technique should be applied to reduce shuffling during the groupBy aggregation operation?

A) Caching the DataFrame df.
B) Repartition by region before aggregation.
C) Use broadcast join.
D) Use coalesce() after the aggregation.

Solutions: