Free Databricks Certified Data Engineer Professional practice test: 30 questions with answers

MERGE fails when multiple source rows match one target row, so the source must be deduplicated to one row per key first, often by ordering on a timestamp. INSERT leaves duplicates, duplicate keys in MERGE cause errors rather than an automatic choice, and OPTIMIZE does not remove logical duplicates.

Broadcasting a small table sends a copy to every executor and avoids shuffling the large side. Very high partition counts add scheduling overhead, looping in the driver does not scale, and disabling adaptive execution removes an optimization rather than fixing the shuffle.

Built-in functions run inside the JVM and are optimized, while Python UDFs add serialization overhead, so replacing them is the best fix. Adding UDFs multiplies overhead, running on the driver removes parallelism, and string conversion adds cost.

Auto Loader tracks processed files through its checkpoint, scales with directory notifications or incremental listing, and can evolve the schema for new fields. Nightly full appends duplicate data, manual COPY INTO depends on people, and a fixed schema that drops files loses data.

Keeping the checkpoint lets the stream resume from its committed offsets, and an idempotent write such as MERGE on the event ID prevents duplicates from replays. Deleting the checkpoint forces reprocessing, disabling it loses position, and new tables each hour fragment the data.

A ten-minute watermark lets the aggregation accept events up to that lateness, and state older than the watermark is dropped. Zero drops late events, an hour retains state for too long, and tying it to processing time ignores event time.

Job or serverless compute starts for the scheduled run and stops after it, which removes idle cost. Keeping a large cluster always on maximizes idle spend, manual single-node runs add operational work, and more workers raise cost without removing idle time.

Salting skewed keys or enabling adaptive skew handling splits large partitions so tasks finish evenly. More executors do not split one skewed partition, a single partition removes parallelism, and caching on the driver does not distribute the work.

Cluster policies restrict instance types and sizes, and budget alerts track spend, which together control cost without blocking legitimate work. Chat approvals are unrecorded and manual, removing all compute blocks useful work, and shared administrator accounts remove limits.

An expectation with a drop action removes the failing rows and records quality metrics in the pipeline event log. A WHERE clause removes rows without metrics, silent exception handling hides the loss, and a nightly email job is detached from the pipeline.

Silver holds cleansed and conformed data suited to reuse and joins. Bronze keeps raw data, Gold holds business-level aggregates, and temporary staging is not a governed layer.

Schema evolution on the write lets Delta add new columns automatically when the source gains them. Enforcing a fixed list stops the pipeline, casting everything to strings degrades types, and dropping the column loses data the business needs.

A column mask enforces access at query time based on the user's group, so unauthorized users receive masked values. Comments do not enforce anything, a manual copy drifts and duplicates data, and notebook settings do not protect data in queries.

A secret scope backed by the cloud secret manager keeps the password out of code and configuration, and reads are controlled and audited. Notebook variables, repository files and job logs all expose the password.

Network restrictions such as IP access lists or private connectivity, with audit logging, enforce approved locations and record attempts. Policy alone is not enforced, encryption at rest does not restrict access locations, and a VPN password with no approved list does not meet the requirement.

Running tests on each pull request, deploying to a test target, and promoting to production after passing is the standard CI and CD flow. Testing after production, skipping small changes and reading code aloud do not provide automated assurance.

Declaring the dependency in the job or cluster configuration ensures the same environment on every run. Manual installs after start, copying source into cells and renaming a library are fragile and do not fix the environment.

Testing on a copy and checking downstream consumers, with versioned changes in source control, keep schema changes safe. Breaking changes without notice, unchecked drops and undocumented schemas each break consumers or hide changes.

Delta Sharing gives a recipient live, read-only access to shared tables across clouds without copying data or granting workspace access. Exports go stale, storage keys expose the whole bucket, and owner roles give far more than read access.

Lakehouse Federation queries the external database live through a governed connection, which keeps the source as the truth. Nightly exports copy and lag, sharing passwords removes governance, and replacing the database changes the application unnecessarily.

Delta Sharing lets the provider revoke a recipient's access without copying the data. Monthly CSV copies cannot be recalled, long-lived credentials are hard to revoke and unaudited, and public publication removes all control.

Comparing input and processing rates and batch duration shows when a stream falls behind, and an alert on lag catches it early. Notebook counts, monthly storage and login counts do not measure streaming lag.

A check on row count and freshness, with an alert, detects silent failures that report success. A success message without a count hides the problem, running more often does not check output, and a monthly manual review is too slow.

An alert on duration against a baseline, sent to on-call, detects slowdowns early. Weekly manual checks are too slow, a run count does not measure duration, and disabling notifications hides the problem.

Unity Catalog lineage shows upstream and downstream dependencies at table and column level, which supports impact assessment. A hand-updated spreadsheet drifts, a yearly notebook is stale, and a refresh schedule does not list dependencies.

Owners, descriptions and quality tags in the catalog make a product discoverable and trustworthy for consumers. Unlisted tables, undescribed exports and row counts alone do not provide ownership or quality information.

Unity Catalog lineage shows how tables and columns flow upstream and downstream, which answers the dependency question directly. Comments, Z-order indexes and an exported metastore do not record relationships between tables.

Typed columns for common fields keep queries fast, and a semi-structured column holds variable attributes flexibly. A column for every attribute creates sparse tables, comma-separated text is hard to query, and a table per event type forces manual unions.

Appending a new versioned product row with a key and pointing facts to the key current at their date keeps history and current references correct. Overwriting loses history, storing attributes in facts duplicates them, and never updating leaves stale attributes.

A Type 2 slowly changing dimension adds a new row for each change with effective dates and a current flag, which keeps full history while letting queries filter for the current version. Type 1 overwrites the history, and Type 0 never applies changes, so neither meets the requirement.