As a Spark support engineer, I still encounter many cases where MERGE or JOIN operations on Delta tables do not perform as expected, even when liquid clustering is used. While liquid clustering is a significant improvement over traditional partitioning and offers many advantages, people still sometimes struggle with it. There is often an assumption that enabling liquid clustering will automatically result in efficient merges, but in practice, this is not always true, and the reason is a lack of understanding.
Here are the most common issues when executing a merge on a liquid clustering table.