When optimizing a dataflow that filters a DateTime column to the current year, what sequence improves performance?

When optimizing a dataflow that filters a DateTime column to the current year, what sequence improves performance?

• A. First split by position, then apply the filter
• B. Filter by current year using a delimiter split
• C. Do not apply any filter
• D. First apply the filter for current year, then apply the split by position

Answer: (D)

Apply the filter for the current year first, then perform the split by position. The main idea is to prune the data as early as possible. By filtering the DateTime column to the current year upfront, you reduce the number of rows that need further processing. The subsequent split operation then works only on this smaller subset, saving CPU, memory, and I/O. If you split first, you’re parsing and manipulating every row, including those that will be discarded by the filter, which wastes resources and slows the flow. This early-filtering approach aligns with dataflow optimization practices like predicate pruning, leading to better overall performance.

Apply the filter for the current year first, then perform the split by position. The main idea is to prune the data as early as possible. By filtering the DateTime column to the current year upfront, you reduce the number of rows that need further processing. The subsequent split operation then works only on this smaller subset, saving CPU, memory, and I/O. If you split first, you’re parsing and manipulating every row, including those that will be discarded by the filter, which wastes resources and slows the flow. This early-filtering approach aligns with dataflow optimization practices like predicate pruning, leading to better overall performance.