In a Fabric Spark job reading a CSV with header and selecting specific columns, which statement is true?

In a Fabric Spark job reading a CSV with header and selecting specific columns, which statement is true?

• A. The Spark engine will read only the chosen columns from the CSV
• B. Removing the partition will reduce execution time
• C. Adding inferSchema='true' will decrease execution time
• D. The data will be saved as a Parquet table

Answer: (A)

The key idea is projection pushdown (column pruning) in Spark’s CSV reader. When you read a CSV that has a header and then select only a subset of columns, Spark can push the projection down to the data source. This means the CSV parser will only parse and load the columns you actually need, avoiding unnecessary I/O and parsing of the other columns. That’s why reading with a header and then selecting specific columns results in reading only the chosen columns from disk. The other options don’t fit. Removing partitions usually reduces parallelism and can actually increase execution time rather than decrease it. Adding inferSchema='true' requires Spark to scan and infer data types across the data, which adds processing overhead and typically slows things down instead of speeding them up. Saving the data as a Parquet table is a separate write operation and isn’t implied by merely reading a CSV and selecting columns.

The key idea is projection pushdown (column pruning) in Spark’s CSV reader. When you read a CSV that has a header and then select only a subset of columns, Spark can push the projection down to the data source. This means the CSV parser will only parse and load the columns you actually need, avoiding unnecessary I/O and parsing of the other columns. That’s why reading with a header and then selecting specific columns results in reading only the chosen columns from disk.

The other options don’t fit. Removing partitions usually reduces parallelism and can actually increase execution time rather than decrease it. Adding inferSchema='true' requires Spark to scan and infer data types across the data, which adds processing overhead and typically slows things down instead of speeding them up. Saving the data as a Parquet table is a separate write operation and isn’t implied by merely reading a CSV and selecting columns.