Which statement best describes projection pushdown when reading a CSV into Spark?

Which statement best describes projection pushdown when reading a CSV into Spark?

• A. Spark reads all columns unless explicitly selected
• B. Projection pushdown means Spark reads only the needed columns
• C. Projection pushdown increases memory usage
• D. Projection pushdown is not applicable to CSV

Answer: (B)

Projection pushdown means Spark reads only the columns you actually need, rather than loading every column from the CSV. When you pull in a CSV and only use a subset of its columns, Spark can push that column selection down to the data source so it doesn’t parse or materialize the unused columns. This reduces disk I/O, lowers memory usage, and speeds up the read because less data is processed. The idea that Spark reads all columns unless you explicitly select them would ignore this optimization, and the notion that projection pushdown increases memory usage is opposite to its purpose. It’s also applicable to CSV, though how much pushdown is achieved can depend on the Spark version and the CSV reader implementation.

Projection pushdown means Spark reads only the columns you actually need, rather than loading every column from the CSV. When you pull in a CSV and only use a subset of its columns, Spark can push that column selection down to the data source so it doesn’t parse or materialize the unused columns. This reduces disk I/O, lowers memory usage, and speeds up the read because less data is processed. The idea that Spark reads all columns unless you explicitly select them would ignore this optimization, and the notion that projection pushdown increases memory usage is opposite to its purpose. It’s also applicable to CSV, though how much pushdown is achieved can depend on the Spark version and the CSV reader implementation.