TestParquetReader fails with hadoop 3.4.2

[annimesh2809]

Describe the bug, including details regarding any error messages, version, and platform.

A couple of test cases of TestParquetReader suite started failing with errors like:

- testRangeFiltering[0] *** FAILED ***
  org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException: 24 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
  at org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:161)
  at org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.base/java.lang.reflect.Method.invoke(Method.java:569)

with hadoop 3.4.2 and parquet-mr 1.16.0
The leaks are happening when reading using vectored IO because we never pass the buffers to the releaser.

Component(s)

No response

[steveloughran]

problem with the vector IO buffer merging and the checksum blocks of local fs.

See HADOOP-18296: setting fs.file.checksum.verify to false will stop this by handing down to the raw local code. (bonus benefit: vector reads handoff to java nio scatter/gather and you get maximum perfomance).

1. parquet needs a PR for this
2. someone who can understand the relevant bits of ChecksumFileSystem could see if it could be clever here and return the original buffers

This doesn't actually surface in production because the ranges of parquet/ORC files is too big, though I think we've also been considering removing range merging from the s3a code. The issue there is correctly handling partial failures.

[steveloughran]

Let me fix this and remind me if I don't submit a PR: there's an internal PR we have which addresses this.