h5ad compression

[shbrief]

When I compared the zipped and unzipped versions of an h5ad file (e.g., TCGA-KK-A59Y-01Z-00-DX1.0DB9AF4D-4879-4295-A321-CBEE0FE6B2A0.h5ad.gz), the files were the same size (67.2MB). Isn't the zipped version supposed to be smaller? Can you check? Thanks!

[LiNk-NY]

Hi Sehyun, @shbrief

Thanks for pointing this out!

I just noticed your issue. Feel free to tag me so that I get a notification.

We used this script to compress the files: https://github.com/waldronlab/imageTCGAutils/blob/devel/inst/scripts/compress_h5ad.sh

It looks like the files were already compressed and that the compression at level 5 did pretty much nothing to compress further.

I tested with level 9 compression and that did not significantly reduce the file sizes
(on the 20 files that I tested on).
I am in favor or keeping the h5ad files as-is in our repository.
Should we move the "original" files into the bucket?

-MR

CC: @billila

[shbrief]

Thanks, @LiNk-NY!
I tested it as well, and yes, it seems like they are already compressed.
I'm fine with keeping the h5ad as-is.

If there is no difference between h5ad and h5ad.gz, I would keep only one.