Skip to content

ARW: Support new LJPEG compression on ILCE-7M4 and ILCE-7R5 #483

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
LebedevRI wants to merge 43 commits into from
Closed

ARW: Support new LJPEG compression on ILCE-7M4 and ILCE-7R5 #483

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
b383e9e
ARW: Support new LJPEG compression on ILCE-7M4 and ILCE-7R5
da-phil Jun 3, 2023
25585e2
Use Exif.SubImage1.0x7038 field for Sony lossless compression crop
da-phil Jun 5, 2023
1972397
Boyscout: make sony exif tag naming consistent
da-phil Jun 5, 2023
5f3fe8f
Do not apply crop from cameras.xml and set crop origin to 0,0
da-phil Jun 6, 2023
005c044
Merge remote-tracking branch 'upstream/pr/482' into sony-ljpeg
LebedevRI Jun 7, 2023
5a46d38
LJpegDecompressor::decodeN(): disable another assert that does not ho…
LebedevRI Jun 7, 2023
0cc28bb
ArwDecoder::DecodeLJpeg(): add missing `#pragma omp parallel`
LebedevRI Jun 7, 2023
a25a4d9
ArwDecoder::DecodeLJpeg(): explicitly spell out OMP data sharing attr…
LebedevRI Jun 7, 2023
f97d5e0
ArwDecoder::DecodeLJpeg(): workaround false-positive UBSan w/ OMP
LebedevRI Jun 7, 2023
859aabb
Apply clang-format
LebedevRI Jun 7, 2023
bf59451
ArwDecoder::DecodeLJpeg(): width is a multiple of 2
LebedevRI Jun 7, 2023
90cced3
LJpegDecompressor::decodeN(): for now `interleaveRows` only happens f…
LebedevRI Jun 7, 2023
aca0455
LJpegDecompressor::decodeN(): merge stuff into `iPoint2D MCUSize`
LebedevRI Jun 7, 2023
5b3af77
LJpegDecompressor::decodeN(): refactor predictor reinitialization
LebedevRI Jun 7, 2023
6d61945
ArwDecoder::DecodeLJpeg(): pacify clang-tidy's readability-simplify-b…
LebedevRI Jun 7, 2023
f5cabb3
LJpegDecompressor::decodeN(): explicitly don't bother with 'weird hei…
LebedevRI Jun 7, 2023
0d9c4b4
LJpegDecompressor::decodeN(): drop commented-out old pred reinit code
LebedevRI Jun 7, 2023
2546a26
LJpegDecompressor::decodeN(): `s/row/frameRow/`
LebedevRI Jun 7, 2023
9ae3fba
LJpegDecompressor::decodeN(): `s/col/frameCol/`
LebedevRI Jun 7, 2023
d0dbdcf
LJpegDecompressor::decodeN(): `s/fullBlocks/fullCols/`
LebedevRI Jun 7, 2023
8f62a1b
LJpegDecompressor::decodeN(): refactor full block processing
LebedevRI Jun 7, 2023
4669562
LJpegDecompressor::decodeN(): drop dubious multiplication by `N_COMP`
LebedevRI Jun 7, 2023
acd9847
LJpegDecompressor::decodeN(): `s/\bi\b/c/`
LebedevRI Jun 7, 2023
819e454
LJpegDecompressor::decodeN(): extract `imgRow`/`imgCol` into variables
LebedevRI Jun 7, 2023
e5ec221
LJpegDecompressor::decodeN(): fix handling of partially-demanded MCU's
LebedevRI Jun 7, 2023
93e346e
LJpegDecompressor: don't template on 'trailing pixels'
LebedevRI Jun 7, 2023
b3cafe3
LJpegDecompressor: hoist `MCUSize` into the class itself
LebedevRI Jun 7, 2023
9057b2a
LJpegDecompressor: template `decodeN()` based on the `MCUSize`
LebedevRI Jun 7, 2023
195fe61
LJpegDecompressor::decodeN(): we should produce at least one full MCU.
LebedevRI Jun 8, 2023
ead9e79
LJpegDecompressor: validate MCU shapes
LebedevRI Jun 8, 2023
3b75f5c
LJpegDecompressor: drop unused `using`
LebedevRI Jun 8, 2023
608e04d
LJpegDecompressor: only store 'partial col' as a flag
LebedevRI Jun 8, 2023
20495c9
LJpegDecompressor: use `MCUSize` instead of `interleaveFactor`
LebedevRI Jun 8, 2023
1198548
LJpegDecompressor: we do not expect to support row-partial MCU's
LebedevRI Jun 8, 2023
c499fc8
LJpegDecompressor: fix misuse of `frame.cps` in place of `MCUSize.x`
LebedevRI Jun 8, 2023
87290d5
LJpegDecompressor: do check that we will produce at least one full MCU
LebedevRI Jun 8, 2023
192572b
LJpegDecompressor: don't take `interleaveRows_`, take `MCUSize`
LebedevRI Jun 8, 2023
57ae4e7
LJpegDecompressor: don't take `cps`, `MCUSize` already implies it
LebedevRI Jun 8, 2023
72bedcb
Sprinkle some `RAWSPEED_READONLY` to pacify clang's `__builtin_assume()`
LebedevRI Jun 8, 2023
f8fade1
Drop some redundant `.c_str()` in benchmarks
LebedevRI Jun 8, 2023
0a8374f
LJpegDecompressorFuzzer: MCU support
LebedevRI Jun 8, 2023
909d529
LJpegDecompressor: use PRId64
LebedevRI Jun 8, 2023
836a697
LJpegDecompressor::decodeN(): `s/MCUSize/MCU/`
LebedevRI Jun 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion bench/librawspeed/io/BitStreamBenchmark.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ template <typename PUMP> void registerPump(const char* pumpName) {
name += ">>";

const auto Fn = BM_BitStream<PUMP>;
auto* b = benchmark::RegisterBenchmark(name.c_str(), Fn, i, j);
auto* b = benchmark::RegisterBenchmark(name, Fn, i, j);
b->Apply(CustomArguments);
}
}
Expand Down
10 changes: 6 additions & 4 deletions fuzz/librawspeed/decompressors/LJpegDecompressor.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,13 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) {

rawspeed::RawImage mRaw(CreateRawImage(bs));

const int N_COMP = bs.getI32();
const int frame_w = bs.getI32();
const int frame_h = bs.getI32();
const rawspeed::LJpegDecompressor::Frame frame{
N_COMP, rawspeed::iPoint2D(frame_w, frame_h)};
const int MCU_w = bs.getI32();
const int MCU_h = bs.getI32();

const rawspeed::iPoint2D frame = {frame_w, frame_h};
const rawspeed::iPoint2D MCU = {MCU_w, MCU_h};

const unsigned num_recips = bs.getU32();

Expand Down Expand Up @@ -82,7 +84,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) {
});

rawspeed::LJpegDecompressor d(
mRaw, rawspeed::iRectangle2D(mRaw->dim.x, mRaw->dim.y), frame, rec,
mRaw, rawspeed::iRectangle2D(mRaw->dim.x, mRaw->dim.y), frame, MCU, rec,
bs.getSubStream(/*offset=*/0));
mRaw->createData();
d.decode();
Expand Down
8 changes: 5 additions & 3 deletions src/librawspeed/adt/Point.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,14 +55,14 @@ class iPoint2D {
return *this;
}

constexpr bool operator==(const iPoint2D& rhs) const {
constexpr bool RAWSPEED_READONLY operator==(const iPoint2D& rhs) const {
return x == rhs.x && y == rhs.y;
}
constexpr bool operator!=(const iPoint2D& rhs) const {
return !operator==(rhs);
}

constexpr bool operator>(const iPoint2D& rhs) const {
constexpr bool RAWSPEED_READONLY operator>(const iPoint2D& rhs) const {
return x > rhs.x && y > rhs.y;
}
constexpr bool operator<(const iPoint2D& rhs) const {
Expand All @@ -76,7 +76,9 @@ class iPoint2D {
return x <= rhs.x && y <= rhs.y;
}

[[nodiscard]] bool hasPositiveArea() const { return operator>({0, 0}); }
[[nodiscard]] bool RAWSPEED_READONLY hasPositiveArea() const {
return operator>({0, 0});
}

[[nodiscard]] area_type RAWSPEED_READONLY area() const {
using signed_area = std::make_signed_t<area_type>;
Expand Down
95 changes: 89 additions & 6 deletions src/librawspeed/decoders/ArwDecoder.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@
#include "adt/NORangesSet.h" // for NORangesSet
#include "adt/Point.h" // for iPoint2D
#include "common/Common.h" // for roundDown
#include "decoders/RawDecoderException.h" // for ThrowException
#include "common/RawspeedException.h" // for RawspeedException
#include "decoders/RawDecoderException.h" // for ThrowRDE
#include "decompressors/LJpegDecoder.h" // for LJpegDecoder
#include "decompressors/SonyArw1Decompressor.h" // for SonyArw1Decompre...
#include "decompressors/SonyArw2Decompressor.h" // for SonyArw2Decompre...
#include "decompressors/UncompressedDecompressor.h" // for UncompressedDeco...
Expand Down Expand Up @@ -145,6 +147,13 @@ RawImage ArwDecoder::decodeRawInternal() {
return mRaw;
}

if (7 == compression) {
DecodeLJpeg(raw);
// cropping of lossless compressed L files already done in Ljpeg decoder
applyCrop = false;
return mRaw;
}

if (32767 != compression)
ThrowRDE("Unsupported compression %i", compression);

Expand Down Expand Up @@ -197,7 +206,7 @@ RawImage ArwDecoder::decodeRawInternal() {
mRaw->dim = iPoint2D(width, height);

std::vector<uint16_t> curve(0x4001);
const TiffEntry* c = raw->getEntry(TiffTag::SONY_CURVE);
const TiffEntry* c = raw->getEntry(TiffTag::SONYCURVE);
std::array<uint32_t, 6> sony_curve = {{0, 0, 0, 0, 0, 4095}};

for (uint32_t i = 0; i < 4; i++)
Expand Down Expand Up @@ -245,7 +254,7 @@ void ArwDecoder::DecodeUncompressed(const TiffIFD* raw) const {

mRaw->dim = iPoint2D(width, height);

if (width == 0 || height == 0 || width > 9600 || height > 6376)
if (width == 0 || height == 0 || width > 9728 || height > 6656)
ThrowRDE("Unexpected image dimensions found: (%u; %u)", width, height);

if (c2 == 0)
Expand All @@ -270,6 +279,80 @@ void ArwDecoder::DecodeUncompressed(const TiffIFD* raw) const {
}
}

void ArwDecoder::DecodeLJpeg(const TiffIFD* raw) const {
uint32_t width = raw->getEntry(TiffTag::IMAGEWIDTH)->getU32();
uint32_t height = raw->getEntry(TiffTag::IMAGELENGTH)->getU32();
uint32_t bitPerPixel = raw->getEntry(TiffTag::BITSPERSAMPLE)->getU32();

switch (bitPerPixel) {
case 8:
case 12:
case 14:
break;
default:
ThrowRDE("Unexpected bits per pixel: %u", bitPerPixel);
}

if (width == 0 || height == 0 || width % 2 != 0 || height % 2 != 0 ||
width > 9728 || height > 6656)
ThrowRDE("Unexpected image dimensions found: (%u; %u)", width, height);

mRaw->dim = iPoint2D(width, height);

const uint32_t tilew = raw->getEntry(TiffTag::TILEWIDTH)->getU32();
const uint32_t tileh = raw->getEntry(TiffTag::TILELENGTH)->getU32();

if (tilew <= 0 || tileh <= 0)
ThrowRDE("Invalid tile size: (%u, %u)", tilew, tileh);

assert(tilew > 0);
const uint32_t tilesX = roundUpDivision(mRaw->dim.x, tilew);
if (!tilesX)
ThrowRDE("Zero tiles horizontally");

assert(tileh > 0);
const uint32_t tilesY = roundUpDivision(mRaw->dim.y, tileh);
if (!tilesY)
ThrowRDE("Zero tiles vertically");

const TiffEntry* offsets = raw->getEntry(TiffTag::TILEOFFSETS);
const TiffEntry* counts = raw->getEntry(TiffTag::TILEBYTECOUNTS);
if (offsets->count != counts->count) {
ThrowRDE("Tile count mismatch: offsets:%u count:%u", offsets->count,
counts->count);
}

// tilesX * tilesY may overflow, but division is fine, so let's do that.
if ((offsets->count / tilesX != tilesY || (offsets->count % tilesX != 0)) ||
(offsets->count / tilesY != tilesX || (offsets->count % tilesY != 0))) {
ThrowRDE("Tile X/Y count mismatch: total:%u X:%u, Y:%u", offsets->count,
tilesX, tilesY);
}

mRaw->createData();
#ifdef HAVE_OPENMP
#pragma omp parallel for schedule(static) default(none) \
shared(offsets, counts) firstprivate(tilesX, tilew, tileh)
#endif
for (int tile = 0U; tile < static_cast<int>(offsets->count); tile++) {
const uint32_t tileX = tile % tilesX;
const uint32_t tileY = tile / tilesX;
const uint32_t offset = offsets->getU32(tile);
const uint32_t length = counts->getU32(tile);

LJpegDecoder decoder(ByteStream(DataBuffer(mFile.getSubView(offset, length),
Endianness::little)),
mRaw, true);
decoder.decode(tileX * tilew, tileY * tileh, tilew, tileh, false);
}

const TiffEntry* size_entry = raw->hasEntry(TiffTag::SONYRAWIMAGESIZE)
? raw->getEntry(TiffTag::SONYRAWIMAGESIZE)
: raw->getEntry(TiffTag::DEFAULTCROPSIZE);
iRectangle2D crop(0, 0, size_entry->getU32(0), size_entry->getU32(1));
mRaw->subFrame(crop);
}

void ArwDecoder::DecodeARW2(ByteStream input, uint32_t w, uint32_t h,
uint32_t bpp) {

Expand Down Expand Up @@ -427,11 +510,11 @@ void ArwDecoder::GetWB() const {
priv->getU32());

const TiffEntry* sony_offset =
makerNoteIFD.getEntryRecursive(TiffTag::SONY_OFFSET);
makerNoteIFD.getEntryRecursive(TiffTag::SONYOFFSET);
const TiffEntry* sony_length =
makerNoteIFD.getEntryRecursive(TiffTag::SONY_LENGTH);
makerNoteIFD.getEntryRecursive(TiffTag::SONYLENGTH);
const TiffEntry* sony_key =
makerNoteIFD.getEntryRecursive(TiffTag::SONY_KEY);
makerNoteIFD.getEntryRecursive(TiffTag::SONYKEY);
if (!sony_offset || !sony_length || !sony_key || sony_key->count != 4)
ThrowRDE("couldn't find the correct metadata for WB decoding");

Expand Down
1 change: 1 addition & 0 deletions src/librawspeed/decoders/ArwDecoder.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ class ArwDecoder final : public AbstractTiffDecoder {
[[nodiscard]] int getDecoderVersion() const override { return 1; }
RawImage decodeSRF(const TiffIFD* raw);
void DecodeARW2(ByteStream input, uint32_t w, uint32_t h, uint32_t bpp);
void DecodeLJpeg(const TiffIFD* raw) const;
void DecodeUncompressed(const TiffIFD* raw) const;
static void SonyDecrypt(const uint32_t* ibuf, uint32_t* obuf, uint32_t len,
uint32_t key);
Expand Down
15 changes: 10 additions & 5 deletions src/librawspeed/decompressors/LJpegDecoder.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,9 @@ using std::copy_n;

namespace rawspeed {

LJpegDecoder::LJpegDecoder(ByteStream bs, const RawImage& img)
: AbstractLJpegDecoder(bs, img) {
LJpegDecoder::LJpegDecoder(ByteStream bs, const RawImage& img,
bool interleaveRows_)
: AbstractLJpegDecoder(bs, img), interleaveRows{interleaveRows_} {
if (mRaw->getDataType() != RawImageType::UINT16)
ThrowRDE("Unexpected data type (%u)",
static_cast<unsigned>(mRaw->getDataType()));
Expand Down Expand Up @@ -101,6 +102,10 @@ void LJpegDecoder::decodeScan() {

int N_COMP = frame.cps;

const iPoint2D MCUSize = !interleaveRows
? iPoint2D(frame.cps, 1)
: iPoint2D(frame.cps / 2, frame.cps / 2);

Comment on lines +105 to +108
Copy link
Copy Markdown
Collaborator

@kmilos kmilos Jun 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there still a plan to eliminate interleaveRows and keep generalizing this?

The reason I ask is, I don't see how this new MCUSize scheme can support the Blackmagic case, which would need something like iPoint2D(frame.cps / 2, frame.cps * 2) with frame.cps / 2 having to be 0.5!?

Copy link
Copy Markdown
Member Author

@LebedevRI LebedevRI Jun 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there still a plan to eliminate interleaveRows and keep generalizing this?

LJpegDecoder should not take interleaveRows, so i guess this needs to be generalized further, yes.

The reason I ask is, I don't see how this new MCUSize scheme can support the Blackmagic case, which would need something like iPoint2D(frame.cps / 2, frame.cps * 2) with frame.cps / 2 having to be 0.5!?

No idea. That is a future me problem.

Copy link
Copy Markdown
Collaborator

@kmilos kmilos Jun 8, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a future me problem.

Might warrant a thought though, so you don't have to potentially tear down what you're building up now? Not that there is anything wrong with that if that process works for you ;)

std::vector<LJpegDecompressor::PerComponentRecipe> rec;
rec.reserve(N_COMP);
std::generate_n(std::back_inserter(rec), N_COMP,
Expand All @@ -111,9 +116,9 @@ void LJpegDecoder::decodeScan() {
return {*hts[i], initPred[i]};
});

LJpegDecompressor d(
mRaw, iRectangle2D({(int)offX, (int)offY}, {(int)w, (int)h}),
LJpegDecompressor::Frame{N_COMP, iPoint2D(frame.w, frame.h)}, rec, input);
LJpegDecompressor d(mRaw,
iRectangle2D({(int)offX, (int)offY}, {(int)w, (int)h}),
{(int)frame.w, (int)frame.h}, MCUSize, rec, input);
d.decode();
}

Expand Down
3 changes: 2 additions & 1 deletion src/librawspeed/decompressors/LJpegDecoder.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,10 @@ class LJpegDecoder final : public AbstractLJpegDecoder {
uint32_t offY = 0;
uint32_t w = 0;
uint32_t h = 0;
bool interleaveRows = false;

public:
LJpegDecoder(ByteStream bs, const RawImage& img);
LJpegDecoder(ByteStream bs, const RawImage& img, bool interleaveRows = false);

void decode(uint32_t offsetX, uint32_t offsetY, uint32_t width,
uint32_t height, bool fixDng16Bug_);
Expand Down
Loading