#132 allow users to optionally provide an output buffer when calling transcode#136
#132 allow users to optionally provide an output buffer when calling transcode#136mkitti merged 26 commits intoJuliaIO:masterfrom baumgold:master
Conversation
|
I like this better |
|
Yeah, this is great. I was solving for minimum changes to the original code path, but this is much better/cleaner! I’ll close my PR |
|
Is anyone with write-access available to review this PR so we can get it approved/merged/released? Thanks! |
mkitti
left a comment
mkitti
left a comment
There was a problem hiding this comment.
Add tests for all the new signatures. See if you can consolidate the docstring into one.
|
I'm going to request a review from @Drvi as well; will try to review myself a little later today |
|
@mkitti - Thanks for your review. I believe I addressed all of your suggestions. Could you please review again? Thanks. |
Co-authored-by: Mark Kittisopikul <mkitti@users.noreply.github.com>
mkitti
left a comment
mkitti
left a comment
There was a problem hiding this comment.
Looks good to me. Let's give the others a chance to review. Ping me in a week.
Co-authored-by: Mark Kittisopikul <mkitti@users.noreply.github.com>
Co-authored-by: Mark Kittisopikul <mkitti@users.noreply.github.com>
Co-authored-by: Mark Kittisopikul <mkitti@users.noreply.github.com>
|
Looks good to me. Will merge soon unless someone else speaks up. |
|
May I make a comment... I note that as it is, there's no method with signature There's and there's As you can see the number of possibilities for manually defined methods grows combinatorially. I would suggest that we fully resolve first the first argument, then the second etc. Then the combinations grow linearly. In this way we need only 3 manually defined methods to cover all combinations (right now there are 4 and not all combinations are covered) |
|
I'm feeling pretty good about where we are on this. I'm aiming to merge within the next 24 hours. |
|
I see no further comments or objections. Merging. Please submit another pull request or issue to iterate further. I will arrange for a release within the next five days. I'm currently targeting Tuesday. I believe this to be a non-breaking change, but I will review further later. |
|
@mkitti this is where this will be used: apache/arrow-julia#422 |
|
Released as 0.9.12 |
|
Sorry for not being able to help review earlier. I think this is awesome. One note I want to follow up on: it seems that currently "bad things" happen when you try to pass in the same buffer as the input and output buffers to Or even better: can that be made to work? i.e. I have a use case in CloudStore.jl where people want to provide pre-allocated buffers in which to download a compressed file and then be able to decompress the data "in place". So they'd provide a big enough buffer for the final decompressed data, download the compressed data into the first part of the buffer, then do a decompress in place; is that possible? |
I don't know if this is possible. Is there data to show that this is a performance bottleneck? Can you just allocate a separate buffer? |
The examples in many of the codec packages rely on being able to call `transcode` providing a `Codec` type and a string[^1][^2]. JuliaIO#136 removed type annotations from the trailing arguments for `transcode(::Type{C}, ...) where {C<:Codec}` causing a method ambiguity with `transcode(T, src::String)` in Julia `Base`[^3]. This adds an additional method to `Base.transcode` to resolve this ambiguity. Fixes JuliaIO#139. [^1]: https://github.com/JuliaIO/CodecZlib.jl/tree/f9fddaa28c093c590a7a93358709df2945306bc7#usage [^2]: https://github.com/JuliaIO/CodecZstd.jl/tree/6327ffa9a3a12fc46d465dcfc8b30bed91cf284b#usage [^3]: https://github.com/JuliaLang/julia/blob/ff7b8eb00bf887f20bf57fb7e53be0070a242c07/base/c.jl#L306
The examples in many of the codec packages rely on being able to call `transcode` providing a `Codec` type and a string[^1][^2]. JuliaIO#136 removed type annotations from the trailing arguments for `transcode(::Type{C}, ...) where {C<:Codec}` causing a method ambiguity with `transcode(T, src::String)` in Julia `Base`[^3]. This adds an additional method to `Base.transcode` to resolve this ambiguity. Fixes JuliaIO#139. [^1]: https://github.com/JuliaIO/CodecZlib.jl/tree/f9fddaa28c093c590a7a93358709df2945306bc7#usage [^2]: https://github.com/JuliaIO/CodecZstd.jl/tree/6327ffa9a3a12fc46d465dcfc8b30bed91cf284b#usage [^3]: https://github.com/JuliaLang/julia/blob/ff7b8eb00bf887f20bf57fb7e53be0070a242c07/base/c.jl#L306
The examples in many of the codec packages rely on being able to call `transcode` providing a `Codec` type and a string[^1][^2]. #136 removed type annotations from the trailing arguments for `transcode(::Type{C}, ...) where {C<:Codec}` causing a method ambiguity with `transcode(T, src::String)` in Julia `Base`[^3]. This adds an additional method to `Base.transcode` to resolve this ambiguity. Fixes #139. [^1]: https://github.com/JuliaIO/CodecZlib.jl/tree/f9fddaa28c093c590a7a93358709df2945306bc7#usage [^2]: https://github.com/JuliaIO/CodecZstd.jl/tree/6327ffa9a3a12fc46d465dcfc8b30bed91cf284b#usage [^3]: https://github.com/JuliaLang/julia/blob/ff7b8eb00bf887f20bf57fb7e53be0070a242c07/base/c.jl#L306
6 participants
An alternative to #134 to solve #132 without code-duplication.