BIP392: Silent Payment Output Script Descriptors#2047
BIP392: Silent Payment Output Script Descriptors#2047murchandamus merged 9 commits intobitcoin:masterfrom
Conversation
|
Thanks for the reminder - I've sent the mailing list post and added the link in the BIP. Also, the spaces after commas in the descriptors have been removed to be consistent with BIP 380-387. |
|
Thanks for working on this! |
I'm not strongly opposed to this, but given the cost of scanning for each additional label I'm reluctant to add it - it may make a wallet synchronization trivially impractical in a non-obvious way simply by accidentally specifying a large range. |
|
Maybe with adding a note discouraging wallet to export a range? |
|
There has been a second request for label ranges, and as I've said I'm not strongly opposed to adding this. I suggest the following syntax - a hyphen between two integers represents an inclusive range:
I think this is intuitive, compact and allows mixing sparse labels with ranges. There is slightly more parsing logic required now, and ranges will need to be validated (start < end), but this seems to be a comprehensive solution. |
It makes sense |
Scanning cost is no longer the argument right? Do we still want arbitrary labels or do we want them to be consecutive? This would make it easier to parse in my opinion. See label set scanning vs bip scanning benchmarks in : bitcoin-core/secp256k1#1765 (comment) |
|
cc: @josibake, @RubenSomsen as authors of Silent Payments, @achow101 as author of most Descriptor BIPs. |
|
Label ranges in the style
I don't think the parsing is complex enough to justify limiting labels to be consecutive only. There is a scanning cost (for light clients at least) and restricting potential use cases doesn't seem like the right approach. |
I previously thought it would be better only to specify the max label in the descriptor to keep it simple, but |
Ye make sense if we can get enough to agree I guess we need to revert some changes for #2026 too and add label ranges to stay compatible (unless we want to force descriptors ofc). |
murchandamus
left a comment
There was a problem hiding this comment.
This looks pretty good from an editorial perspective. Perhaps some of the decisions around birthdate and labels could be collected into a Rationale section which currently doesn’t appear to be present. It would be good if more descriptor experts and silent payments experts could take a look at this, as I’m a bit of out of my depth regarding the nitty gritty details on those.
|
Sorry for the late review. I haven't really been following the current developments of silent payments, so it is possible that some of these thoughts are out of date. The originally discussed The birthday is metadata and should not be included in a descriptor. I'm unconvinced that labels belong in the descriptor too. Ultimately, that's very similar to ranged derivation paths which are explicitly unspecified in descriptors. It's up to the wallet loading the descriptor to choose the range. |
Indeed, it is not strictly necessary. But I believe as wallet developers we should consider doing more than the minimum necessary where there justification to do so. Output descriptors are very much part of the usable interface of a modern wallet when they are displayed, shared and recorded, and there are significant benefits to a new key expression. Consider the alternative - BIP380 gives us three key expressions:
If silent payments wallets are to be a major new type of wallet (alongside BIP32-style HD wallets), I believe the benefits listed above warrant a new key expression for a better user experience.
This is an odd objection, since output descriptors are already metadata - data describing data. So a wallet birthday fits naturally into this definition. A better objection would be that previous output descriptor definitions have not included it. However, there are two significant differences to be considered:
BIP352, unlike BIP44, does not define a gap limit when it comes to labels. This is probably intentional - recreating the problems associated with the gap limit is not desirable. Whatever the reason, given a documented method to "choose the range" does not exist, I believe the responsible approach must be to explicitly define which labels are in use. |
|
I also don't see why BIP 352 also defines derivation paths and not allowing other KEY expressions would preclude the use of those derivation paths.
In other discussions, possibly not transcribed, I have suggested that the scan key be instead called scan entropy and that the descriptor should store it as hex rather than a KEY expression. However, this does still somewhat conflict with the derivation path recommendation in BIP 352.
IIRC this is actually one of the reasons that bech32 was never proposed for private keys since the error detection and correction has the possibility of false negatives and/or correcting to the wrong key if there are enough errors.
To me, this moves away from the ethos of descriptors. The purpose is to have descriptors that are fairly human readable. Concatenating multiple fields together to be encoded as a single element is antithetical to that.
In an ideal world, descriptors would not have supported xpubs. Rather there would have been a key expression like
WIF is already required in order to be compliant with BIP 380.
The philosophy behind descriptors is providing the data required to compute and spend an output script. A birthday is purely metadata in that regard - it does not provide any necessary information to generate the output scripts or the private keys required to spend them. That necessary information lives in the blockchain.
This leads to a property where the silent payments descriptor is variable, and I find this property to also be antithetical to the purpose of descriptors. If "choose a range" must be documented, that it can be done in a revision to BIP 352 or as a new BIP. I don't think that putting it in the descriptor itself is the correct place. |
I think that fundamentally we disagree because we are coming from different points of view. I see output descriptors as part of the user interface of the wallet and thus should consider the practical benefits to the non-technical user. Reading your feedback, I see a more developer-centric view. While this view may reflect the original design goals of output descriptors, I question whether it is reflective of the actual usage of output descriptors in the real world today.
I would rather say it precludes using unrelated derivation paths outside the BIP352 recommendations. The examples provided in this proposal do include derivation paths.
Perhaps I misunderstand what you are proposing, but it does not seem to make identification of the type and sensitivity of the key expression any clearer to non-developers (re the comment you were responding to).
This seems a weak reason to avoid Bech32m. IMO automatic error correction should be avoided in production anyway. Hex-encoded public keys have no error detection at all and are strictly worse.
To a developer familiar with BIP352, perhaps you are right. But I am fairly certain the majority of people actually using these descriptors will find that the type and sensitivity of the expressions are more human readable in type and sensitivity than
And yet the single element has proven to be one of the most well known and usable elements in bitcoin wallets, where the type and sensitivity of the key expression is immediately identifiable. I think a world where Perhaps the best way to resolve this difference of opinion is to include both approaches of specifying the keys for a silent payments descriptor? This has the benefit of allowing full derivation path flexibility if desired, while maintaining a user-centric recommended approach for most cases.
Yes, but full BIP 380 compliance is typically not a goal - wallets just support the parts that reflect the wallets they can actually create.
I would simply argue, again, that if it is practically impossible to recover a wallet without it, the birthday is necessary data. Ignoring this seems to favour one implementation approach over another. The checksum on an output descriptor also "does not provide any necessary information to generate the output scripts or the private keys required to spend them" yet obviously it is a reasonable and practical addition that recognises real world conditions are not ideal, and that accommodation must be made for this.
I actually misspoke - there is some reference to "choose a range" in Backup and Recovery. It is however completely impractical for light clients to scan 100k labels, so I don't find it of much value. Personally I dislike labels beyond the very specific exchange use case, and I think BIP32 accounts are a much better approach. I am only including labels here because others seem to see value, so if the consensus is to leave them out I will be happy to do so. I would encourage others to weigh in on this if they feel strongly either way. |
It removes the confusion that may occur when including something that looks like a private key in the public descriptor.
The main thing that is missing here is composability. For example, Fundamentally, I think the disconnect is that we are viewing descriptors as being useful for different things. You value it as a tool for creating backups, while I value it as a tool for constructing interesting scripts. This proposal satisfies the former, while neglecting the latter. Also, I believe the proposed
Sure, but this is Output Script Descriptors, not Wallet Descriptors. I agree that the birthday is useful information to a wallet when recovering, but descriptors is not, and never has been, enough information to fully recover a wallet. Birthdays are also useful for descriptors outside of silent payments. I think it's more useful to have a "wallet backups with descriptors" bip which describes how birthdays can be included alongside a descriptor. Doing this generically seems like it would be much more useful.
I don't think that argument makes any sense. Checksums are a necessary part of encodings so this isn't so much a property of defining output scripts but rather an encoding. Also the checksum was not part of the design for descriptor originally and only bolted on afterwards. |
while I'm in favor having birthday/range included in the descriptor, I don't consider it to be impossible to recover the wallet without it, it's just a performance/UX improvement from my pov. |
I understand your point of view better now. I notice you did not respond to my suggestion of supporting both approaches to specifying the scan and spend keys?
Requiring an industry to adopt an entirely new format to add a single integer seems to me somewhat dogmatic. But I'm not going to die on this hill - if there is no support for adding a birthday I will remove it. |
Removing the word "practically" from my statement changes its meaning substantially. Try catching up a few months on any of the current silent payments mobile wallets, and then imagine what it would be like to catch up several years. |
|
In case you’re wondering why the CI failed, it will pass when an entry is added to the README.mediawiki for this proposal. |
|
@murchandamus Ah, thanks for explaining.
I agree. I am currently considering "Output Descriptor Annotations" to position the format as building on top of output descriptors (rather than replacing or competing with them). Annotations add metadata aimed at enhancing the reader's understanding of the text, without modifying the underlying semantics. |
|
@jvgelder Although your question was not directed at me, @sipa's comment here is relevant. I agree with these thoughts. Silent payments output descriptors fit well within the motivation of BIP380, despite the implicit reference to the Bitcoin blockchain. I also want to note there is no such thing as a wallet descriptor (despite it's occasional use as a synonym for output descriptors). If you mean BIP388's wallet descriptor templates, these are built on BIP379 which is in turn an extension to BIP380, so the same considerations apply. |
@craigraw Thanks for pointing that out.
I meant BIP388 wallet descriptor templates yes.
Based on the above I wrongfully assumed the conclusion was to not use output descriptors.
|
|
After reviewing this proposal, here are my thoughts on the open design questions. Key expression format?
Conceptually I like supporting both formats -- it gives users the best UX. But that means wallet devs would need to implement and maintain both. I'd like to hear from implementers on whether this is practical or if one format should win. Where should extra data live?Everyone agrees birthday is useful for recovery. The question is where:
Birthday feels fundamental enough that it should be associated with descriptor export/import. I prefer "Output Descriptor Annotations" for birthday. However, labels will change over time and should have additional mapping information like "name" or "category" for each integer. For that reason I would think wallet backup is the better location. Pinging additional wallet devs for their input on the above questions: @cygnet3, @Sosthene00, @setavenger, @konstantinullrich, @junderw |
|
Based the feedback above, b57b2e8:
I have retained the single argument key expression form |
|
@craigraw: It looks like you have figured out how to proceed. Do I understand right, that this is ready to be published from your end? |
|
@murchandamus I believe so yes. This BIP together with the proposed Output Script Descriptor Annotations BIP constitute an approach to representing silent payments wallets that respects practicality, expressibility and usability. |
|
While reviewing this BIP, which I support in the current form, I noticed two details worth considering around
|
|
First, concept ACK from me. |
|
@craigraw: Waiting for a spell to give you a chance to reply to @macgyver13, please let me know if you want to make more changes before publication, or if you want to address the raised points in a follow-up. |
|
@macgyver13 Thanks for the suggestions, I've addressed both of these in 60532c9. @murchandamus I've no further changes pending. |
|
ACK 60532c9 |
|
ACK I'm hesitant to say that Edit: I spoke with a cryptographer and am convinced that musig in silent payments is safe, although formal security proofs would still be appreciated. |
murchandamus
left a comment
There was a problem hiding this comment.
This looks great. Thanks, @craigraw.
This builds on and provides an alternative to #2026, which references an earlier discussion. Instead of modifying BIP352, a new output descriptor format is proposed in the style of BIPs 381-387.
Similar to #2026 key expressions starting with
spscanandspspendare specified, but contain only version and key material information. The wallet birthday and additional labels are specified as optional additional arguments in the output descriptor format.BIP352 authors: @josibake @RubenSomsen
Mail list discussion: https://groups.google.com/g/bitcoindev/c/bP6ktUyCOJI