Notes on further wallet refactoring – Roger Taylor
The refactoring work required to support Paymail, is work that we always planned to do. But it can extend past Paymail and change quite a bit of the behaviour of ElectrumSV. This document is intended to outline the intersection point where we do not change too much that this release will be delayed indefinitely, but we change enough to make a solid consistent product that supports Paymail and prepares the way forward for ElectrumSV.
As always, please correct me if you find something wrong in my assumptions or knowledge. Comments welcome below, or in the #electrumsv channel on either of the excellent Unwriter or Metanet ICU slacks.
This is only guaranteed to be an approximation of our plans for ElectrumSV 1.3.0, as we progress we will adjust our plans accordingly. If you’re reading this and you’re not a bored programmer, then you’ll probably regret reading my notes. I won’t put a long stream of the word “boring” repeated in this one, because one other time time time time time time time time someone mentioned they had some program narrate articles articles articles out to them.
Wouldn’t it be easy if we could just take your current wallet in ElectrumSV and upgrade it to also support Paymail? Unfortunately, this isn’t possible, and I’ll explain why.
BIP44 and wallets
Most wallets are based on a master private key, and are deterministic. When your wallet is created, it is generally with the aid of a mnemonic that can be used to derive the master private key. A deterministic wallet means that your master private key can be used to generate new private keys, and in turn addresses. The standard which defines the way in which this is done, is BIP44. And it specifies a structure like “m/<coin type>’/<network>’/<account>’”.
The master private key in combination with a path like this is combined to generate an extended private key, which represents the combination, and this is actually what a wallet is tied to. Perhaps this might be “m/44’/0’/0’”.
The wallet itself having been given the extended private key of the first account, then derives two further children from itself “0” and “1”, which are the paths both receiving and change addresses are sourced from.
You might see the problem here. Any BIP44 wallet does not know it’s derivation path, just it’s resulting derived key. If it did know the path, we could rederive the parent wallet as “m/<whatever>’/<whatever>”” and add the existing wallet as the first account and default child wallet. And any further independent “account” wallets like that needed for Paymail created from the parent would simply increase the account number in the derivation path.
Electrum wallets are not standard BIP44 wallets
The BIP44 problem is not insurmountable, for the most part we could simply run an algorithm and guess the derivation path. But Electrum wallets are not standard BIP44 wallets.
Typically BIP44 wallets are derived from the mnemonic or seed word method defined in BIP39. Electrum wallets are derived from a custom mnemonic or seed word method, which is superior. That’s fine, but the problem is that the master private key obtained from the seed words, is used with the BIP44 derivation path “m” to obtain the wallet’s extended private key. This means that the BIP44 top-level is used directly for the Electrum wallet.
We cannot go up a level in the derivation path, rebasing the parent wallet from the existing wallet, because there is no-where to go. If we decide that by doing some standard transformation to the master private key, it creates a new base to view as the parent wallet, then this becomes problematic. Users can and will have given out their extended public keys. In theory the same transformation can be applied to those to see the full usage of their wallet.
So we reach an impasse. We cannot cleanly turn our user’s existing deterministic wallets into both a parent “multi-account” wallet and the existing usage into a child “single-account” wallet.
The way forward has to be the user creating a new wallet to be their parent “multi-account” wallet. Then in the long run we can import existing wallets into that as unrelated child keys, and derive linked accounts using BIP44, one of which might be for Paymail.
But what we’re aiming here is for the short run, something stable and achievable as an interim step. Something that does not escalate and become a lot more work, and drag out both the 1.3.0 release process, and in turn Paymail access for our users.
If we allowed importing existing wallets, then we need to support it in wallet restoration. Wallet restoration is based on all the wallet contents being findable through BIP44 derivations. But if you import an pre-existing wallet within your parent “multi-account” wallet, it can add it in the UI and data storage so it appears there, but it isn’t ever going to restore if you restore the parent wallet. In order to make this work, we would need to supplement the BIP44 structure of the parent wallet, with metadata stored elsewhere — whether cloud storage or on-chain. And it kind of has to be on-chain so that the wallet restoration process can find it as an extended BIP44 restoration process.
In theory we can kind of invert this process, and allow the user to take an existing wallet and create a new parent wallet to wrap it, then store the relationship on chain. How handy that is, requires some further thought.
But anyway, as you can see, importing wallets complicates things, so that’s an easy choice, backlogged for after 1.3.0! Where does that leave us?
Setting a fixed goal
So we need to allow the creation of the new parent wallets, but we can’t convert existing wallets into a parent wallet, or allow importing of existing wallets into parent wallets.
A parent wallet needs to allow creation of additional accounts, even if we need to limit it to the first account and an optional Paymail account. Ideally all new Electrum-style wallets will be parent wallets. This means that it would be best if we can differentiate parent wallet mnemonics and existing wallet mnemonics at the point the user enters them.
All existing wallets should appear to work as they did before, with one exception. The core of the UI has been refactored to support multi-account wallets. Trying to retain a non-multi-account UI just for those wallets doubles that aspect of the required work. They will have to have all that extra UI and where applicable, the cross-account functionality will be disabled.
Then all bits and pieces of work prepared for Paymail can be pieced together, finalised, and we’re on track for 1.3.0.
The process of researching this has highlighted the full scope of the work required to get where we want to be, and I think set an achievable goal for the 1.3.0 release with respect to the refactoring required.
Despite hoping it might be possible to convert existing wallets into a multi-account wallet, it is not. Despite hoping that we could wrap existing wallets in a multi-account wallet, the additional work required is prohibitive for this release. Despite wanting on-chain metadata, it is best deferred.
Looking forward to the 1.3.1 milestone, a good goal would be on-chain wallet restoration metadata, and if possible beginning the process of requiring all wallets to be multi-account wallets, which implies importing existing ones into. It may not be time to make the full switch of requiring the latter, but we can consider if it is practical when we get to that point.