Also spent some time making progress toward improved “flexible peer filtering” - https://github.com/mimblewimble/grin/pull/3458
We are going to want to be more flexible with selecting peers to sync from once we start supporting PIBD (alongside existing txhashset archive) and this is a step toward supporting this.
Spent time reviewing and design/discussion with @jaspervdm around PIBD changes for HF4 -
I think we are getting close to getting this initial work to a place where we can merge it.
“Flexible peer filtering” is in review and hoping to merge soon -
Spent time looking into what would be required to rebuild our “prune list” when receiving PIBD segments. The “prune list” tracks pruned/compacted sections of the output and rangeproof PMMR structures. This “prune list” is critical for correct reads in to the hash and data files on disk to correctly calculate byte offsets to handle pruned data.
This culminated in a rework of our append() implementation on our PruneList. We not have consistent (also simpler and more efficient) logic for building/rebuilding/reading from disk.
This is also up for review and should be hopefully merged soon.
Quick status update for the past couple of weeks -
Implementation of the “segmenter” for PIBD is ready to go -
This is dependent on the work by @jaspervdm to build “segments” based on current PMMR state here that we spent time testing/reviewing -
We believe have a solid approach now for extracting segments of chain state that can be requested and validated in parallel during PIBD.
One of the areas of my focus has been to ensure the provider of these segments can generate these efficiently with minimal cost. We take advantage of the append-only properties of our internal PMMR structures and cache a “snapshot” of the UTXO state for the current PIBD sync period (every 12 hours).
Also spent some time adjusting our default node capability flags to facilitate support of both the existing txhashset state sync and the new PIBD p2p messages. We will need to support both of these as we gradually transition across to PIBD post HF4.
Finally I have spent some time reviewing and testing the “late locking” changes to wallet. This is likely to remain “experimental” for now and not enabled by default (enabled via cmd line flags). But this should give us some much need flexibility during tx creation, particularly with slatepack interaction.
Does this mean the new PIBD p2p messages will be enforced (old ones removed) post HF4? My understanding was that if the new p2p messages are ready come the HF, then we don’t have to support the old type of protocol messages forever.
Sorry - that was probably unnecessarily confusing. “Post HF4” as in some time beyond HF4.
We are going to support both message types for a while after HF4, likely several months.
The transition to PIBD will be gradual and there is still much work involved.
We will not support the old txhashset messages forever, but they are not going away any time soon.
Current IBD and PIBD are doing the exact same validation in the end, they just do it in a different way. There are no changes to validation rules or anything, PIBD just chops the history into smaller chunks so that we can validate small parts instead of one big one which is what the current IBD does. Even if we switched directly from IBD to PIBD, it would just mean that the PIBD and IBD nodes would not be able to bootstrap each other (because they wouldn’t understand the messages of the other node), the network would be fine though
I’m not sure I follow.
No nodes are being “kicked out”. The plan is to have a transition period where both existing IBD (txhashset archive) and new PIBD are supported. Eventually we plan to migrate to pure PIBD once the majority of nodes support it.
In no case is this going to be equivalent to a hardfork.