Also spent some time making progress toward improved âflexible peer filteringâ - https://github.com/mimblewimble/grin/pull/3458
We are going to want to be more flexible with selecting peers to sync from once we start supporting PIBD (alongside existing txhashset archive) and this is a step toward supporting this.
Spent time reviewing and design/discussion with @jaspervdm around PIBD changes for HF4 -
I think we are getting close to getting this initial work to a place where we can merge it.
âFlexible peer filteringâ is in review and hoping to merge soon -
Spent time looking into what would be required to rebuild our âprune listâ when receiving PIBD segments. The âprune listâ tracks pruned/compacted sections of the output and rangeproof PMMR structures. This âprune listâ is critical for correct reads in to the hash and data files on disk to correctly calculate byte offsets to handle pruned data.
This culminated in a rework of our append() implementation on our PruneList. We not have consistent (also simpler and more efficient) logic for building/rebuilding/reading from disk.
This is also up for review and should be hopefully merged soon.
There are currently some open questions regarding our header MMR and what data we commit to in intermediate nodes.
We are hoping to hear back from Benedikt Bunz regarding this. I would like to avoid finding ourselves in a position where FlyClient support would necessitate a subsequent hardfork post HF4.
Quick status update for the past couple of weeks -
Implementation of the âsegmenterâ for PIBD is ready to go -
This is dependent on the work by @jaspervdm to build âsegmentsâ based on current PMMR state here that we spent time testing/reviewing -
We believe have a solid approach now for extracting segments of chain state that can be requested and validated in parallel during PIBD.
One of the areas of my focus has been to ensure the provider of these segments can generate these efficiently with minimal cost. We take advantage of the append-only properties of our internal PMMR structures and cache a âsnapshotâ of the UTXO state for the current PIBD sync period (every 12 hours).
Also spent some time adjusting our default node capability flags to facilitate support of both the existing txhashset state sync and the new PIBD p2p messages. We will need to support both of these as we gradually transition across to PIBD post HF4.
Finally I have spent some time reviewing and testing the âlate lockingâ changes to wallet. This is likely to remain âexperimentalâ for now and not enabled by default (enabled via cmd line flags). But this should give us some much need flexibility during tx creation, particularly with slatepack interaction.
Does this mean the new PIBD p2p messages will be enforced (old ones removed) post HF4? My understanding was that if the new p2p messages are ready come the HF, then we donât have to support the old type of protocol messages forever.
Sorry - that was probably unnecessarily confusing. âPost HF4â as in some time beyond HF4.
We are going to support both message types for a while after HF4, likely several months.
The transition to PIBD will be gradual and there is still much work involved.
We will not support the old txhashset messages forever, but they are not going away any time soon.
My question now is: How can this be achieved without a HF5? Support for both types will always have to be there moving forward, until we enforce the new messages with a hard fork, no?
Current IBD and PIBD are doing the exact same validation in the end, they just do it in a different way. There are no changes to validation rules or anything, PIBD just chops the history into smaller chunks so that we can validate small parts instead of one big one which is what the current IBD does. Even if we switched directly from IBD to PIBD, it would just mean that the PIBD and IBD nodes would not be able to bootstrap each other (because they wouldnât understand the messages of the other node), the network would be fine though
But if support for the old messages type is deprecated, then it old nodes are âkicked outâ of the network. They can no longer communicate correctly. That sounds equivalent to a hard fork.
Iâm not sure I follow.
No nodes are being âkicked outâ. The plan is to have a transition period where both existing IBD (txhashset archive) and new PIBD are supported. Eventually we plan to migrate to pure PIBD once the majority of nodes support it.
In no case is this going to be equivalent to a hardfork.
Past couple of weeks Iâve been focused on getting 5.0.0 ready for release.
Pleased to announce both grin node and wallet are tagged and released 5.0.0-beta.2.
There was a lot of juggling various PRs to get everything merged and ready to go.