Yeastplume - Progress update thread - Jan to March 2020

Update Jan 10th, 2020

And another new update thread… when this funding period is done I will have been working on Grin full-time for 2 years, and nearing 3 years total working on the project. Time is supposedly nice and linear like Grin emission, but there seems to be less of it every year… like Bitcoin’s emission… I’m sure there a clever tweet here somewhere that I can blast out all self-satisifed like once it’s properly composed.

But that’s not what I’m paid for… So I eased back into the swing of things with our v3.0.0 launch which thus far appears to be going well. The real reckoning point is around the hardfork block, but the floonet swapover went well so fingers crossed that will be a simple repeat performance for Mainnet.

This past week has mostly been about tending to [https://github.com/mimblewimble/grin-wallet/pull/289](payment proof export and validation), something which needed a bit of time to do properly so I wasn’t quite able to get it in before the pre 3.0.0 code-freeze and holidays, so that’ll be a 3.1.0 feature.

We’ll be looking into our roadmap + features for upcoming releases over the next few weeks, but here’s what I have right now off the top of my head as what the next few weeks should look like

  • Adding an interactive CLI mode to the wallet, which will help catching issues with the API as well as hopefully finally turn the reference wallet itself into something a bit more usable (for those command-line loving types). I hope it’s common knowledge that the focus has always been on enabling upstream developers via the API rather providing a shiny out-of-the-box experience, but it will be of great help for further API development and testing to have a command line mode where the wallet is ‘running in the background’ (like what the API expects) instead of executing a single command and exiting.

  • Better documentation on how to use the API, and particularly sample (node) code on how to init and call the secure API in advance of removing the non-secure owner API in 4.0.0.

  • Testing strategy for both the node and wallet… all of our automated integration testing badly needs restructuring and rework, so I think this should become a priority for the entire dev team for early 2020.

  • Possible SQL wallet backend to support the needs of larger institutional users.

  • And on a more ‘roadmappy’ note, I would very much hope that by the end of this year we have collectively decided on and performed some implementation toward a standard for how to send offline transactions, building on the TOR support discussed and implemented last year.

And that is by no means definitive… there are plenty of potential roadmap issues to discuss, and as-per usual plenty will come up as we go along.

Looking forward to the coming year’s work. Grin is markedly better than it was one year ago, and I fully intend to be saying something similar around this time next year.

11 Likes

Update Friday, Jan 17th, 2020

‘Boring and uneventful.’ Terrible review for your new Netflix series, but the highest praise you could possibly receive for your Hardfork, so well done all there!

I’ve been bashing away at smaller bugfixes and minor issues over the week, and it looks like we might be going for a small 3.0.1 wallet update fairly soon… nothing major and you only need to update if you’re one of those affected by the issues in the listed bugs.

I’ve also got work underway on a larger piece the cli-wallet mode. A bit of refactoring needed there to make sure the interactive mode works identically to a 3rd party application, followed by a tiny bit of glue to call all of the existing commands in the context of a command-line, followed by tons and tons of small fiddly things to ensure absolutely all of the commands work correctly and return errors where and when they should.

This process is basically putting together a minimal sample client application for the wallet APIs. It will be a bit of a slow process going through all the commands one-by-one and ensuring they all behave correctly in the command-line, but the payoff here is that each little fix to an error message here and the expected API flow there is another small improvement for all upstream 3rd party wallets.

Thinking a little bit about 3.1.0 features, I think the payment proof export and round 1 of the CLI mode should be more or less it… there was one other thing that came up but it’s not coming to mind, will fill it in later when I remember. However, there’s no longer a real need to keep the node and wallet release schedules in perfect sync, so I hope we’ll be able to achieve slightly shorter release cycles, particularly for non-breaking quality of life additions.

That’s all for now, enjoy V3 Grinning!

9 Likes

Update Friday, January 25th, 2020

Lots of background work this week, a bit of refactoring, some triaging and organizing outstanding issues, and a few interesting discussions in node dev that always require me to stop and burn out some brain cells thinking about them. Plenty done, but nothing hugely exciting from an outside perspective. Highlights are:

  • I’ve put together the list of new features and fixes for the 3.1.0 release of the wallet, aiming for the end of March as a release. TLDR: CLI Mode, Payment proof export/verification, V3 Node API and various other fixes and QOL Changes. If you think something else should be included as a matter of priority, please drop into the wallet dev channel.
  • And dev work this week was mostly focused on one or more of these issues. In particlular, finished, tested and merged the payment proof functionality, then some refactoring and cleanup in another PR, also ensuring that onion v3 addresses are used consistently throughout. Also a bit of progress on CLI Mode

And that’s all, expecting the next couple of weeks to look more or less like this one to get 3.1.0 together, then most likely thinking about some more serious work with larger roadmap items.

Keep enjoying the general post-holiday absence of distractions. I know I do!

6 Likes

Update Friday, January 31st 2020,

As expected, this week looked much like the last, mostly working through the list of bugs and issues for the 3.1.0 release. In no particular order:

  • Reworking directory setups at launch quite a bit of rework and testing here to sort out directories at wallet startup. Should resolve several issues with paths and log files appearing where they shouldn’t.
  • Several other bugfixes on the list
  • More work on the CLI mode, particularly trying to get status messages to appear via the prompt and fixing a few commands

Also preparing a new Governance RFC which should be ready for initial PR next week, as well as starting to turn my attention to some documentation, particularly updating the wallet guide and a long-awaited primer on how to invoke the wallet’s V3 API! More to come next week.

4 Likes

Update Friday, Feb 07th, 2020:

And once again, relatively quiet on the front but loads of things done in the background. Some quick highlights:

  • Biggest wallet news of the week is that the the long talked-about cli mode has finally been merged into master, ready for testing and community feedback. Note that cli mode is still very much experimental, still a bit jankesque and not 100% ready for prime time (whatever that means,). However, it doesn’t affect any existing functionality as is very much a standalone feature-add, so there’s no harm getting it in early, letting people play with it, and refining over time. If you feel like building from master go ahead and play with it, otherwise there should be a beta 3.1.0 release in a week or so.

  • Several other feature adds and bug fixes knocked off the 3.1.0 milestone list, I won’t go into each one in detail, feel free to browse.

  • Also spent a good amount of time updating the wiki’s Wallet User Guide to reflect 3.1.0 (with a bit of advance 3.1.0 thrown in). It had fallen badly out of date and several users had pointed out that it needed an overhaul.

  • And finally, a new WIP RFC on General Fund Spending Guidelines. I thought it might be a good idea to get an RFC together that clarifies how the general fund is spent. I’m sure this will spark plenty of discussion (please don’t make anyone tell anyone to keep it civil,) so will reserve further thoughts to that RFC and thread.

So that’s it for now, next week I’m going to get the last large item for 3.1.0 out of the way, address the other two outstanding issues one way or another, then hopefully have a 3.1.0 beta ready by the end of it. Hope it keeps the excitement going during this inter-fork period.

Have a weekend.

6 Likes

Update Friday, Feb 20th, 2020

Hey, it’s Valentine’s Day! I’m going to respect the usual Valentines day tradition of ignoring Mrs. Yeastplume and the Yeastlings and focus my love and affection on YOU… you lucky member of the Grin community!

And even luckier for you, I’m not going to show up at your house with flowers and chocolate and expectations… I’m just going to give you a link to the first beta of Grin-wallet 3.1.0 announcement, and assume this token of affection is silently but firmly reciprocated in your head somehow.

You’re welcome. See you next week once the blissful embrace of this gesture has worn off.

7 Likes

Update Friday, Feb 28th 2020

Happy leap year day’s eve! The past couple of weeks have been a lot of perfunctory work on the Grin Wallet 3.1.0 release, quite a bit of testing, and a quite a few messy merges directly into master to comply with a few upstream changes to Grin core dependencies. Nothing particularly earth shattering, all just necessary engineering work leading to what I hope is a solid release with a few decent new features.

The 3.1.0 release of the Node is looking in good shape as well, so we’ll likely tag and officially release both on Monday.

We should be starting to ramp up planning and activities for 4.0.0 at next week’s meeting but in the meantime I’ve also been plugging away getting a small library/framework project together that pulls in both node and wallet libs and runs them. That’s all fine and well, but In doing so I’ve found that it leads to a massive amount of redundancy in configuration and setup. This has given me some ideas about restructuring both projects, which we may get around to discussing either that meeting or the next.

Currently have builders in to support the ever changing needs of the Yeastbrood, so most of this has been done to the relaxing sound of a hammer drill on concrete. Enjoy the weekend, I know I will.

5 Likes

Update Friday, March 6, 2020

Onwards and upwards to 4.0.0 changes and features! Or perhaps sideways, around in circles and off in a random direction. Time will tell.

Particularly eventful and productive development meeting earlier in the week where we started to identify some v4.0.0 features. We’ll be filling the list in any day now … ahem. Much discussion on our post 5.0.0 forking strategy/consensus mechanisms and other interesting topics with as many opinions are the are people in the discussion. Good to see such a thriving level of involvement, and exciting times ahead.

So in my own personal sphere, The theme was very much 4.0.0, and a lot of mechanical work to support some of the features we’ve been discussing. In particular, a ‘slate-lite’ mode kicked off by this issue, that will enable the first leg of the any transaction to be significantly reduced in size and the data it presents to the other party. This is a decent privacy enhancement, and leads to some interesting possibilities like the ability to insert QR codes in the transaction workflow or serve up the initial slate as a relatively small string for cutting and pasting somewhere. (whether you’d want to is another story, and if you have an opinion I’d encourage you to join the chat in the wallet-dev channel)

And a bit more specifically in terms of PRs to support all this:

  • Removing the V2 Owner API bye bye insecure API. If you’re a wallet developer please pay attention to this, as you will now need to use the secure owner API exclusively. I’ve added a small sample node project in the repository that demonstrates how to do this.

  • Adding a V4 Slate, that looks exactly like the V3 Slate so far, but at least we all of the versioning and upgrade code to support a seamless transition around HF time is in place.

  • And as the first part of the work to Lite Slates (Slatelets?) Make the transaction element in a Slate optional which turned out a bit rabbit-holey but I think it’s there and doing its job without changing anything from an external perspective. Once this is in place the next step is to start experimenting with sending excess sums instead of a list of inputs and outputs.

And quite a lot of reviews and thinking about outstanding PRs, RFCs and concepts that have been making the rounds, trying to get myself at least somewhat informed about all of the ideas and development threads that are going on (which can hurt my limited brain sometimes, bear with me).

Right, that’s all for now. Remain calm and have a decent weekend.

6 Likes

Update Friday, March 20th 2020

World has gone slightly mad since my last update, and with a Yeastlair full of 7 people with varying degrees of neediness, some challenges have been thrown into my usual routines. However, these issues are hardly unique to me at present so I won’t dwell on them.

4.0.0 Release planning is firming up, and most of my time right now is taken up with still-experimental work of ‘compacting’ the Slate, the idea being that we keep the first leg of a slate journey very minimal to allow for a greater wealth of transaction initiation options. There will be an RFC introduced shortly to go alongside this PR, and the hope is that the V4 Slate will be as compact as possibly, ready for the rigors of a possible post scheduled Hardfork world.

With the existing work that’s already gone into the 4.0.0, and the potential changes to allow duplicate outputs, there should be plenty to do and test for 4.0.0. Needless to say there’s plenty to keep me out of trouble in the interim.

Going to keep it short for now. You most probably have greater worries at the moment than the state of a particular crypto-currency, but rest assured myself and everyone on the Grin team plan to keep plugging away as best we can.

Remain strong and sensible, get your news from reliable sources, stay safe.

8 Likes

Update Monday, March 30th, 2020

The hybrid tx building RFC is a hotbed of activity, and while I haven’t yet fully formed opinions on what transaction flows we should be advocating, I do know that ensuring the slate is as minimal as possible will enable options that we may not currently have. My focus continues to be on the experimental slate reduction PR.

Last week was very rabbit-holey in service of this PR, and involved a trip back to our secp-256k1 fork to implement a Pubkey to Commit function, the absence of which was making a particular piece of code that calculates and stores the excess for either part very awkward. It’s been a good while since anyone made any changes to that repo, so also took a bit of time there to perform some maintenance, cleaning up warnings, the build (thanks @quentinlesceller,) etc…

Then a similar round on rust-secp256k1, and the rest of the week was spent integrating changes into the slate-reduction PR and continuing to reduce, and I’m happy with progress there so far. The current state of the minimised slate is even more reduced than what’s currently in the comments, and I’ll be updating the PR with details of changes and progress later in the week. There are still a few more obvious reductions that can be done, and I’d like to get them all in first so as the tx-building PR can be fully informed.

So that’s it for my work update. These are very harrowing times, people’s nerves are likely to be frayed for reasons far more important than Grin. People aren’t at their best, and that’s okay and perfectly understandable. Though my actions are far from perfect, I’d still like to make a little appeal to everyone to at least try to make an extra effort to apply a little bit more civility and understanding then they otherwise might. Everyone’s sanity depends on it.

10 Likes