Blog

There have not been an update from me last week, how come? Well, it didn’t seem like there was anything particularly substantial to share, mostly some refactoring, so I decided to skip it. This week though I have a few words to say about the path towards block production (not there yet, but getting closer). Some this is a boring process, I procrastinated some too, diving into various topics with some interesting performance improvements and new learnings.

This week has been another of those weeks where I am pretty happy with the progress made. The highlights of the week are the following: (i) we now have a pretty good sense of the end-to-end operation to commit shard segments into the global history of the beacon chain (as described in the discussion of PR267); (ii) and Nazar had an idea to tackle the verification of the availability and correctness of shard segments without requiring an independent data availability mechanism, by leveraging the longest-chain rule and farmers membership allocation (which was an issue that was really bugging me). Let’s jump into the details of these two topics.

I started the week thinking about the mechanics of shard segment commitment into the global history of the beacon chain. If you recall from previous updates, we already had a pretty good idea of how the information about child shard segments flow up to the beacon chain, but there were still a few questions that were really bugging me.

The question might seem somewhat obvious: you have a header and a body with transactions, many blockchains have it, what might be so difficult about it? Well, as I mentioned in the previous update, there are some complications and part of the challenge is related to the fact that we’re dealing with sharded architecture, which most blockchains don’t need to deal with.

I am pretty happy with the progress this week. Funnily, the main culprit for all of this progress has been the the work that Nazar has been doing on the definition of the block structure for our hierarchical consensus. I’ll let Nazar dig deeper into what he’s been doing here, but let me share in this post what this block structure entails, and how this has unblock several lines of work, and solved many issues for me.

This week has been mainly focused on clearing the fog around shard archiving and trying to start fleshing the low-level details for the protocol. It feels like in the past few weeks we’ve been surfacing more questions than answers, and I honestly think this is a good sign. It means that we are getting to the point where we can start to see the details of the protocol and how it will work in practice.

The discussions with Alfonso and attempts to start building an actual blockchain led to spending a big part of last week working on block structure. That work is not done yet. However, I like to have some sort of accomplishment at the end of the week if possible, so I spent some time to finally implement the formatting of addresses, which will be the main part of this update.

This week was very similar to the last one with a bunch of refactoring in cleanups. There were important archiver improvements/fixes (depends on point of view) and more work on Merkle Trees. Two more crucial crates were moved from subspace to crates.

We keep iterating on the best way to discuss and make progress on the design of the protocol. Using issues for discussions have shown less efficient than originally expected. The inability to make in-line threads, and having to quote every single detail of the spec that we want to discuss about was really cumbersome. I started the shard block submission issue as an attempt to start iterating the low-level details of specific protocol mechanisms in a way that is narrow enough and easy to track, but it didn’t fulfill all our needs. The solution? Creating discussion PRs that I don’t expect to get merged, but gives us all that we need to have low-level discussions about specific parts of the protocol, track our progress, open ideas, and discussions, and have them public so anyone can contribute or follow along.

The last week was lighter on major changes, but there was a lot of cleanups and refactoring done to prepare Subspace components reuse for building a new blockchain from scratch. Also some improvements based on new developer feedback.