Blog

We started last week with two PRs that attempted to describe in detail the operation of sharded archiving (PR192), and the data availability layer of the system (PR193). When I started writing this spec, it was meant to be for a broader audience, but we realised after a few rounds of feedback that the project is still in a really early stage and in constant change, so it would be more efficient to focus on detailing the parts of the protocol that are currently under-defined instead of trying to give a deep overview of the overall operation of the protocol from the get-go. The actual goal behind this protocol specification is to unblock the implementation of a prototype that can help us gain certainty about the design decisions that we are making, and surface potential blind spots in the design, and not to have a reference spec (just yet).

Most blockchain implementations are pieces of software that include the logic to support many different roles/features, possibly all at once: bootstrap node, block producer, RPC node, archival node, light client, etc. That is one way to do it, but one thing I learned over the years is that you can do a lot of interesting optimizations if you can apply additional constraints during the design phase. So why is basically everyone trying to combine all into one? Let’s consider different roles separately first and see what is special or interesting about them.

Over the past weeks, my updates have highlighted many of the ideas emerging from our open design discussions. Now that we have a clearer direction for the design, I wanted to consolidate these ideas into a draft spec. This will serve as a foundation for implementing the first few prototypes, while also providing a structured way to gather feedback and uncover potential blind spots. I expect this spec to suffer significant changes, but it felt like the perfect way to consolidate the ideas, get feedback from the community, and unblock Nazar in case he wants to start prototyping some of the ideas we’ve been discussing.

Last time I mentioned that I was looking into Merkle Trees to replace KZG. This week it happened, the whole codebase is basically free from KZG. The only place that is not fully fixed and where I am looking for help is GPU plotting, it broke with all these changes and isn’t the highest priority for me to fix right now.

Last week was lighter on code changes and more heavy on research. Specifically, I’ve been looking into commitment schemes generally and Blake3 hash function in particular, which was already used in the codebase, but turns out can be applied in more interesting ways than just a hash function.

This week has been another good week of progress. I finally have a good idea of how shard archiving should work in the happy path, and I’ve started writing a low-level spec for it (that I am hoping to push to this repo soon). Unfortunately, there is still a slight gap in the spec that we need to fill before we can move forward: the data availability problem.

Welcome post mentioned briefly initial set of constraints that led to the creation of this project, but I figured it might be helpful to have a short writeup about it that might be helpful for sharing. In short: we’re building a blockchain. By “we” I really mean just me and Alfonso so far, but I hope more people will join over time if they find it interesting.

The majority of last week I spent tinkering with Subspace codebase after importing it here in preparation for building an actual blockchain.

This week I’ve gone a bit deeper into the design of the multi-shard Subspace protocol idea which I briefly introduced in my last update. The protocol is conformed by the following parts: Sharded archiving, responsible for creating a global canonical history of the whole system, and of creating the history records that will eventually become part of farmers’ plots. Sharded plotting, which takes records from the global history and seals them in plots that include segments of the history of every shard of the system, and that will be used for the farming process. And finally, merged farming, which is the protocol responsible for challenging farmer plots, and deriving the corresponding winning tickets that elect block proposers in specific shards. Let me introduce the high-level operation behind each of these sub-protocols, while digging deep in the one that I’ve focused the most on this week: sharded archiving.

After a lot of thinking, this week I came to the realisation that a sharded architecture like the one we are trying to build can be designed leveraging the current design of the Subspace protocol and all its underlying mechanisms as a base. While this was the idea from the beginning, either for lack of familiarity with the protocol or plain ignorance, I was missing the big picture of how this could be done.