Blog

After adventures with rust-gpu, which I still monitor periodically, I moved on to the client database implementation, which is required for proper blockchain operation, and which is one of the bigger undertakings. Unfortunately, the database as such isn’t quite ready yet, but I did some preparation and would like to share some details about the database architecture.

This week has been mainly focused on refining a bit the design for plot identification and sector expiration. I think that I finally have a model with which I am comfortable with, and that I think solves all of our previous problems. Nazar had this idea to drastically simplify how plot IDs were derived, and how sectors were linked to plots. The high-level idea made sense, but there were still some details that weren’t clear. This week I managed to come up with a design that I think satisfies all of our requirements.

There are buzzwords in any industry that are thrown around easily, and blockchains are no exception. In this post, I want to focus on “scalability”. Turns out when you say “blockchain scalability” different people hear different things. The prevalent opinion seems to be that scalable blockchains are able to process more transactions than non-scalable ones or something along those lines. Essentially making the ability to scale equivalent to peak performance. Sure, peak performance is an important metric, although it is often a theoretical one. But I don’t think that is the most useful property, especially without clarifying the conditions under which it can be achieved.

I have to admit that I am a bit disappointed with my progress this week. If you recall from last week’s update, I started the week with a base proposal to handle the linking of sectors to plots based on history window ranges. The idea was to limit the number of parallel sectors that could be created in parallel, linked to the same plot, and hence allocated to the same shard. While the approach seemed quite elegant because it didn’t require any changes to how piece selection and expiration currently works in Subspace, it turned out to be pretty complicated (even to explain) and not the most effective solution to prevent the attack I was trying to mitigate.

GPU plotting was one of the items on the roadmap last week and that turned into a week+ long side quest, so let me share some details about that.

As mentioned on last week’s status update, one of the key pieces that I was missing to have the detailed operation of plot membership allocation was the impact of sector expiration on the protocol. By having a unique plot identifier, we are able to uniquely link sectors to plots, but these sectors need to expire in a way that does not require farmers to re-plot while archiving the most recent history. Fortunately, we can leverage the current expiration mechanism of the Subspace protocol, and build a layer on top of it to adapt it to the sharded version while maintaining the original guarantees in terms of plot expiration, sector re-plotting, and history archiving.

Last week I shared a model that can help us reason about the security of shards assuming an honest majority in the beacon chain. The model evaluates what are the trade-offs in terms of the number of shards, the number of farmers per shard, and the proportion of malicious farmers in the system. But if you recall from the overall design of the system, shards are periodically submitting segments and blocks to the upper layers of the hierarchy and to the beacon chain. We need to verify that these are valid, available, and correctly encoded before they are included in super segments and the global history of the system. Can we do so without relying on fraud proofs? This has been one of my focuses for the week, let’s jump right into it.

This was a lighter week on meaningful changes, but there are still few things to share. First of all, last week’s block import was (and still is) and incomplete prototype, but this week it was extended to become a bit more complete. I was also working on bringing up more components frm Subspace, including farmer, which lead me to attempt GPU plotting in Rust, so let’s get into it.

This week has been all about objectively assessing the security of the system. After all of the work around the membership allocation protocol and its security there are still two questions that we need to answer to understand the feasibility of the protocol: (i) what is the security bound of the protocol as a whole (from beacon chain to shards), and (ii) how can we ensure that plots are uniquely identified and that farmers cannot cheat by committing the same plot to different history sizes to try and game the shard allocation mechanism.

There was no update last week again since I didn’t feel like there was enough to share at the time, but now that more things have settled I’d like to share what I’ve been busy with.