Tales from the ZK Residency: Herodotus Builds a Coprocessor with SP1
At the Succinct ZK Residency, Herodotus developer Pia Park contributed to building hdp-sp1, a ZK-coprocessor that allows you to easily access verified onchain data with SP1. In this interview, we speak with Pia about her developer experience, her motivation for building this project, and her time at the residency.
Q: What is the Herodotus HDP and why did you choose to attend the Succinct ZK Residency?
A: The Herodotus Data Processor (HDP) is a ZK-coprocessor that allows you to access onchain data in a trustless way, by verifying the data against a Merkle Mountain Root (MMR) on Ethereum. It lets you read data from multiple chains, including their historical data, using ZK. I wanted to attend the Succinct residency to understand how SP1 could be integrated into the HDP project, and compare it to an existing implementation.
Q: You came into the residency already having a Cairo VM implementation of HDP. What was the motivation for reimplementing it with SP1?
A: The Cairo VM implementation was efficient, but writing Cairo is more demanding compared to Rust, as it’s a domain-specific language. Rust-based zkVMs are developer friendly and don’t need special management for Ethereum-related types and data structures. We wanted to compare the SP1 developer experience to the Cairo experience, and reimplement HDP using Rust.
Q: How was your experience working with SP1?
A: We were able to finish the SP1 implementation in less than a month, with less than two weeks of full-time work at the Succinct Residency. The SP1 SDK was really useful and communicating with the prover via the SDK was an amazing experience. The Cairo implementation took 5 months, although this included research, design and some iteration. Our main conclusion was that the zkVM has a much faster development time.
Q: What is the functionality you were looking for in a zkVM when implementing your project?
A: We needed a zkVM that could generate proofs that can be verified on Ethereum. The zkVM therefore needed to have GROTH16 or PLONK support, which SP1 does. It was also useful to have hashing precompiles such as keccak256, which makes proof generation very fast for our use-case. Being able to write normal Rust was also a great added functionality of using SP1. SP1 also comes on with built-in proving of long programs (“continuations” or “shards”) which offloads solving scalability problems to the zkVM engineering team.
Q: What were the technical and design choices that you made when working with SP1?
A: HDP operates in two steps. First, there is an online mode where the data is fetched and a zkVM mode where proving occurs. With SP1, we figured out a way to do these two steps in one abstracted interface in several instances.
With Rust, we were able to use macros. Cairo doesn’t support sophisticated macros and there was a limitation to the interfaces we could make. The macro allowed us to abstract all the SP1-related stuff from the end user. The end user just needed to write a normal Rust program by using the hdp-sp1 macro.
This same principle allowed us to build an abstracted SDK that splits up the online mode from the zkVM mode. Rust also allows you to easily switch the target OS.
Finally, while doing proof verification, in the online mode, we are fetching untrusted data. We then put that data inside the zkVM as an input and do verification of that data against the MMR root which we have been growing on Ethereum. Rust allowed us to do this in an abstracted way.
Q: Thanks for the insight into HDP! How was your experience at the residency?
A: I enjoyed the residency a lot! There were quite a few teams building on top of SP1 and participating in the residency. I had a great time hanging out. Also, since it was in person, we could ask for help. I was able to go into technical depth with Succinct’s engineers and learn more about SP1 for my personal curiosity.
