Everything is in the title. In this new blog post about DAC, we are going to deploy our own one.
- The excellent article, published by Nomadic Labs, introducing Data Availability Committees.
This article is essential because it explains what a DAC is, its architecture, how it works, the role of the all the DAC actors (Coordinator, Members and Observers).
- Our previous blog post announcing Community DAC on Tezos Ghostnet.
In that blog post, we explain how to interact with DAC and how to do a “complete loop”:
- Push data,
- Retrieve certificate,
- Retrieve data.
We are going to configure and run:
- One DAC Coordinator,
- Two DAC Members,
- One DAC Observer.
What are we going to configure and run?
In this blog post, we are going to configure each actor of DAC:
- One DAC Coordinator
- It is responsible to receive payloads from client, propagate them to DAC Members and Observers. It also stores and provides the DAC certificates of payloads stored by Members.
- Two DAC Members
- They have a forever connection to the Coordinator and will get notified of new payloads from the Coordinator, after which they will download the data and send back a signature to the Coordinator to attest that they will make the data available upon request
- One DAC Observer
- Like the Members, Observer will also download payloads from Coordinator, but it will not sign it. It is also responsible for retrieving missing data from DAC Member. This actor is meant to be used to provide data requested by Smart Rollup node via the reveal data channel.
The schema below, from Nomadic Labs blog post, shows the previously explained interactions.
We will not dive into technical details about DAC implementation in this blog post.
Create data directories and setup environment variables
Set up the environment variables to ease usage of next commands to configure and run actors:
Set up the directories listed in the environment variable above:
Configure the accounts
If you want to use already existing keys, you can import them:
Save both the public key and the public key hash of the members, we will need them later
Configure and run the DAC infrastructure
1. Configure the Coordinator
2. Run the Coordinator
You must see the logs of starting Coordinator:
3. Configure the two Members
4. Run the two Members
And see the starting logs:
Then next member:
And also see its logs:
We have twice this line since we have started two Members. This line will also be printed one starting the Observer.
5. Configure the Observer
6. Run the observer
And once again, on the Coordinator logs:
🎉 You just successfully deployed your full DAC infrastructure!!!
This task was relatively straightforward to complete and it will enable a better TPS for your Smart Rollups!
Interact with each actor.
In this last step, we are going to fetch HTTP APIs of each actor to make sure everything is working fine.
Push data to Coordinator.
The first interesting thing to do to validate the entire infrastructure is working well is to push data to the Coordinator.
Since you have read the previous blog post announcing Community DAC you are supposed to already know how to post data, retrieve corresponding certificate and finally fetch back data, but I will re-explain it here to add the logs of each actor!
While on the Coordinator logs you must see:
Which means our two Members had successfully donwloaded the data and signed it, and sent back their signature to Coordinator.
Let’s now retrieve the Certificate!
Retrieve the certificate
We are going to fetch the JSON certificate, which is human readable:
Which will return:
There is no specific logs on DAC infrastructure side.
Retrieve the data
Will return the original payload prefixed with 5 bytes:
You’ve successfully deployed your own complete DAC infrastructure! Now, you can operate a Smart Rollup alongside your Observer to achieve improved TPS!
As you may have noted, our initial setup was only local, which isn’t ideal for production use.
In the next section, we’ll guide you through deploying it on Kubernetes!
Production-ready DAC infra on Kubernetes
In this section, we will outline the steps to deploy a full DAC infrastructure on Kubernetes. This tutorial is cloud provider-agnostic.
Prerequesites: a good understanding of Kubernetes.
Kubernetes manifest files
You can access the YAML files referenced in this section from this public repository, feel free to reuse and adapt them according to your needs.
Outlook on the YAML files we got:
- configmap-env-vars.yaml: various environment variables and feature toggles.
- configmap-scripts.yaml: shellscripts stored in a configmap and mounted in the pods.
- deployment-<actor>.yaml: k8s deployment files for the coordinator|members|observer.
- ingress-<actor>.yaml: ingress and service files to publicly expose the DAC actors on the internet.
- pvc-<actor>.yaml: persistent volumes to store the data on.
Docker image used to run the nodes:
- This setup demands the Octez binaries alongside a few other utilities, all encapsulated within a Docker image. The most straightforward way to obtain this is by using the official images available on DockerHub. These images come preloaded with the octez-dac-node binary.
- To initialize the nodes, we will run multiple shell scripts. For ease, we have opted to include Bash as a binary within the Docker image. However, for a sleeker setup and enhanced security, you might want to execute these shell scripts in an InitContainer.
Env vars and feature toggle flags:
- In Kubernetes, updating a ConfigMap not automatically trigger a restart of pods that have mounted that ConfigMap. Pods using ConfigMaps retain the old version until manually restarted. Pods will continue to use the previous version of the ConfigMap until they are restarted. We will use that behavior to conveniently manage some feature flags.
- Example of toggles and env vars we set in the configmap:
Storing scripts in a configmap:
Bash scripts are stored in a configmap and mounted in the pods.
Basically each script does the following:
Cloud native security & management of secrets:
If you’re operating the DAC on a Kubernetes instance or a private server managed by a cloud provider, setting up Authorization and Authentication is essential. Most often, this is done using the cloud provider’s IAM (Identity and Access Management). Such implementations ensure that only authorized users can access sensitive cloud resources and that they can execute only permitted actions.
For maintaining security specifically within Kubernetes, it’s imperative to follow its security best practices, which include:
- Role-Based Access Control (RBAC)
- Network segmentation and policies
- Encrypt data stored in Kubernetes’ etcd key-value store
- Implement policy engines
- Limit access to Kubernetes API
- Configure security context for pods and containers
- Use namespaces to isolate workloads
Exposing DAC endpoints:
As illustrated in the provided ingress manifests, we use the Nginx ingress controller as a reverse proxy to supplement the CORS headers:
Additionally, some ingress configuration options need tweaking:
In our case we also use Nginx ingress controller as a way to expose DAC endpoints over the internet on a public endpoint.
You can also reach your DAC from within Kubernetes on its corresponding service: <name-of-service>.<k8s-namespace>.svc:80
Backup and recovery solutions:
DAC data not only resides on Persistent Volume Claims (PVCs) – a dependable storage solution within Kubernetes – but is also replicated across all DAC community Members and Observers. Despite this inherent redundancy, it remains prudent to have an external backup and recovery strategy for your PVCs.
We opted for Velero, an open-source tool, that offers the ability to safely backup, recover, and migrate Kubernetes cluster resources and persistent volumes, and comes equipped with a disaster recovery feature.
In this long blost post, you successfully:
- Run your own full DAC infrastructure locally
- Deployed it!
In our next blog post, we will show how to use DAC inside your Smart Rollups!