Temporal Kubernetes Stack — Scaling IPFS Clusters & Sharding the Pinset
We are proud to announce the addition of Kubernetes currently in development for Temporal! This was an easy decision for our open-source platform, Kubernetes will scale Temporal to interplanetary means and is an excellent step in the right direction for IPFS. Below is a brief explanation as to why.
Currently, Temporal is limited to rudimentary docker containers, which can be managed with docker-compose. However, there is still a lot of manual effort in maintaining the system, whether or it is the non-container based HA rabbitmq or postgres setup, configuration management, or general systems administration. The first iteration of our Kubernetes deployment will focus on a single-node cluster, capable of quickly being deployed with services like minikube. While the second iteration will be a full-blown Kubernetes cluster, capable of scaling across multiple geographic regions.
Kubernetes allows us to create and scale highly available IPFS clusters on command. Along with finer control for monitoring and configuring an IPFS cluster. Using technologies such as IPFS, libp2p and Kubernetes, Temporal plans to bring interplanetary mainstream, allowing developers and users to have more freedom and control over their data. This will be a great new addition to Temporal, allowing enterprises and users alike to enable scaling up or down much more quickly and efficiently.
Using composite clusters, we can shard the main cluster into sub-clusters. What this accomplishes is distributed pinning along with sharding the IPFS dag. When an IPFS cat request is called, IPFS resolves the Merkle dag and finds the required objects regardless of where its located in the cluster. This also allows for a highly scalable IPFS cluster using composite clusters.
- Machine learning and big data
- Backup storage
- Distributed Datastore
- Distributed pinning
- Cluster Sharding
- Composite clusters
Stay tuned as we will be releasing great in-depth documentation and deep-dive articles regarding Temporals Kubernetes stack in the near future.
Temporal, an open-source API with scalable IPFS Infrastructure, allowing users to build or plug into production-ready apps/dApps on top of IPFS quickly. Temporal offers a free starter package featuring turnkey APIs, toolkits & storage to achieve enterprise-level scale and security without the need for expensive or cumbersome infrastructure.
Temporal’s platform handles all the heavy lifting to provide even basic users with powerful IPFS products like pinning services w/ 3GB free, IPFS HTTP API proxy, TemporalX enterprise IPFS nodes, folder uploads, private network management and much more.