It is part of the culture of the EOSIO ecosystem to spin up new chains when new needs arise, when different governance structures are required, or when different resource models are proposed. This is in line with the broader vision of letting a thousand chains bloom, gathering learnings from everywhere, and improving at each step.

In this world with many chains, it becomes essential to have great tools to spin up, reset and/or reboot chains. As EOSIO hits the Enterprise, you can easily imagine a corporation spinning up dozens of small databases, some of which are corporate-wide, others department-centric, and some shared with clients, partners or vendors. Others chains may be ephemeral, starting from an agreed upon state living on another chain, in order to process millions of transactions in a burst, and then be torn down, with the final state going to settle back on the originating chain (think of it like as an ad-hoc Lightning Network between two parties).

All of this becomes easier with the new dfuse Migration Tools

Benefits

  • Accelerate your contract development flow: evolve your data model faster without painful on-chain mutations, yet keep all your other team members happy by keeping all their state intact.
  • Boot new chains, honoring all the accounts previously existing on another network (think EOS Mainnet Chain Extensions).
  • Trim the history of long-running chains, without disrupting your users.
  • Deterministic boot: requires only an agreed upon snapshot block height plus your scripts to modify the state (if any). A decentralized group can then verify the integrity of the new chain independently, and perhaps sign a transaction to activate the new chain.

How it works

The latest release of dfuseeos contains a new command:

$ dfuseeos migrate --snapshot=./path/to/snapshot.dat 

This command will take a portable state snapshot (which is created periodically for you if you are using dfuseeos‘s node-manager), and will lay it all out on disk, in an easy to navigate directory structure (under ./migration-data), comprised of .json files, .wasm contract binaries, and their corresponding .abi files. Collectively these files represent the complete state of the blockchain at the time of the snapshot, with each account in its own folder.

You can then write simple scripts to clean up, mutate contracts, change ABIs, modify data rows, modify secondary indexes, add or remove accounts, change key structures — tweak any part of the state. Those scripts need only to interact with the filesystem and .json files, so they can be written in Python, NodeJS, Haskell, C#, Go, even awk or sed if that’s your cup of tea.

Once you’re done, two more commands will boot your new chain:

$ dfuseeos init
Wrote dfuse.yaml
$ dfuseeos start
Booting and injecting your new chain...

This will boot a new chain, pick up the bootseq.yaml file generated by the migrate step, and inject all the accounts, contracts, data rows, indexes, and permission structures found in the migration-data folder. Special care is taken to ensure any dependency cycles between account permissions are all restored, as well as that all data is still assigned to the correct RAM payers, etc.

That’s all you need, to boot a fully functioning EOSIO distributed database.

Big thanks to ULTRA for sponsoring some of that work, to cc32d9 for his contributions, and to the community for its feedback.

Start today, at https://github.com/dfuse-io/dfuse-eosio and connect with us on Telegram or Twitter.