Ever since the release of our open source single binary “dfuse for EOSIO“, we have received many requests to explain the foundational architecture that it encompasses. In this video, Alex reviews manageos – the dfuse node manager – which wraps nodeos and adds functionality through many additional endpoints. He also introduces our mindreader service, for offering block logs over gRPC.



Welcome back to the overview of the dfuse architecture, a series of videos.

In this video, we’re going to introduce manageos, what is mindreader, what is a node manager and the bells and whistles this offers.

Ok, today we introduce manageos, and manageos lives in this repository here. Which you can see lives in dfuse-io/manageos. It’s what we call a “node manager”, it is a thing that actually wraps the execution of nodeos, or a blockchain node (it supports also Ethereum), it wraps the execution of nodeos and adds a bunch of bells and whistles. Namely, the endpoints that you can query. Let me come and see that here, the http server.

This is the operator which lives in /manageos/operator/ down there, okay? The repository is manageos, subdirectory operator, and it adds all these endpoints on a separate port. It doesn’t confuse with /v1/chain or things that are sent directly through nodeos. But it adds all sorts of things. We’ve been using that for EOS Canada to run our own production nodes, our own pool of nodes we were offering. I assume Block Producers have something like that, this is our version, but we’ve augmented it because it now has all the things that you need to pipe data out and feed it to the dfuse infrastructure.

There’s a series of endpoints, for example, /maintenance and /resume, that work together to shut down the node properly, without making it a dirty state. Shutting down properly so you can do things, change the configuration, whatever, and then you can resume, starting again. And then there is also, for example, /reload, which does something similar. Just goes through a reload, it waits until it’s properly shut down and then boots it again with the right parameters and all that.

So very interesting ones are /backup and /restore, and there’s three different ways to backup actually. There’s /backup and /restore, which will give you a pitreos backup. And a pitreos backup — we’ll introduce what pitreos is: point in time recovery for EOS — is a dedicated backup solution, we have. We’ll have another video on that.

The other aspect we have is /snapshot, the portable snapshots that are managed by nodeos itself. We can send the instruction to send backups. Actually manageos can do that periodically for you without interrupting your node, and stores that in a location, which you can then directly call /restore. And then it’s going to do the management of shutting down the node, putting the right parameters just to boot from a snapshot, just simplifies operations.

And there’s a last volume snapshot thing, and in there you have service definition. This is what we call the blockstream, this is the first thing that is outputted by the mindreader to start being the source of blocks in this system. It is a server, it’s implemented in mindreader here where we create a new blockstream server.

Here I’m in /manageos/mindreader. It’s the feature of the mindreader. So we run the node, it offers that gRPC server and it’ll take information, build blocks and then serve them as a blockstream server. Anyone coming in and asking for a block request, asking for a certain burst for content type, whatever, with order, for logging here. Well, they’ll get a stream of blocks starting to be fed. And we’ll look inside, we have another video on merger and relayer, how we built topology, I’m not going into this right now. But just to say that, when you run dfuseeos, let me show you that here in the terminal. So if you run `dfuseeos init`, if you answer yes (you remember that), it produces that file there.

So mindreader and node-manager, these two things are actually instances of manageos. One is for running the production node, one is for running the mindreader-instrumented node, it’s just an added layer. The mindreader one will offer block logs, as a gRPC service.

So, that’s the crux of it. Mindreader has an option to automatically produce those 100-blocks files, already merged. We’ll have an introduction on these files in the bstream video. That’s an option so you don’t need a merger or relayer.

I hope this is helpful, for understanding better the mindreader, if you have questions put them in the comments, come to our Telegram channel, don’t hesitate. I’m gonna do some videos to cover a few of the things that you had as questions in these topics also.

I hope this one was useful, so I’ll see you next time