Cardano Progress Update by Sebastien Guillemot (Sept 2019)

Hello everyone! My name is Sebastien and I’m the Engineering Lead at EMURGO, one of the companies behind the Cardano project and it’s September already, so it’s time to go over all the work that we’ve done in the month of August A lot of has happened this month and we’ll be talking about the Shelley networking testnet, the Shelly incentive testnet. I’ll be talking about Yoroi and Daedalus and the work we’re doing, we also will be talking about some new products, notably a new product by EMURGO called Tangata Manu Which is meant to replace Icarus So if you’re a business who is using Icarus to power your Cardano integration will definitely want to listen to this video We’ll also be talking about new partnerships. For example EMURGO’s partnership with trade associations in South Korea Let’s get started. The most important thing in this video is this blog post By IOHK describes how the Shelly incentive testnet will work What is the Shelly incentive testnet? It’s a way to make sure that all of Shelley behaves as expected So you need to be able to create stake pools, join stake pools, create blocks, send transactions Everything is included To make sure that everything works the way we expect, we need real incentives, we need people to actually care to behave in this test network as they would in the real network, and to do that we need to make sure that the rewards you get in the test network are somehow real, they correspond back to ADA on the main net This blog post describes how we plan to do this Notably, we plan to take a snapshot of all balances on the Cardano network, and the incentive testnet will essentially be a fork based on the snapshot Using this you can create stake pools, join stake pools everything and once the testnet is done, and we’re happy with the result, the result of the testnet will be taken and merged back with the mainnet and that’s what will become the Shelley mainnet There’ll be a link to this blog post in the description of this video, so you can read this blog post for yourself and really understand what this means for you and how you can be part of this incentive testnet We have a lot of stuff coming up in this month, in September The networking testnet, which is the part before the incentive testnet is planned to launch in September, according to IOHK. We also have some more research papers published in September such as Ouroboros Hydra, which describes how we plan to charge the Cardano network We’ll also be having our second year anniversary in September in Bulgaria So if you want to join us in Bulgaria for the 2nd year anniversary, definitely check out the Cardano foundations post on the Cardano forums. I’ll have a link in the description I hope to see you there because I’ll be there The next thing I want to talk about is this new project by EMURGO called Tangata Manu Let me give you the story behind this Last year we worked with IOHK to ship a product called Icarus. Icarus was an open-source library to run light clients in a way that can easily power projects like wallets and explorers The problem with Icarus is that it’s made for Byron. Not for Shelley So we need to update the system in order to be compatible with the existing environment release but also with the Shelley release And that what Tangata Manu is That means that if your business, who is currently using Icarus You will have to migrate to Tangata Manu or something similar in order to keep your project alive during the transition to the Shelley testnet and Shelley mainnet release Tangata Manu is just like Icarus is open source. You can find it on GitHub I’ll have the in the description so you can set it up yourself, or you can work with EMURGO and we can help you With that out of the way, let’s get started on talking about jormungandr Jormungandr is the project for the Rust team, that is going to be the Shelly incentive testnet They’ve been doing continuous work on this project. In fact, they had a release 18 days ago So if you’ve been trying out the testnet so far – try out the new version we launched this month The main thing that we’re working on for jormungandr is network support so the testnet

we have released currently for Shelley is a single node testnet so you can run a node but you cannot communicate with other nodes on the network Now we’re adding the networking layer support So you might remember last month I showed you this plan that we had for how to add networking support to jormungandr, and we basically described the structure and all the work that has to be done and see at the bottom We have a list of pull requests, features that need to happen for this new way to store blocks to enable the networking support. In the past month we got all those tons You can see all these pull requests over here, the L corresponds to the prerequisite work we had done here to essentially change the way the testnet blocks are stored, and the way to easily attach them to our networking layer So with that, we’re doing more work on networking layer, a lot of the last parts to it So you can see we have some other pull requests for stuff like how to go through trusted peers if you want to do fast sync, how to essentially test multi-node networks, and how to basically do all the work required with certificates to make sure that certificates can be broadcasted across the network and all this kind of stuff That is basically required for a networking testnet The plan is still to launch this in September, and we’re kind of doing the final of work Last month we had some previous work for how to connect the peers, download blocks all this kind of stuff. So hopefully, if you do not see the networking testnet this month, it’ll be very soon because as you see from these pull requests, we’ve already got basically the work done. Just kind of some last polish to get it out the door With that, we need to make sure that everybody can easily not only run the node, but debug the node in case anything goes wrong To do that we have APIs, that you can use to connect to your node The first version of the Shelley testnet contain these APIs and now we’re making them more easy to use for businesses or users who want to get information about the network through their node To do that, we’ve been working with something called open API, which is a well-known standard Sometimes known as Swagger, which probably a lot of developers who are watching this video already know For how to easily create UIs for your API, editors for your API, documentation for your API all this kind of stuff. So all the work to connect to open API has been done in the past month. And so that means that if you are running a jormungandr node in the networking testnet, you’ll easily be able to create tools that talk to your node, to debug, to monitor, to trigger actions. Everything you need to do. It’ll all be set up and fairly easy to handle We’re making more changes to the API still Nobody as we realize that more things are required. You can see we have some pull requests for adding some extra endpoints If you’re running a node for the test net and you have an endpoint you think would be useful, definitely reach out to us. You can create an issue on this GitHub repository or we have a Telegram group where stake pools talk with the developers about their needs and what they need to be changed, and we take those suggestions and implement these into this codebase We also have some more tooling on top of the API. So we don’t just have an API We’re also adding some extra tooling such as an explorer mode. Sometimes it’s not enough to just be able to create your node, you want to get a view of the whole network. With that, we have an explorer mode, which is essentially a specific configuration for an API to get information about the network as a whole. Note, that this explorer mode is not an explorer like Seiza or Cardano Explorer It’s just an API so developers can easily get information from their node and possibly build tools with it in the future To make wearing these API is easy we’re building a graphQL server and if you’re not familiar with graphql, it’s essentially a way to easily query your API in a way that makes it easy to change in the future, it’s very performant if you only need subsets of data and just in general, much nicer to play with than traditional or breast API’s. That was added in the past month,

if you want to run your jormungandr node, get information out of it, maybe get information on the network through an explorer. you can now do all that fairly easily with graphQL To kind of show how all these tools that we’re working on hook together, we’re building a sample or react application. So a react application is a way to develop a UI that works both on websites as well as native desktop applications on your computer, and this react web app will basically show you how you can use the various tools we’ve built to build a system that essentially lets you monitor your node on your network, so you can see it. You have some pictures here It’s not super readable because this is made for developers, not for general public But the kind of gives you an idea of what we’re building. You can check out the code on this GitHub We end up closing this pull requests because it’s going to go in a separate repository Definitely check out the IOHK GitHub repository, which will eventually contain a new repository for this example of how to build your own website to monitor your node and monitor the network Another thing that we changed is the way address prefixes are generated on the testnet They used to all have a constant prefix based on BIC 32, but now they can be custom. If you’re a regular user you don’t have to really worry about what I just said, but if you’re building a tool that parses jormungandr, Shelley addresses you kind of need to be aware of this change With that we have a lot of tooling built for the networking testnet, we have a lot of code done through networking testnet and we’re planning to ship it in September. Where do we go from there? As I mentioned the next step is the incentive testnet and work has already started on that so you can see this branch here called incentive and this GitHub repository already has a code being contributed to it We’re not taking a break. We’re going straight into the next step because we want to get Shelley released by the end this year Let’s say you’re super passionate about jormungandr, about this and you don’t only want to run your own node You want to create a wallet or some sort of user-facing application that connects the Shelley. To do that we have WebAssembly bindings, and I covered this in the previous month and now they’re much more stable. So what that means? Is that the documentation is much more well written, complete. You can see how to use the WebAssembly bindings, how to set up with webpack, which is a very common tool for JavaScript developers, how to connect to the faucet so you can build wallets using WebAssembly bindings, connect to the Shelley testnets faucets and get some Ada into it Now it should be much easier for companies, businesses, developers to build an application that connects with Shelley and figure out how to set it all up. We’ve also added some functionality that was missing. Basically, when I covered these WebAssembly bindings last month, we’re almost done the first version of the Minimum Viable Product and these kind of last changes get us the rest of the way there. So obviously although the WebAssembly bindings right now they’re fairly stable, we’re looking for feedback from other developers about what needs to change, especially what we need to add to connect with different products. So us at EMURGO, we are hooking Yoroi up to the Shelley testnet and so to do that obviously we will use these WebAssembly bindings to power Yoroi, and I assume other companies will do the same As we do this integration will come up with some new ideas. The other people come with some new ideas. Hopefully, we can get all these changes included in this WebAssembly binding repository to make the whole experience good for developers and easy for everyone If you’ve been waiting kind of on the sidelines and you want to build them with Shelley support now’s the time to get off the sidelines, come in and start building. This is a viable enough for the first version and it’s time to start building because we’re releasing this year so you don’t have that much time to get all out the doors. Now’s the time to start Speaking of which, as I mentioned we’re obviously integrating Shelley into Yoroi, but that’s not all that we’re doing, we’re doing a lot more changes The first one I want to mention is we’re changing the way you connect your Ledger device

From a user’s point of view, I think this is not so important to know, but it may explain some stuff you run into. Currently, if you connect to your Ledger to Yoroi, you’re using something called u2f And u2f is being deprecated. Then means it will no longer be supported and to keep supporting all Ledger devices we have to change the way Yoroi works to support something called webauthn This pull request basically describes all the changes that need to be made and essentially it implements them. And so that means that the end impact for users is that probably next month as you connect your Ledger device to Yoroi, you’ll see a new UX and new UI to facilitate this new protocol Don’t worry too much about it This is expected, we have to do this. So if you see a change as you connect your Ledger device, don’t worry too much about it, we are making changes. If you are a developer, who also used u2f to power a Ledger integration for Cardano, definitely look at our work and see what changes you need to make to be compatible with this new connection method for Ledger devices We’re also working a lot on multi-wallet support. So if you’re Yoroi user you’ll probably know that right now we only support one wallet, and we want to change that to be able to support multiple wallets Not just because that’s a useful feature, but also because that’s required for Shelley We’re trying to make a very general scheme that allows us not only to support multiple wallets, not only allows us to support Shelley, but we also want to allow read-only wallets, multi-account wallets. A whole variety of different combinations that are possible, and so to do that requires a lot of work and that’s what we’ve been working on This will take a lot of times, you can see it’s been quite some work But hopefully get there and we hope to have this in time for the Shelley incentive testnet We’re also working on some other features, such as the ability to choose your unit of account Right now if you open up Yoroi, you’ll see your balance in Ada, but will also allow you to see your balance in common fiat currencies like USD, Euro and so on, and also other common cryptocurrencies like bitcoin and ethereum And lastly, I want to mention that we’re working on memos Sometimes, when you send a transaction you want to attach some metadata to your transaction, and do that out-of-band doesn’t knock through the blockchain, but through some data storage mechanism like Dropbox, and so we’re adding support for these memos in Yoroi If you’re curious about how we’re designing these features, and you want to learn more. We created a new GitHub repository called EMURGO Improvement Proposals These are specifications that we’ve written usually for Yoroi, that we thought are generic enough and useful enough that other companies in the blockchain ecosystem that are building on Cardano may be interested. You can see we have some specifications all over in there. For example, how to create account number plate, which is a very popular Yoroi feature, and we have some specifications that we’re still working on, like how to do message signing which is useful for memos and so on What is the Daedalus team doing? Obviously, they’re doing a lot of work. Mostly their work revolves around the v2 API, which is also in preparation for Shelley So what is v2? If you think about how Daedalus works. Daedalus is just the front end that connects to a node running on your computer The node that is running on your computer has an API to connect to it. The version that Daedalus uses right now is called v1. It connects to the v1 API The team that is working on the Cardano node for Shelley needs a new version that supports stuff like stake pool creations, stake pool joining. This kind of information and this is called v2 For Daedalus to connect to the Shelley version of the Cardano node you need support for this v2 API, and so if you look at the Daedalus GitHub repository right now, we have a huge number of pull requests all to do migration to the v2 API Everybody’s busy getting everything ready for Shelley and the Daedalus team is no exception. One thing

I want to mention also that they’ve been working on, is a new way to have numeric input If you ever tried to type numbers in Daedalus, there are some very subtle h cases we don’t really get quite the behavior you want, and that’ll be fixed. The Daedalus team has been spending some time on it But in the past month, the most important thing I want to mention is that Daedalus had a new release We’re now at Daedalus 0.14 and they have changelogs about everything has changed. I’ll have them in the YouTube video description So definitely check it out But they also have a video that goes through all the changes, and the changes are very pretty, very well done If you’ve used Daedalus and you want to check out the new look, the new UI and the better syncing Definitely give it a download, give it a shot and give it a try The Daedalus team is really looking for your feedback on this new UX and so if you have any feedback definitely give it to them through GitHub as usual What is the Plutus team up to all these Shelley things happening? Well, they’re still working on getting Goguen shipped. Step after Shelley is open, which is the smart contract integration in Cardano There are a few steps to it. One is making sure all the work we’ve done in Plutus is done correctly and that there are no flaws, and that means we need formal verification and that’s what we continue to be working on for Plutus. You can see that we have two pull requests for even more verification of all the Plutus stack. We’re not working just on verification. We’re also working on the usability in the feature set. To make sure that everybody can use Plutus from day one as soon as it’s available we’re working really hard to get documentation such as this new book on writing Plutus smart contracts, that’s published in the last month If you want to learn more about Plutus, how to write Plutus smart contracts Definitely go check it out. Read the book They have a lot of examples and there’s a website where you can actually write, compile and run your own Plutus smart contracts on a test network The next thing we’re working on is a book for Marlowe We have a book for Plutus as you see, and Marlowe is another smart contract platform for Cardano, that it’s meant to be much easier to use. You don’t have to be a developer to use it. Even if you have just a basic knowledge of blockchain and UTXO You’ll be able to create your own smart contract. And so to explain that to people we’re also working on a Marlowe book In that word kind of refining specifications that we’ve written Mainly because Plutus as you know, it’s still in development, so every now and then we have to kind of go back and change the documentation a bit to make sure it reflects all the new changes and we’ve also got two new papers by the Plutus team. One is called system F in agda, and the other one is a much more complicated name It’s the paper about compiling intermediate language with mutually recursive data types and mutually recursive functions into System F-omega, which ifix I know this kind of sounds like a bunch of random words, but I swear to you if you search up each of these independently they’re a liberal terminology used quite common in the functional programming space and the field of programming language design This is just the way that we develop in Cardano, we write the code we write the formal specification and we get those written into papers and publish them in conferences and journals and everything to basically make sure, that not only we’re convinced that we’re right but the rest of the academic and programming community is also convinced that were right And a part of peer review, of course, is taking feedback and integrating it into your product So after we released the Plutus book, we got some feedback about some parts that need to be changed, some parts that could be made clear We’re taking that feedback into account and helping to improve a book and will maintain the book over time as we make more changes in the language Obviously, the Plutus smart contract programming language itself is improving. We have some main changes that I want to mention this month The first one is work on a type system for transaction interface So basically, if you think about the way Plutus works, it has multiple steps of compilation Haskell is the top level, but as you compile you kind of go down to Plutus, IR Plutus core. These different representations that are used by a virtual machine to basically execute your code on the blockchain and

this pull request helps basically type check your Plutus core code using Haskell. So you can check out how that works in this pull request, how you can use it with the example Smart contracts that are already posted on the Plutus testnet website, if you’re a developer who’s interested in Plutus and type-checking, you can check out this pull request and I’ll have a link in YouTube description Another change we made is taking into account transaction fees. So on this test network we kind of simplified a lot of the transaction fee mechanism or remove them entirely just to make sure that we can easily test a smart contract system as opposed to having to test all the regular blockchain logic. But now we’re adding transaction fees into account across all the code and that’s what this pull request essentially does Obviously, we now have a lot better documentation on Plutus and we keep improving every month and the language itself continues to improve and we’re working on usability of the various to all APIs and the steps required to get the Plutus testnet eventually integrated into the actual Cardano mainnet Part of making Plutus usable is having a way to connect to it from websites and other off-chain components and so we have a GitHub repository for this. We’ve covered it in previous months Called smart contract back-end. A lot of work has happened in the past month all in smart contract backend You can read about the various changes that happen here, but essentially we’re trying to make it easy to run your smart contracts in a way that is statically type check and compiles directly from your Plutus smart contract When you write your Plutus code and you compile it, you get an on-chain component off-chain component You take the off-chain component, you hook it up with your website, with this library that uses typescript to basically make sure that everything that happened in Plutus is type-checked to your typescript implementation If you want to check it out and see how it works then check out this GitHub repository and see all the new changes that we’ve made to make the system faster, safer and just in general easier to use If you want to figure out how it works There’s a README in this aside this GitHub repository that basically explains how to get it all up and running with NodeJS and Docker So let’s talk about Formal Ouroboros So Ouroboros is the underlying protocol we’re using for Cardano, and as we’re implementing it in Haskell, in Rust, we need to make sure that the proof itself in the Ouroboros research papers are correct We continue to write a very formal specification for Ouroboros, and you can see a lot of pull requests have merged this month to basically continue working on this formal specification of Ouroboros that uses basically Isabel and a few other custom tools we’re in to verify that there are no bugs and the paper. All the proofs in the paper can be formally verified How is the Haskell team going? The Formal Ouroboros folks, the people working on this project are working very closely with the Haskell team That essentially has a pipeline to go from the formal specification all the way up to the executable code Let’s see what the Haskell team is doing the most exciting they’ve been working on is something called Cardano Explorer What is Cardano Explorer? Cardano Explorer is a new code base that essentially allows you to use the Shelley Haskell node to power and explore This is different from Seiza, because it’s not a website, it’s not a UI This is an API that you can easily use to power and explore on your local machines You can query information about blocks and transactions and wallets and all this kind of information So if you are a company who wants to build using the Haskell codebase Now you have an option for an explore API also. As I mentioned earlier in this video we have an explore API for the Rust codebase and this GitHub repository provides an explorer API for the Haskell codebase They have a lot of documentation about exactly how it works and you

can read through it, and see how it’s designed, how we use tools like Postgres to power this database for the Cardano Haskell Explorer This it’s not 100% done yet. Right now they’re still employing a lot of the endpoints for querying their Explorer It’s it may be early to build on it, but if you’re super keen on this project, definitely check it out, try and get it up and running Use some of the endpoints and maybe suggest some features that you think should be added So the Haskell team also has a project called Cardano wallet This is what I mentioned earlier that the Daedalus team is using Daedalus connects to the Cardano wallet which connects to the Cardano node, which runs in your computer which is the Haskell codebase, and so they added some new functionality for externally signed transactions, which is required for hardware wallets So the Daedalus team plans to release hardware wallet support. They’ve been working on the API on the JavaScript side for it. And this is some the equivalent work on the node side for it to all yet working As you probably know hardware wallets right now are currently supported on Yoroi, and AdaLite and the Daedalus team is also working towards hardware wallets support What is the Haskell networking team been up to? If you don’t know, Cardano is basically running a whole networking stack from scratch, and it’s basically required because blockchain is a very specific use case. And so we can’t really just use an existing library We have to do a lot of novel research, about exactly what do we need for networking stack on Cardano on a blockchain This project has been going on for a long time and I cover every month and this month is no exception We’re still working on, we’re adding some new features and a lot of it is in kind of three categories One of them is basic audits, memory and resource usage. We wanted to make sure that the memory usage of running the network on the Haskell code base is not too big, it’s a reasonable performance. And wanted to make sure that we’re easily able to handle stuff like multi-threading resource allocation and that’s what one of the main things that we’ve been working on. The other one is the transition from Byron to Shelley The networking layer needs to be able to sync Byron transactions, Byron blocks, everything, and smoothly transition into Shelley. We’re working on extending the transactions The transitioning the datastore on disk, so you can see it if you open Daedalus right now and you go to your Daedalus folder. You’ll see a bunch of files on disk representing all the blocks, so running system to convert the Byron mainnet blocks to the new Shelley representation We’re working to transition to Praos and Genesis in a step by step manner You might remember that Charles is talked about in the past how we do not want a Shelley to be kind of an on and off switch It’ll be a gradual step-by-step process, where 5% of the network will be decentralized, and 10% of the network will be decentralized We’re working on kind of implementing this transitional step and as always we want to do this with respect to the formal specifications we’ve written. Remember that the Haskell team is doing this very rigorously step by step. We had a formal specification, I showed you earlier and we have the executable specification, I showed you in previous months Now we have a networking layer which connects to the next step, which connects the next step, and every step is a check to make sure everything is working correctly As I mentioned the transitional prowess this pull request is because we will essentially switch the Cardano mainnet to use BFT and once we’re in BFT, we need to transition to Praos, where step by step we determine how many blocks are created by the BFT system, and how many blocks are created by decentralized Ouroboros Praos and Genesis And we have some other work basically taking care of technical debt to continue to get this Ouroboros network project ready for Shelley

What’s going on the formal specification side? As I mentioned just a few minutes ago The Haskell team is done step by step, and we have executable specifications, that say exactly how things need to behave at run time It’s different from a mathematical specification, because with the executable specification you’d only can verify that it’s mathematically accurate, but also that you can execute it and you get the right result We’re working on the specification for a multi-signature. Previous months we’ve been talking about multi-sig support for Cardano, and we had a few different proposals The two main proposals were implementing multi signature as a native functionality to the Cardano network, the other one was is implementing multi-sig as a functionally on top of smart contracts, on top of Plutus, and then we will ship part of Plutus inside Shelley We had a long discussion about both sides, the benefits, the disadvantages, and it sounds like the teams have agreed that we will move forward with native multi-sig scripts That means that will not be shipping part of Plutus as a part of Shelley. Rather multi-sig will be natively included inside Cardano, inside the core protocol If you want to figure out exactly how multi-sig works, you can now read the specification for native multi-sig and we’re much more certain that this is exactly how to be implemented You can start looking into how to implement this for your wallet, that supports Cardano. What else have we been doing on the formal specification side? Cardano’s a step by step process and we need to make sure all these steps are in sync. So anytime we change the mathematical definition, we need to make sure that this change is reflected in the formal specification, reflected in the next step, reflected in the next step So it’s kind of like a cascade The Leger specification team has a busy two past month basically making the formal specification match the executable specification. I show you three pull requests right here, but a lot more, where you’re just making small changes here and there to make sure that every part of their code matches the original formal specification for Cardano That’s it for a code and I want to go over some other news before we wrap up this video One of them is that EMURGO signed an MoU with some trade associations They’re supported and proved by the South Korean government Notably, our two partners are the Korea Blockchain Contents Association and the Korea Mobile Game Association We have a press release to talk about how we plan to work with them for the respective fields. We’re really excited that companies and associations are working with EMURGO to help leverage blockchain technologies in their respective domains Another thing I want to mention is Runtime verification published a paper about IELE. If you don’t remember IELE is one of these smart contracts languages, that IOHK helped develop, health fund. That’s a way to basically firmly specify a virtual machine, where you can use these formal semantics to easily write smart contracts and prove that your smart contract is secure. They recently published a new paper about IELE, and the work that IOHK helped fund If you re interested in IELE, definitely check out this paper, that explains kind of a summary of all the work that’s been done so far IELE will probably node ship with the first version of Goguen We’re still very close friends with Runtime verification We follow all the work we do and we’d love to see this included in Cardano one day I’m happy to see them kind of summarize all the work that they’ve done so far, and we’ll definitely be keeping track of the work they continue to do. And hopefully implement all this technology, this awesome technology into Cardano in the future The last thing I want to mention is that IOHK has been doing a lot of Cardano development progress demos to help keep the community informed about exactly what’s happening with all the products So if you want to know exactly what the Haskell team is doing. For example, in this video how the network team is storing blocks and reading blocks from a disk in this Ouroboros network project I just mentioned, they now have a 20-minute video that explains exactly how it works

If you’re interested in very deep details, you want to know exactly how it works Definitely check out these demo video that IOHK has been working on recently. So this is the second one. I think they had the first one late last month, and will we be having more of them in the future. It’s not just about sharing the technology we have with the rest of the ecosystem It’s also about transparency and making sure everybody and the community sees the work they were doing and that’s very important. Speaking of talking to the community. The Cardano Foundation recently appointed Alex Park to lead global community expansion. The Cardano Foundation has recently expanded in China and I think some other countries They’re basically getting something to help lead this growth effort, this expansion effort and Alex Park is helping them do that Welcome to the team, Alex That basically sums up all the work this month, huge stuff has happened We’ll be having a lot more in the month of September So definitely follow us on Twitter, on YouTube, all the platforms and stay up to date with this exciting project