Continuous Integration (CI) with GitHub Actions

– Hello everybody, thank you for joining We’ll wait a minute more for everyone to join Alright Hello everybody and thank you for joining My name is Philippe Ozil, I’m the Principal Developer Evangelist at Salesforce And today, I’m going to be talking to you about continuous integration and most importantly about GitHub Actions So, for today’s agenda, we’re gonna be starting with a few things And before I jump into the presentation, I need to give you the Forward-Looking Statement As always, we are a publicly-traded company, so please make your purchasing decisions based on what is commercially available in the product Again, I’m not so much gonna talk about the product as in my previous sessions, I’m more gonna talk about GitHub So, that should be another problem So, today’s agenda is pretty simple I’m gonna talk a lot about GitHub Actions, and then I will talk about CI with Actions, and this is most importantly in the context of Salesforce DX projects And if we have time, I may also add a few things regarding other types of projects So let’s start Before I go down in the details of the different types of tools that we use and the different concepts, I wanna give you the high-level view And the way GitHub Actions work is by chaining a couple of different things First of all, you have to understand that everything happens in the context of a repository So GitHub actions are working for a single repository at any given time Now, it all starts with an Event within the repository, so there are multiple Events, I’ll get back to that later but imagine, for example, a new poll request This particular Event triggers a workflow So the Workflow is configured to listen for particular Event type And this Workflow execute some jobs, and these Jobs will execute some Steps, or Actions And the orange part in this slide represents GitHub Actions GitHub Actions has several meanings It can be the whole technology, what is represented on this slide, but it’s also the low-level artifact at the most granular level, steps, basically So let’s now take a look

at all of these different types of artifacts Starting from the lowest level, starting from actions or steps Basically, this is a single unit of work And what this means is it can be something like sending out a command, writing a comment on the GitHub message flow It can be also modifying source code or running a test It can be starting a VN, things like that And in the context of Salesforce DX, it can be creating a scratch org, or pushing some code in there The idea is that this is just a single thing, that is, a unit and it expects some inputs and outputs something that the next action can also leverage There are multiple languages that are supported to write Actions These can be written in JavaScript, TypeScript, Python, Java, or even simple Shell script which is very convenient And all of that runs in a darker image of a Linux system That’s for technical part I don’t expect most of you to write their own GitHub Actions but it’s always interesting to know what happens behind the scene And talking about them, you can actually leverage a lot of these Actions without having to create your own There is a marketplace on GitHub that has hundreds, if not thousands of different Actions that you can leverage And you can also, most interestingly, link to a repository to directly load an action from a repository So if you want to try your own custom Action, it’s very simple You don’t even need to share it with the rest of the community, you can just have your own Repository, and link to that repository to trigger a new action So again, an Action or a Step, which is the same thing in this context, is part of a bigger entity, which is called a Job, a Job has multiple Steps So what is a Job? Well, a Job is an ordered set of Steps, with some of them that are optional So, the basics is that your Job will execute Steps, one after the other, and if at some point there is a failure, the Job terminates And some of these Steps will normally be skipped, but some can be marked as always executed, that’s basically the way it works Now, V Job works also as a unit in the sense that it is working in isolated container So all the Actions that run in the Job can communicate with each other, but one Job cannot communicate with another Job So that’s really important This is a specific docker image that runs a Job, so it’s really a unit Now, a Job is also part of a larger entity, which is called a Workflow Workflow is a top-level artifact for GitHub Actions and it contains a set of ordered Jobs Now these different Jobs will execute also one after each other, and then they can have dependencies So, for example, Job number two may require Job number one, or it may not Some Jobs are allowed to fail, and the others will continue to execute How is that configured? Well, it’s all happening in YAML file, .yml But it is located in the same directory for all repositories, it’s .github/workflows And I will give you a little tour of one of the examples of such a configuration in a few minutes So, it’s in there, but there can be more than one Workflow But these Workflows are separate, they don’t communicate with each other again And in VN, after a Workflow has run, there can be only a pass or fail status And this is something that is reported in the GitHub user interface And there are multiple places where you can see the status of a Job, and also get back to that later You can also cancel a Job which is running, that’s also another status that exists Now, back to the real beginning of the story, what triggers the Workflow is a GitHub Event And there are multiple types of Events that can happen, but you can also filter on sub-properties of those Events, and let me get back to that now So these Events happen based on GitHub activities, things you’re actually doing, or your users are doing, or even things that are automated that are happening on GitHub For example, these can be Poll Requests, Issues being open, or Branches being pushed, and deployment of your Source Code, for example Now, given those different types of activities, there’s another level of GitHub activity that you can use to filter in your Workflow, the trigger of the Workflow So for example, for Poll Requests, you can filter to only file your Workflow when a Poll Request is signed, edited, or a review is requested on a poll request And another example, for example for Issues, you can trigger an Event when the Issue is opened, edited, or closed There are many more different Event types, and the list is actually quite rich

There are a large number of things you can do, it’s really amazing Now, let’s take all of that and apply this to a Salesforce DX project, to bring about more context with things that you’re most likely familiar with So, I’ve taken the same Color Code here, and what we’re looking at right now is a CI environment So the CI environment that we have here is a Workflow When this Workflow is configured to listen for push, so when coded, adding push to a Branch, it fires the first Job The first Job is something that you would need if you were running lining node components in your project It contains five different steps, and I’m gonna list them out The first thing it needs to do is to check out the Source Code So, remember that the Job works on a darker image, so it’s a stand-alone thing And it does not necessarily know about the Source Code So you’re gonna run first a Git command to check out your current Source Code Of course, there are helpers of that, and this is nicely packaged in one of the provided Actions So that’s the first step that you need to perform, and this is probably gonna be true for all of your Actions, checking out the Source Code Then, since this is a lining with components project, we need to actually to install Node JS Most of the tasks that you can perform on the development side of things for a lining with components project require Nodes So we’re gonna install Nodes and several dependencies And these are gonna be dependencies we’re gonna be using in the three upcoming steps Next step is to verify the code format So that’s something that we tend to do, for example, in the sample applications that are provided on the Trailhead and I’ll get more in details on this configuration What this does is actually it lines the format with a specific set of rules So it’s making sure that the formatting respects those rules At any point, these different steps can throw in an error or basically return with a non-zero exit code And this will actually fail the Step, which will in turn fail the Job, and halt the Workflow So assuming the format is okay, the next step is to lint the JavaScript code with a tool called ESLint, which is pretty much of a standard for linting JavaScript file It’s something that is owned by the community And final step is actually running your JavaScript tests, and we are using Jest for that, to run lining with components test So, this can take a bit of time, but it doesn’t really matter, all of that is isolated Now, there’s a rule that says that a Job can last for up to half an hour, if I remember well, something like that so you have a lot of time here And this is probably something you can work with GitHub, if you have a paid-for subscription So that’s the first Job Once the first Job is completed, and only if the first Job is completed, we’ll move on to the second Job, which is a bit larger The second Job I call scratch-org test So it’s gonna do several things, it’s gonna make sure our Salesforce DX environment can be deployed, and then we can do a couple of things with it So, remember that this is a different darker image, it does not know about the previous Job, it doesn’t know about the previous context All it knows is that it’s being triggered after another Job So new Job, this means that we’re gonna have to check out the source code, of course But we also have to install the tooling to work with Salesforce project So this means we need to install the Salesforce CLI, or Salesforce DX So this is basically done with a Shell script, so we’re writing a Shell script, which will download an image of the Salesforce CLI in Styra, and run it Then we check out the Source Code, we authenticate with a Developer Hub So because we’re gonna be using scratch-orgs to run our CLI environment, we’ll need to need to authenticate with the Developer Hub first The Developer Hub lets us create scratch-orgs And here, we’re using something that I haven’t written on the slide here, but we’re using action or we’re using Secrets, sorry And Secrets basically are something that allow us to hide certain parameters from the configuration overflow With an Action you can patent certain settings, like the authentication URL for your Dev Hub Let me just hide my head now from the next part because you probably wanna see it So let me just move to this screen only There we go So after you have authenticated with the Dev Hub,

you’re gonna run a command from the CLI, which lints your Aura Code So it’s basically making sure that the Syntax is okay, that you’re using the right settings Then you’re gonna create the scratch org So, starting from there it gets interesting because you’re now okay to access the org With that, we continue with the other steps that are still part of this flow, I had to return line just to put everything here But what we’re doing here is that we’re not pushing our Source Code using the CLI commands, we’re maybe assigning some permission set, importing some sample data, all of that with CLI commands Then we run the Apex test commands So this is probably what will take the longest in all the entire flow But this is what is the most critical that your Apex Code works on scratch-org And then finally, the last Step or last Action here is to delete your Scratch-org at the end of the flow Now, this is something that you wanna do always to make sure you tidy up after finishing your CLI flow So this particular last Action is Jerry-setup to always run So that even if you had a crash in your Apex test, for example, or your org, in lint, you would just jump to the last step and still execute the Delete scratch org instructions Another option, of course, is to simply set the scratch org over a very short lifetime Back to the slides here Now, I suggest that we take a look now at some actual code So we’ve seen Actions, sorry, we’ve seen Actions from a theoretical point of view Let’s now take a look at some Source Code, see how this runs So as an example today, I’m gonna take a simple product here, I’m gonna take one of the applications from the sample gallery And I’m just gonna take the first one called LWC Recipes This is available on GitHub and all that I’m gonna show here today, can be seen from this particular application You can see here there’s a little View on GitHub link here, which takes us to the GitHub repository of this particular project Now, when you look at the project, if you scroll down, you’ll see those little badges here These are tied to a GitHub Workflow Right now, I have a little problem with my packaging flow here so it’s failing But we’re gonna be looking at the Salesforce DX Workflow, which is the one that runs the test and runs linting and everything So these badges are a dynamic link here, which reports the status of our flow If I click on it, it will take me to the Actions interface So this is something which you can also get from the top level tabs here, or from the repple, Actions Now there are multiple flows that are running in this particular project, it’s a bit advanced But the one we’re gonna be looking at is the Salesforce DX scratch org workflow So before I continue with the user interface here, I’m gonna jump to the Source Code, and we’re gonna see how this is configured Let me just increase the font size a bit so that everyone can see, I’m just gonna reduce that So we’re going to be looking at our .github/workflow folder And there you can see I have a couple of yaml files So let’s start maybe with a very simple flow that we use here called branch-naming, this is really a simple, simple flow here And this is something that runs all the time except for the master branch So the way it works is that you name your flow here, this is what will be reported in the user interface You then have a block here, which lets you specify on which Event the flow runs So this is happening each time there’s a pusher on branch And then you have some sub filters that you can apply for this particular type of Action In this particular example here, we’re ignoring the master branch So basically this Workflow here will happen each time there is a push on any branch except for master You can also have filters that work the other way around, but only include specific branches Now, after that, there are the list of Jobs This particular Workflow only has one Job here, which is validate branch name And this particular Job here is running on the Ubuntu Linux image, simple And then there’s a set of Steps here Again, it’s a very particular example, there’s only one Step in this one Job Now, this is a GitHub Action The Action here is coming from a GitHub repository

And you can see here, this is the username, the name of the repository and this is after that version So this is taking the latest version of the Action branch name from this particular user Here, we can supply a name so that we have something that is reported in the user interface And we can pass in some parameters to this particular Action So we’ve been looking at basically the readme from this action and specifying the right values So what we expect here is basically to have something I need to explain to you about the regex here but we expect to have something with characters and a slash and another thing with character This is because our branching naming convention is for example, my username slash the task I’m doing, So there’s a minimum length and certain the rules This is just to keep things tidy because we have quite a few Poll Requests This is really a simple example but this is not yet tied to a specific Salesforce DX project So maybe let’s now take a look at a Salesforce DX project So we’re gonna be looking at this particular product here called scratch-orgs-sfdx-ci-master This is something that runs each time there’s a push on the master branch for the Recipes Workflow So you’ll find some similar patterns What is interesting here is that, well, quite a lot of steps on this one, this is pretty complex And this is gonna be something similar to the slide I showed you earlier with all the different steps So let me just compress this a bit so that we can read it in an easier way So the name of the workflow is just Salesforce Dx scratch org And this is what you saw in the badge that I showed you in the readme This is happening each time there’s a push on the master Branch The other Branches are not targeted by this specific flow And you’ll see that we have other overflows like the Pre-release flow which are running on the Pre-release Branch Now this flow now has two Jobs, formatting and linting, which also does lining of components test, by the way, and the scratch org tests And so these are more or less what I showed you earlier in the slide So, we’re gonna go over these things very quickly here But first thing is we check out the Source Code We even load some cash for Node JS installation So this is just a shortcut basically, to install Node JS project very quickly With these two Action, we’re just restoring the previous install that we had with a cache mechanism I won’t go into the details of that particular stuff, it’s a bit too advanced for now but if you’re interested, you can look into that It’s a way to quickly install Node JS to save sometime Now, there is the rest of instructions which is really more important for the for the flow We are running Preacher And this is using a Node command So here in the run, this particular Action is just a Shell Command This is not a GitHub Action in the sense that there is no repository tied to that particular action It’s just running a Shell Command So this is very different from something like this here, which is actually a custom GitHub Command, GitHub Action, sorry So you see the use here means that we’re calling an Action from the repository But when it’s just run, this is just doing some Shell Script So if you actually do these things on your own computer, you can run those things provided that you have no JS installed then you’re in the right project So these are actually calling scripts that are defined on the project And these are defined here in the package JS So very quickly, there is a script section in your package json file, which is the node configuration file which contains a list of different scripts And doing NPM run with the name of the script, like lint or test will run those commands So this is what we’re doing basically with the commands that we’re doing, like, run Preacher, Verify So now we have verified our Source Code with Preacher, the format is okay, we’re doing some linting with ESLint, same thing Shell Script command, but calls no JS So NPM for no JS run, and then JS script name And finally, again, one last thing, we’re running the unit test with Jest and we’re also reducing the coverage argument here to run a test coverage for lightweight components Now, if this first job works, fine Let me just minimize that If formatting and linting works,

we’ll be able to start the second Job called Scratch org test And you see what this line here means that this first job needs to validate before we run the next job And again, remember, this is a whole different environment, whether it’s another image, but it’s being spun up, there’s no link to the previous Job, it just needs to run after the other The Installation Command that I mentioned earlier to install the Salesforce CLI So this is just a Shell Command again, but combined with, actually it’s more than one Shell Command It’s a set of different Shell commands that are chained together inside one Step Checking out the Source Code and setting up the Salesforce DX Dev Hub authentication with a Euro Now, I wanna bring your attention to this particular Syntax here This Syntax with dollar in double curly brackets allow us to read values from Secrets And Secrets as a user interface within GitHub, which lets us add configuration variables, which are kept secret, so this is replaced by a URL, which is uniquely allowing me to authenticate with my Dev Hub So this is being injected in here and saved in a text file called DEVHUB-SFDX-URL And we’re gonna use that in the next command here to load that particular DEVHUB_URL and store it in our Salesforce DX environment So with that, we actually authenticated with the DEV HUB and we’re saving it as DEVHUB alleys Then a few more commands, linting our components, creating the scratch org, pushing the source code to reorg, assign the perm set, importing sample data, running the Apex test with code coverage and finally, important, doing some housekeeping by deleting we re-scratch-org Note the if always means that this Action will always, or Step will always run, even if one of the previous steps fails, that’s important to make sure that we always clean up our mess after we’ve finished with the scratch org That’s a good practice So that’s how the flow works There are many more examples that I can show you And in particular, we’re working with those GitHub Actions to do packaging as well So when we’re actually releasing a new version, there are other scripts that are running in here, like this one here that actually do the same thing we saw earlier, but run on master and are triggered when the first app folder changes And, for example, what is different with that one is that there is a packaging which is being set up And we’re using our other commands to actually create a new package diversion And you we’re sending that package version later on in the readme So we’re extracting, it making sure it propagates and we’re saving that in objects configuration and pushing that to readme But this is a bit too advanced I’m not gonna go more in details on that one Do note that everything in here is public So because the lightning web components, recipes, application is public, you can actually find all of that when doing on the repository Going in .github/workflows in here you’ll find the files that I’ve been looking at, just in my JS code environment And if you want to follow one of these flows, you also have access to this action tab here again, which lets you see what happened in the execution So the last comment we made here was this one And this was made by my colleague, and the flow ran on master So let’s check out what happened on this one This was running the Salesforce DX workflow That’s the one we saw And you can see here the two Jobs When you’re clicking on the Job here, you can see the list of different Actions And if you wanna see more details that will be checked as they go But you can also expand them to see the output of the different commands Now, most of the time on the commands, Actions, sorry, are successful, they don’t output anything And finally, that second Job that was run safely, you can see all the Steps that are being reported as fast even up to the point where we delete the scratch org, or where we run the Apex test That’s gonna be more interesting So this is exactly what you would have in your terminal if you are running those things except GitHub actions lets you automate all of that So that’s very convenient So that’s it for the overview of GitHub Action Let me get back to the slides now Okay, so we saw that A summary, we saw that GitHub Actions allow you to automate quite a lot of tasks

The examples I showed you are very specific to CI, but you can do more than that For example, you can automate the processing of Issues, for example If you want triage issues automatically, this is something that can help you You can, for example, automatically assign labels to Issues whether there are questions or bugs, for example And you can also do some things like sending you messages, because you can trigger whatever custom code you want from GitHub event There’s no really no limit to what you can do Second point, GitHub Actions are ready code driven So all that configuration happens in that yaml file we saw earlier There are a few things you can do, of course, with Secrets, that lets you add configuration, but almost everything is done without clicks And that is very convenient, because, for example, if we take the way we’ve configured or sample apps, what we’ve been mostly doing is aligning our CI workflow environments by just copy and pasting this workflow directory over to the next apps, and then fine-tuning a couple of those Steps But that’s super powerful because you can start a project by just copying an existing CI configuration, and it will work in the context of that project Finally, I think you saw it, this is really extensible I haven’t even scratched the surface on the list of Events that you can use But there are already tons of those and you have access to the entire GitHub API’s in the context of those Actions that you can really create your own Actions, you can share them with the marketplace, or even keeping them for yourself if you wanna link to your own adversaries So I really suggest that you give them a try, take some time to look at the Source Code from the sample apps, inspect it and understand a bit how it works It’s really powerful We’ve tried different CI solutions before and I find GitHub Actions is really the simplest and the most convenient because there’s almost no setup required So that’s really interesting And there’s a very large community also, lots of different Actions that are available from this marketplace That being said, if you’re interested in learning more about the different Events or different configuration options for those Actions, check out the documentation site from GitHub Actions, it’s very complete lots of examples And we’re also quite a large community trying those things out so it’s quite easy Also, if you want to read a bit more in the context of Salesforce, my colleague, Rene Winkelmeyer recently published, I think it was in January, published a blog post on GitHub Action So if you go on the website, for the blogs, you can find Best Action blog by searching Finally, again, reminder that all the Actions and Workflows are configured and are publicly visible in the sample app, so make sure you check those out, That’s it for me I’d like to thank you for joining and I hope you had a great time and you learn new stuff Do give it a try And see you next time Thanks again Bye