How to set up a secure environment on Azure Government – Part 2

>> Hi. This is Steve Michelotti of the Azure Government Engineering team Today, I’m joined by Joseph Bloom of Microsoft Business Productivity, as well as Paul Fisher, the Microsoft Modern App Solution Center Welcome, gentlemen >> Thanks >> Thanks >> This is actually part two of a two-part video where we are talking about setting up a secure development environment on Azure Government In the first part, we talked about setting up that development environment, we set up our VPN connections, our Vnets we set up our secure environment basically This part, Paul is going to show us how to use it >> Correct. As in the first part, we’ve set up the secure environment In this part of the videos, we’re going to actually create a build server, actually a pair of build servers We’re going to run automated unit tests and integration tests as well as deploy infrastructure two Docker Containers during the run It’s just to show how to actually use that secure VPNs that all of your resources are behind the secure firewall >> Okay. So, you were using Docker even for things like SQL? >> Correct. While there’s other technologies that you could use for integration testing such as SQL Azure, the goal of this example is to show how you can set up something in VSTS, run your automated build in integration testing and then transfer it to a secondary environment for deployment, which may be disconnected environment that doesn’t have reached back into Azure or SQL Azure The goal will be to create a pair of build servers that will run unit testing and integration testing inside of the VPN In a production environment, you may have other machines where you’ll run your integration testing For this example, we’re just going to use to build servers However, the build server will authenticate the Visual Studio Team Services, they will bring the code back and they can run everything inside of the VPN security environment You may have other pieces that you want to run outside of the security environment but that’s really by your choice Your security concerns will dictate which pieces are allowed to be outside of the window that we created in part one >> Okay. So, you just cleared up a couple of things for me First off, one of the questions I had was why not use SQL Azure? But I think you just answered that I can literally take this and put this anywhere I want I can even deploy it into a classified environment >> Absolutely. When I’m done and the example that you see, the builds up run in VSTS, we run almost identically in an on-premises TFS >> Okay. The other thing I find interesting about this diagram is we think of VSTS, Visual Studio Team Services, as something running in the cloud, a SaaS model, but in this case, we’re actually pulling the build down to our secure environment We’re running the build It’s not up in some cloud somewhere but in our secure environment >> Correct. By default, Visual Studio Team Services will run in a hosted environment that’s hosted by Visual Studio Team Services However, once you set up your VPN network and your secure environment, your integration tests could take quite a while The whole discussion that we had when we were working on this was, “Well, if you’re running these long-term integration tests, is the code vulnerable while it’s being built?” I mean, if your integration tests take an hour or two, what occurs during that time? So, the answer that we came up with as well, the best thing would be able to do is to bring everything into your VPN and run your integration tests there This way you don’t care if they run for one hour, two hours or six hours Nobody without VPN access can access the build while it’s running >> They’re going to be in that secure environment in Azure Government I also noticed you have a pair of build servers Why do we have two here instead of just one? >> So, we’re going to build the build servers in an availability group So, an availability group is an Azure construct whereby if certain machines are taken down for maintenance or other things inside the cluster, Azure guarantees of the other one will not be taken down for maintenance at the same time So, by putting them in an availability group, you’ll always have a build server whenever you need it >> Okay, sounds good So, what’s the first step here? >> All right. Well, the first step is create and build servers All right, so here we are in my Microsoft Azure for Government dashboard So, I have logged into my account using dual-factor authentication which I strongly suggest that everybody does I actually logged in using my Microsoft Authenticator app on my phone So, now, I’m into the dashboard environment I’m going to just create my build servers So, I’m gonna go over and I’m going to click on “All Resources” I’m going to search for Visual Studio Now, the build servers running out of VSTS run VSTest as the test engine They require Visual Studio to be on the build servers I’m going to use a Server 2016 instance that already has Visual Studio installed, but even if you installed it separately, you would end up being about the same thing >> So, here we are installing this for the purpose of running the build server builds >> We’ll run the builds We’ll run the unit tests and it will run the integration tests all automatically >> Awesome >> Then, you can take this and you could use this as a check-in builds so that every time one of the developers has code that’s ready to be brought into the master baseline, this can be drawn automatically and all of your tests can be run in the code to be checked out This is a really good way to stop defects before they infect the rest of your baseline >> Sounds good, kind of a standard CICD pipeline >> Yes. So, I’m just creating a machine and there’s nothing particularly special about this at this point

It’s just a regular old VM >> So, here we’re just getting an example of how easy it is to set up a VM We can do this on the command line or in the portal But here we are just doing it in the portal filling up a couple of text boxes >> Correct. So, I put in a new resource group I’ve created a password and username I’ve given the machine the name It’s then going to ask me to choose a size Now, the size that you’ll want will depend on exactly how many build you’re gonna have and if you’re going to do integration testing on the machine versus another machine I’ll just pick the default size for now But really, each of the machines list the amount of RAM available and the number of data disks you can have So, you can scale this up to whatever size you need >> Okay >> Here I’m going to bring into an availability group So, both machines are going to get the same availability group So, like I said, this will make sure that one of the machines is always available 24/7 for your use I’m going to set only two update domains because I’m only gonna have two machines Okay. So, now, I’m going to pick a virtual network, and what I’m gonna do is I’m going to pick up the virtual network that Joe set up in part one of this video, that is the VPN network That will give me a default subnet which is what I want because that’s what we did in part one of these videos The key thing that I wanted to go down the road on the screen is to show you that for public IP address, we’re going to make sure that there’s no public IP address connected That’s because once you’re connected to the VPN, your local machine that you’re connected with and all of the machines in the VPN in Azure are speak only through their private IP addresses If you were to put a public IP address on the machine, you would skirt all the VPN functionality and you have no security at all So, in order to do this, we make sure that the machines have no public IP address >> Great. A completely locked down here >> Then I’d pick a network security group, and I click “OK” Now, in reality, I’d have to add this, would have to build and then I have to build the second machine So, I’m gonna switch over to a pair of machines that I already built In order so that we can continue on So, the first thing I’m going to do actually is I’m going to go to my VPN that we set up earlier I’m going to connect with my certificate that I was given >> Okay, so this is a certificate that we got from Azure, that we saw Joe demonstrate in the previous video and here we’re connected to our security development environment >> Absolutely. So, once this is logged in then this machine here will be part of the private network in Azure that we created in part one >> Awesome >> We are connected. So, now, I can come over here to the open one of my machines Now that I’m connected, it will just ask me to log in just like regular old RDP >> So, you are remoting into a development machine or the build server? >> Into a build server >> Okay >> But it’s really the same thing effectively It’s a machine running in the single segmented in Azure So, here I am, I’m on the build server and there’s five things that we need in order to get this accomplished today The first is Visual Studio Like I said, I’m using a template that has Visual Studio installed or if you want you could install Visual Studio after you end up in the same place The second thing I need is Docker Docker doesn’t come by default in Windows 2016, but it can be installed through PowerShell That takes a little while, so I’ve already done that in this case I do want to show, however, something that’s important is that when you install Docker by default, it’s going to install within that network That network is great, and there’s no problems However, each machine by default is going to get a different internal subnet for Docker What you want to do because we’re going to have one single build as you want both machines to have the same internal subnets There’s many ways to launch Docker, but if you launch Docker just through the regular Docker run command, you’re going to want to specify the IP address that your SQL Server is running on In order for each build to have the same IP address for that SQL Server, depending on which build server actually picks up your build, you want them to be addressed the same, and I’m going to show you how to do that right now >> Okay. Great >> Okay. So, here I’m gonna launch PowerShell in order to change the Docker network I’m going to do this as an administrator because Docker only runs as an administrator You can see Docker is running on this machine >> All right, you have already got Docker installed >> I have already got Docker installed So, the first thing we’re going to do is we’re going to check the Container network It’s telling me that my Container network is using an internal subnet of 192.168.224 So, what I’m going to do is I’m actually going to copy that value I’m going to put it outside in my machine and it’s going to put it into Notepad Because what we’re going to do is now we’re going to go on to the other machine in the group and we’re going to change that subnet to match this subnet Okay, so here we’re on this other machine,

we’re going to come we’re going to an Administrative Console >> This is the other Build machine? >> This is the other Build machine >> Okay >> In a pair >> Because what they do is they both are Docker installed but they have different internal slope >> Yes >> Right. Okay so the first thing I’m going to do is I’m going to stop the Docker service, and then I’m going to remove the container network Now, since this machine was just installed, it only has one container network So I don’t need to look and see what container network it is, it will install the default container network that Docker installed when it was installed Okay and then I’m going to open up Trustee Notepad as an administrator, and I’m going to enter the configuration for a static subnet Here, I’m going to take the subnet that I took from the first machine and I’m going to apply it to the second machine That should be good, we’re going to save this as, which is one of the main configuration files of Docker But since it was a new Docker installation, it was not there yet Now we go back to our PowerShell script and we go back where we started the Docker service and we get the container network We ensure that the container network not matches what is on the other machine So now both of these machines have two specific subnets on each machine that match that were in one Build that can go to either servers >> Okay >> Okay, so now that we have Docker all set up So now one Build can run on either of the Build Servers the next thing we need is the VSTS gent VSTS agent runs and it actually pulls out to VSTS in order to pull down the source code and run the builds and the tests This is how it works very securely because VSTS never gets credentials into the VPN, VPN as credentials into VSTS and this is accomplished through a PAT token >> All right >> So, I’m going to show you how to get the Agent and get the PAT token So we’re going to go to VSTS, now this is a private VSTS account that I’m using where I’m disabled dual factor authentication just for this demo But I highly recommend that people do not do that and they always access VSTS through dual factor authentication, which of course log in to in their phone or whatever they’re using for the dual factor The first thing I’m going to do is I’m going to go over to the “Agent Pools”, and right away it tells me here’s where I can download my Agent So, I’m going to download the Agent and it will actually give me instructions on how to download the Agent I’m going to follow these here but we’re going to download the package, we’ll save it Now while that’s running, you’ll see that there are already some agent pools listed on this These are for other machines that are using the same VSTS So, you can have any number of Build Servers and you can have any number of agents running on a Build Server So, depending on the capacity that you need in your environment, VSTS will fully support it Okay, that has been downloaded and now I’m going to close this and we’re going to come over here to “File Explorer” and we’re going to create a directory called Agent Then I’m going to go into my “Downloads” folder where I’ve just downloaded it I’m going to copy all of that into my new folder called the Agent >> Okay. So, we have downloaded the Agent from VSTS onto the machine we want to act as our Build Server You’re copying the files over still in preparation for actually running this Agent >> Correct >> All right >> Okay. So, now that the agent is copied, before we start the agent installation, we’re going to go to VSTS and get our PAT key So, the PAT key can be gotten if you go to your Name, the little icon for your Name and you click on “Security”

it will bring you to the “Personal Access Token” page So, what we want to do is we want to add a PAT key We’re going to type a description and we’ll just call this todaysdemokey, we pick our exploration, the PAT can be good for up to a year Then we will pick the scope that we wanted in The minimal scope that this needs in order to run a build Agent is Agent Pools(read and manage) So, we will then create the token, the token will appear on this page Now it’s important at this point that you copy this token because you cannot regenerate the same key again from the page and once we leave this webpage, the key will actually be gone The key is given by VSTS So, we can actually take this key and apply it to multiple agents like it’s basically giving the Agent permission to attach to the VSTS So, once you take one PAT key, you can put that PAT key to multiple agents or you can create a new PAT key for each Agent however you want to do it We will save this into Notepad for now. Yeah >> Okay. So, what do you do when the PAT token expires? >> When the PAT token expires, you’ll have to go back into VSTS and just generate another one >> Okay >> You can easily remove the Agent and reinstall it, the whole process takes about five minutes >> All right, cool >> So, let me show you how to install the Agent So, here we are, we’re going to go to the “Agent Directory”, and we’re going to install the Agent It’s going to ask us for the URL of VSTS that I was just on It’s going to default to PAT authentication which is what I want Here’s where we will copy the PAT token into the installation Now it’s going to ask for the Agent Pool, an Agent Pool, you can have multiple different pools or run different build agents we’re just going to use the default pool It asks for an Agent name and so I’m going to call this my demoagent It’s asking for a work folder, I’ll just hit the default for this and it wants to know if we want to run the agent as a service We do, so I’ll click “Yes.” Now it’s going to ask for user account In a production system, you’d probably want a service count to run this right now I’m just going to use me I’m going to run this There you go our Agent is now installed Now, if we go back to Visual Studio Team Services and we go back to the “Agent Pools” page, you’ll see now I now have a third agent, all right This is the one that we just installed So this is now ready and if you went to build, the build would be picked up and by one of these three would run The other things we need on the machine in order to run is we need a Docker container with SQL Server on it Lots of times during our own engagements, we actually build these containers as we go So the build step will run, build a container and then use that for testing SQL Server takes a bit of time to build and so, since it doesn’t really change, it’s easier just have it around We also need the Microsoft SQL Server command tools package because in the build on the fly, we’re going to install a database using “SQL” command and “SQL” command is deployed with that package So, for this I actually have two other machines already setup as Build Servers that have all the pieces So, what I’m going to do is I’m going to disable the demo agent, because we don’t want any builds we picked up by the one that’s only half built We’re going to go I’m going to show you how to Build actually runs >> Okay >> So, if I go to Projects and it’s called Build and Test, and if I go to “Build and Release” I have a single build and let’s just walk through it really quickly You’ll see at the top of the build I just remove any old Docker containers and I build the solution This will build all of the code, make sure it builds cleanly and it will build the database which will create a database installation script I then run my unit tests because the first thing you wanna do before you run your integration tests just make sure that your code compiles and runs and all the unit tests run as you think Unit testing normally is much quicker than integration testing so if the built is going to fail, you’d want to fail quickly After that, we’re going to copy the SQL files from where they’ve been built into where we need them to be in order to deploy from SQL command Then we’re going to run the SQL container I’m just using Docker run inside of a build step for Docker I’m actually giving the container an IP address so the container will always come up with the same IP address I do sleep for 75 seconds after this because we’ve noticed that SQL Server after the container comes up,

takes about a minute to become available So this is just a built-in way that we need to do After that, we’re going to deploy the database There is a step here where we have a configuration file for the integration test solution which is on the machine already So we’re going to copy that over the configuration file that came out of this code baseline Then we’re gonna run our integration tests, they run, they’re successful, the containers are destroyed, the database is deleted and we’re done and presumably we have perfect testing Running this is very easy I’ll just come over here and I click “Queue”, it says that my build has been queued and the build is just running >> Okay. So, we’ve gotten the build, actually building in Azure Government rather than upon VSTS that ia really in our secure development environment >> Correct >> Okay, cool. So, to summarize what we’ve done here We have these two parts of the video series and the first part we set up the secure development environment which included our developer workstations as well as the VNet that we need to run the builds In the second video here, you’ve walked us through actually setting up the builds So, we’ve installed our VSTS Agents in Azure Government So the Builds that actually run not on public VSTS but actually inside of Azure Government in our secure development environment >> Correct >> Okay great. This has been Steve Michelotti, with Joseph Bloom and Paul Fisher, talking about setting up a secure environment in Azure Government Thanks for watching