Using the Tooling API to Generate Apex SOAP Web Service Clients

hello and welcome to my talk on extending the support for soap based web services using their tooling API I’m Daniel Belanger I’m a developer for fuse IT an ISV partner based in Nelson New Zealand I’ve been developing on the platform since 2007 I think Bettina’s API version 8 and since then we’ve moved on it we’re doing much more modern things like canvas apps and a lot of integration projects with dotnet based services either then talking to Salesforce or Salesforce talking to them and so we have a couple of products around that area such as one’s for site core CMS an HP trim and that’s a lot of what Drew has driven the development of this tool we’re going to look at today is features we needed to be able to constantly work with soap based web services and how they evolved I’m also an active member on the Salesforce deck exchange which i think is a great place to ask questions and get answers for Salesforce so to sort of sit set the context have you have seen what are you ever seen any of these types of errors when you’re trying to bring a sope of wisdom into Salesforce these are the sort of things that we’re going to be addressing the one at the top for sort of features that Salesforce doesn’t directly currently support with the web-based version of wisdom to Apex and they’ll usually require you to go off and do some effort to get around them and the bottom to still ear is that you can get using wisdom to Apex but they’re more environmental about your specific org it may be that you’ve simply got too much Apex code in that org or you’ve already deployed that caste and you’re trying to replace it again and you want to get errors along that line telling you what’s already there so in this session I’m going to give you an overview of what they are fuse ITES FDC Explorer tool is which we’ll be using and we’ll recap exactly what happens or how what the flow is when you use the existing web-based version of wisdom to Apex and Salesforce and then we’ll do some demos showing how we can address the number of those we hit on that first slide and how we can work around them using our custom tooling and finally we’ll talk a bit about how it was made using the tooling API so the fuse idea is if DC explorer started off life as a very quick and dirty when forms at so it may not always be the prettiest thing but it turned out as we was a proving ground for tools that we were building in other places and it was a quick quick place to put things together but over time a lot of those features actually turned out to be quite useful in our themselves so we have things like we can dynamically build soccer queries using the metadata from the partner API a lot of session management and tools for running debug logs and pausing them and running anonymous APIs we’re not really only cover those in any great detail this talks all about the wisdom support within that tool at this stage it’s a windows-only tool there is however a basic web version of the same tooling that you’ll see in this demo doesn’t have all the features but it’ll get the core part of it done for you but you can get both for those from fuse ID fuse IT comm slash Explorer I want to point out if this is a free tool and I believe the exhibition as free as in beer you can download it distribute it give it away for free we don’t give out the source code at this stage so let’s just go back through what the current process is if you’ve got a wisdom and you want to bring it into Salesforce using the web-based version of wisdom to Apex the first step is and I’m going to use a UPS address validation as an example here you go off somewhere on the web you find your wisdom you download it and you save it to your hard drive next you log into Salesforce go to your Apex classes generate from wisdom then you locate that file you hit on you this and you upload it begin into Salesforce press pause wisdom Salesforce will validate that wisdom as syntactically correct XML and then it will find the namespaces that appear in that wisdom and map them to apex class names you know you’re free to come through there and change those as you need you could even if you want map to a single apex class and then press generate ApS code and all going well you’ll actually get links to those classes that were generated and you can move on to that point that you’re actually interested in which is that last green step where you can add your remote site settings and you can start calling that web service and getting the work you’re actually interested and done there is a problem there however and that’s not always the flow that you get quite often you’re going to define one of those errors that we had in the very first slide and you’re going to sort of

scratch your head and wonder well what do I do now we often end up as in the secondary flow that’s less than ideal flow where you’re going to have to start modifying that we’re still in ways in a hope that you can pass it back into that tool and then get the correct output that you wanted now it might be something as simple as there was an existing class there and you need to either delete that or give it a new name or it might be that your wisdom has what’s called imports or includes which are quite common where they’ve split the wisdom up into multiple parts and you’ll have to manually flatten those down into a single file and then it could be something like the wisdom has both soap 1.1 and 1.2 bindings and Salesforce is quite sensitive at the moment the wisdom tape X web version it won’t like it or reject it if you’ve got 1.2 bindings even if the one point ones are there and perfectly valid worst case it might actually be that your wisdoms bigger than 1 megabyte I don’t know how any you need the web version needs to be less than that it’s not much fun trying to shrink that 1 megabyte file down to get it within the limit it’s kind of like editing like maybe a 500 page book which is quite a big task so all going well you you get your editor bliss do we send it back through the process and hopefully get out to the end or you might go through a few iterations there’s quite a risk that when you’re in this unproductive flow there’s good saying for what’s going on here shaving there is delay you’re going to get more and more unproductive for the more time you spend down there and you might start out thinking oh I’m going to install this little XML tool and it will help me validate it or validate this wisdom and before you know it you’re you’ve got a screwdriver out because you’re gonna put a new hard drive in because you’ve been at a disk space and you don’t want to yeah and you’re just getting way off course and your boss is like what are you doing hmm so what can we do about that or get into our first demo so I’m going to open up UCF DC explorer it should already be logged in and it has a number of tabs I’ve turned most of them off the one of interest here is called wisdom apex and there’s two options to get you started if it’s a public publicly accessible web service with wisdom you can point it at the URL there and it will start from that point or if you’ve got one saved to disk you can start it from them just to give you a bit of context I’ll log in to Salesforce and show you what happens with this wisdom that we’re going to use so we come down you develop a picks classes generate from wisdom I’ve already saved it to my desk so and that’s the UPS address validation one so open that we’ll go through those steps pretty quickly we’ve met the namespaces we’re going to generate and there’s that error so what are we going to do well what’s actually happened there is that wisdom has multiple parts and it’s here’s that wisdom we don’t need to understand this too much but basically the problems up here this message has two named parts one of them is actually used for a header and ones for the body so salesforce is actually quite capable of calling then it just doesn’t understand how that was still structured so let’s take that same file and pass it to easy FDC Explorer there we go so we’re gonna find that same exact same file and this should seem quite familiar we’ve met in the namespaces from that wisdom and we’ve met them two classes and you’re free to come here and eat at them as you why do you like we’ll leave them with the default values proceeding on this is a new step and we’re going to come back to this in more detail and cover what it can do but basically it’s listing the web methods better than that wisdom so we generate apex classes and that we’ll run through that wisdom and generate the required apex code and it should look like they’re good for you that should look quite familiar to a lot of people who have ever actually had to look at what code is generated by the web based version or is built to Apex it’s got all the core parts in there you’ll be your port class and all the call-outs to web services call it and VOC to actually make those calls it’s also generated all the supporting classes but nothing’s actually being deployed to Salesforce yet so if we wished before we do the deployment we can actually come here when you we can make our required modifications and now

we can decide how we want to deploy that to Salesforce weak about options of either deploying each class individually or we can deploy them all at once so we’ll do that and they’ve all gone out I should point out the other you can see if you understand the syntax of wisdom to Apex or web service call out invoke that it has actually generated the required Hidde for that request so now we’ve got our code we can move on to our green stick we can add our remote site settings to allow that call-out and we can actually start calling it again but there’s there’s something missing there something that’s going to cause us a problem what maybe not a problem a lot of work anyway we’ve generated hundreds maybe thousands of lines of code before we can deploy that to production or put it in a managed package we’re going to need to write the code coverage for something that just happened in a few seconds which is going to be a big effort for us to do there we’re going to need to implement the web service mock interface so that we can mock our web and responses to that web service and then we’re going to need to test the invocation of each of those with methods that we want and look at each of the requests and response types that are required for that plus any supporting types such as headers that might be seen with the soap requests we’ve got another demo here showing you how we can address some of that I’m going to ricci it this here I’m going to use the partner API simply because it’s something that some people will be familiar with and it’s readily accessible so I’ll select the partner API it’s got a few more namespaces in it but we’re just gonna leave them as their defaults you see here we’ve generated a hole it’s found a whole lot more with methods in that one so what we want to do and this run is simply check generate test classes and it will pick up some extra that’ll run our extra processes yeah go so as before we’ve generated the main classes that we can use to call it so I think we can find them there does that need to be bigger we did go and find him so there’s our maiden service class and you’ll see things like the end point which is pre familiar the partner API actually has a ton of different headers you can define so we’ll skip over those and then there’s the more interesting part such as the YUM that we’ve missed it’s became cool so I’m going to deploy all those that’s using the tooling API we’ll come back to what that’s doing shortly so that’s great we’ve got that there I haven’t actually I should point out that not only did we generate all the basic classes that we’ve just deployed an implementation of web service mock that are also generated for us Big O and that’s implemented the do invoke method for us and it starred it’s just gonna debug out what’s being called with that’s useful for seeing what’s happening then it’s going to understand each possible request that could be passed to those weave methods and send back the corresponding response type it’s also generated a second test class what this one’s doing is testing the Constructors for cut these are really code coverage tests they’re not business tests it doesn’t understand what exactly that web service is doing but it does understand the structure of what is generated so it’s going to test all the Constructors and all the methods that it’s generated and in addition to those constructors this test method will actually use our mock which is in that other tees case and then test calling those actual methods so those have all been deployed and now we have the option of testing it and two ways it’s not so common anymore but you can actually run synchronous test cases and the advantage of that is it’s really easy to capture the debug log running under that context so I’ll get that started and we should see in a second the debug log and our test cases and what the results of those test cases were perhaps a bit hard to read there but it’s run forty-nine test cases and they’ve all passed and then it’s also pulled back the corresponding log down the bottom now there’ll be a lot of noise in there from the normal logging so we’ll filter some of that out you should be able to see those of the debug info statements that we had the the web service mock two so you can see

it’s calling all our web methods there’s another way we can run those test methods I’ll jump back it’s the web service output we can run them asynchronous which is the more common pattern now for salesforce test cases it’s a bit harder to grab the debug log in that case but what it will do is generate the code coverage so we’ll run that check the status of it again all 49 test cases are completed so we can see there if there was any problems here they usually show up but we’ve done well we’ve got a hundred are all their test cases to pass and now we can actually go back look at our our class what is it how many lines we’ve got were there 3,000 lines that needed to be tested to call the web service proxy and what we can do is we can press the view code coverage now that we’ve run those test cases it’s going to take a second because it’s got to go off through the tooling API and pull back all the line line coverage it came out of all those test cases it looks like there we go and then you can see it’s highlighted and made bigger all the lines that were tested and down the bottom may be a bit hard to read but it’s telling us that we’ve covered two thousand two hundred and sixty of two thousand two hundred and sixty lines that can be covered so we’ve got 100% code coverage for the generated test classes without having to write anything so we could actually go ahead and deploy that now quite easily so let’s let’s move on to something a little bit harder something a bit bigger and again we’re sticking with web services that Salesforce already provide to us the metadata API is quite a useful one to be able to call from a px if out of the box you try and directly import the metadata API you’re going to get a message that tells you class metadata are already in use and what that is is there’s a complex type within the metadata API called metadata that has the same name as the port class itself which is the thing you actually call out to so they’re conflicting our talling will be able to take there and give the port class a different name the second problem that you’re going to run into there is that there’s about 171 objects that use of what’s called a nexus de-extinction and there’s basically a form of inheritance and a wisdom so things like custom object and here it from the metadata complex type and they pull a number of fields from those our tooling is able to detect there and pull the required fields into the base into the actual classes so that we web servers call out invoke uses them it can have access to the required fields web service call out invoke doesn’t understand inheritance structures and wisdom tapings currently so this is a three hundred and thirty kilobyte wisdom to start with anyone want to guess how much APICS is needed to actually call that yeah about six hundred thousand characters at least now when you’ve got three million within your normal org limit that’s about one fifth of your available apex code just to get the required apex classes to call it that’s before you’ve written any test cases are actually even made a call out to it so there’s a big cost and bringing that entire where’s Dylan so what can we do about that well chances are you probably don’t actually need every single method that’s in that web service if we can start chopping out some of those web service methods we can instantly reduce the amount of generated code we need to do if we don’t generate it we don’t need to test it so we can chop that out as well and there may be features and certain methods that outright don’t work and Salesforce but if we don’t want to call those methods we shouldn’t be penalized and we shouldn’t fall over so we can might be able to skip over those completely anyway the other thing we can do is optionally remove a lot of the supporting classes so there’s we can choose to chop those out if they’re not needed for the methods that we’re going to actually use so we’ll run another demo here and I reset this one then we’re going to start from the metadata API this is the current summer 14:1 generate so let’s say we didn’t actually need all about twelve or thirteen methods that are there instead we only wanted the ones that start with the word list of which there is one so select

that now I should point out here and I don’t know if we’ll have time to cover it there’s actually options on each method here about what to generate so the default is just the standard web service call out invoke soap message but if you wanted to if you were doing something more difficult like read metadata write a read metadata which has a polymorphic response you can optionally generate an HTTP call out as well and it gives you a lot more flexibility because it’s going to understand what’s how to build up the soap call-out for you and how to process the response but all stick with just the list metadata the other thing we need to consider at this point is what are we going to do with all the extra classes that we probably no longer need believe it in its default mode is probably gonna generate most of them we have the option here and it says comment out and it will literally comment them out but still generate them but because they commented out they won’t count towards your own limits and the other thing we can do is we can just outright exclude them which is what we’re gonna do here so they pressed generate it takes a second longer because it’s got a process the entire tree to figure out what’s going on and you’ll see here that our actual result in class is a lot smaller where they’re definitely not in the 600,000 line mark I’ve got about a hundred and five lines and there’s seven thousand characters so we’re somewhere between one and two percent of the size of the original wisdom and of course all the corresponding things like the test cases have also got a lot smaller they are much more minimal now so deploy all those to Salesforce in using the tooling API and now just to prove to you that I’m not actually just deploying stuff that you can’t really cool I’m going to open up some anonymous ap X so here’s some I’ve saved a little bit bigger this is for the metadata API so this is going to invite create an instance at that port service I just created and then we’re gonna do to a couple of things here we’re going to take our current position information and assign it to the session header for that metadata API pool and we’re going to tell it to call to our current current domain it’s going to debug out what their end pointers and then we’re going to put together what our list meter data query is and I’m just going to ask for give me all the work flow and validation raw metadata and then finally we actually invoke that weave method that we just brought in using those parameters and we’re going to ask for them from version 31 which is the current one now we should get a debug log back from the anonymous apex it’s quite a bit going on there so we’ll get rid of some of the stuff from the log get rid of the constructors and now you can actually see if you’ve got good eyesight all they are logging and coming back so that’s all the methods being listed and then also there’s the call out requests and response going through they come through a straight strings but we can format that as XML it makes it a bit more readable and so we can as quite useful if you wanted to examine what soap messages were sent out and what came back so how does it work well the process is quite quite linear and it follows basically that what you’re seeing on screen so you provide the wiz delenn and it will reach out and bring that down and it will also look for any imports or includes and bring those down and manually flatten them in for you so you don’t have to go and find those who are it and then it builds up a model of metadata about that where’s all the information that’s going to need that model has passed into a generator that generates an object to model that represents what all the apex classes they’re members and their methods are going to look like that’s passed on to a dotnet technology that’s called t4 templates what that does without going into too much detail as transforms that object model into a string that can be represents the body of that apex class and that’s really for deployment to Salesforce and then finally it uses the tooling API to to deploy those AP X classes to sales to your sales force or if you’d like so part of our object model it should seem fairly familiar we have a px classes apex classes have members or collections of members and

collections and methods methods can take parameters and return types return types might even be void or they might be a primitive or an actual class there’s a couple of interesting parts here the relationship between AP classes and AP class collections at the top level we have an ApS class collection and it holds the apex class that gets generated for each namespace so from the metadata API we’ve generated top-level apex class to represent the namespace and one of them that we put all the AP classes that go in that namespace so there might be all your requests and response objects uni headers and in the case of the port that actually gets called the port class and all the web methods go inside there as well another interesting part is a special type of a member that in a class can have and it’s the stringer a apex member that’s particularly used to hold metadata for weave service call out invoke it’ll look for elements with a special type info name and it’ll pull that metadata from those the first parameter like folder is the name of the element and the soap message and then there’s its namespace its data type in this case string now I don’t believe web service call that invoker doesn’t actually care about that anymore but we still generate it and it doesn’t seem to mine you could put null in there as well and then 0 is the minimum occurrence one maximum occurrence and false is where it’s nillable or not and there’s other occurrences of this such as the field order type info which tells you which order to generate those two in the XML document so how do we actually go about deploying taking this generated apex and deploying it back into Salesforce well the tooling API doesn’t actually have a concept of upsetting an apex class either you have an ID or you don’t if you don’t have an ID and there’s nothing there with the same name currently you’re not going to need to do a create this is modified net code to look a bit more like something that somebody who writes Java or Apex will be familiar with we generate we take the a we make an apex class for each thing we’re going to deploy with a create where we don’t know its ID we give it its name and then we assign the body of the generated apex class to the body tag beer put those into a click ssin and call create on the tooling API when that comes back the only thing we have to do is iterate through the results looking for anything that wasn’t successful in this case we’re going to throw an exception there is an alternative path to this for we don’t have a ID for the class what we could actually do is just create a stub class with no body but the correct name and then use that as a way to get the ID that we can pass along to me today container the advantage of using a metadata container is that it compile you can put a whole bunch of classes and the requests at once for processing and so all understand the dependencies between those classes so if you’ve got a test class that depends on the actual implementation being there you can’t deploy that until the other ones there but with the metadata container approach you can you can pull it push them all up and one can request it and compile them all at once so the metadata container is going to sort of contain all the work that we’re going to do in this job you need to give it a name and the name needs to be 32 characters or less just something unique I just put the date in there so you create that and again check for the response make sure it was successful and you’re going to need to hold on to the idea of that record you just created for some of the subsequent call-outs so now you’re going to need now you’re gonna map all your apex classes into what’s called an AP class member pretty much the same thing as an apex class except it can have a reference to the metadata container we just put in so so yeah the top line we’re going to assign the ID of the class that we already know what it is we’ll put the body of the tea for template in give it at some apex class name the same and then assign it back to the apex class that we knew before and then we take that array of apex class members and call create and again check for the results to make sure that it was successful but at this stage we have an actually deployed anything we’ve just got the data up there so we need to create a container async request link it to the same metadata container so it knows what it’s deploying and then we

need to tell it are we doing a an actual deployment or do you just want to validate this and that can be quite handy and those earlier steps I showed you there’s a little deploy button next to H ApS class there’s a drop down there and you can just say validate and so you can make changes and all tell you what the results of those changes are without actually make so we could create our container a sink request a check for a success raise an exception and then we need to set and poll the container a sink request at some interval to see what’s actually happening it’ll sit in a queued state and when it’s no longer queued it’s either going to be failed or success if it did fail the deploy details will contract and contain all the useful information there about what actually went wrong and on what line so that’s really quite useful and then all going well we’ve deployed and we can just delete now can meter data container because we no longer need bed anymore now I’d like to say that would be able to handle any possible wisdom that you could ever throw at it but the reality is more whistles I see the more crazy ways they seem to be able to be put together some things we’ve service call out and folks simply can’t handle you might be looking at something like Apache exes that might generate arrays with external references to other elements within that soap message again polymorphic responses aren’t directly supported so one option there is you can use the HTTP call our option on the weave methods that gives you the ability it’ll generate the code you might need to modify it a little but it’ll get you a lot of the way there with a very little effort and sort of like my metaphor here if if it’s just outright broken we’re still if it’s not valid week so now we’re not gonna be able to put a bit to give the for you so that it’ll fail I mean if you do have you an interesting or unique quiz de we’d like to hear about it you can send it through to us at fuse 80 comm slash was to help actually sending your wisdom to us as I’m really helpful so that I can look at it a lot of wisdoms are behind firewalls and if I can’t see it I can’t really understand what’s wrong with it so in summary dealing with wisdom imports can be really time-consuming if something goes wrong and it really divert you from the task that you’re interested in achieving with our extended tooling we can bring in a lot of extra support in without having to use you can be on web servers call out invoke so we can handle imports includes and extensions we can generate basic code coverage which could save you a lot of time get to the point where you can deploy it quicker and then if you don’t need all the methods we can use method filtering and greatly reduce the amount of code generated well dude QA but before I start there as this one could point out a few things there’s the URL at the top again if you want to do an download at fuse IT comm slash Explorer while I’ve got your attention I could I’d like some votes on an idea I raised on Salesforce which is the ability to run anonymous APICS as if it was a test case I think there’d be some really good advantages to this you could assuming you could run a test context in anonymous Apex you could call an individual test method from a large collection and when one class may have some nice advantages you’ll be able to see the log in from just that method without having to see everything else that’s going on there of course it’s got other nice things you can just write some enormous apex runner maybe even against a production org and it’ll have no side effects because of the test case any votes would be appreciated for that about two weeks ago Salesforce open sourced via wisdom apex generator it’s on github I put a few pull requests internet so that’s another good option that’s Java based you can compile it run it yourself there’s a little command line and I believe it’s going to be embedded or integrated into the force comm IDE as well so that can be a good place if you need to if you actually want to modify their code and they’ll roll that back into their product I don’t believe that’s going to become part of the web version that’s currently there believe that’s going to stay static so any changes that maybe you’ll need to get into force Comedy there’s a session tomorrow I believe called calling building call-outs without wisdom a peaks and epochs passes I don’t know much other than the title or what some thing but it sounds applicable interesting finally I’d like to say thank you to Girish Aurora who as an intern built a lot of the stalling that we’ve used today thank you any questions now do you want to use the mic so the recording can pick it up presentation great tool thank you does

it run on a Mac I believe people can use wine can you use wine on a Mac if somebody’s used wine and Linux and I have seen it running out of windows okay but there is also a web-based version of that wisdom to a pigs it’s not as advanced but it will do the generation and you can get it from the same place thank you hi great tool and your website is really good too before they open source the Wisla FX generator did Salesforce work with you at all to show you how they generated noticeable no I reverse engineered it I basically looked at all sorts of examples of wisdoms and then tried to generate exactly the same code that Salesforce was generating so the biggest sample of wisdoms I have if you send me your whistle am i add it to that bunch of test cases to make sure we can generate the same it’s good now that it’s open source because that was reverse engineering it for me it took forever I appreciate the work so yeah two weeks before dream for so the session was well well ready before then and now two weeks before dream for stay open source there’s do you know there’s any roadmap no data whistled to ax generator is open source roadmap for Salesforce a roadmap for our stuff for the samples one or well I don’t know exactly what Salesforce are doing with him dear one I do know that they plan to embed it in the force.com IDE well that’s what they said in their blog post and I do know that they initially said they weren’t going to roll those changes back into the into the web version so you can’t just go AP class you know who from wisdom you’ll still be using the old one there but there’s just what they said in a blog post so I don’t work for Salesforce ok thinks they could do whatever they want oh is anybody used Salesforce Stack Exchange I can’t do anything right here but if you want to find me afterwards I might have something but I’m told I’m not allowed to keep stuffy outside cool thank you