Google I/O 2013 – Design Patterns for Maps – Architecture

SEAN WOHLTMAN: Hi, I’m Sean Wohltman I’m a geospatial scientist on the Enterprise Team at Google JOSH LIVNI: Hi My name is Josh Livni I’m on the Developer Relations Team for Maps So welcome, thank you for coming This is Design Patterns for Maps, Architecture If you guys are interested in UX patterns, how to make your map look amazing and beautiful for the end user, you’re in the wrong session That would be the next session coming up right after this, Design Principles Maps So, here we’re going to be focusing on what happens behind the scenes as you build your application, some common patterns we see developers come across on all different kinds of applications, and best practices there SEAN WOHLTMAN: So last year, when Hurricane Sandy made landfall on the Eastern coast of the United States, there was a pretty interesting application that was built It was called MapMill It was built by Schuyler Erle, some crisis mappers, John Crowley, Kate Chapman And the site was fairly simple in it’s design It showed images These images were collected, over 35,000 of them, by the Civil Air Patrol in various states These are guys hanging out inside of Blackhawks or outside fixed wing airplanes, taking pictures of what they saw on the ground And they do that because these are the first eyes on the destruction there So they have all these thousands of images, and what they asked was the public to come in, whether you were a seasoned photo mode scientist, or if you were a novice that was just interested in trying to help out And it was very simple, look at the photo and rate it Is it light damage, moderate damage, or would you say that this is heavy damage? And here this picture of Seaside, New Jersey, obviously, would be heavy damage And the cool thing about it was they took the aggregate of all those votes and created a grid And as soon as we saw this, we saw an immense potential This was information that was going to be really helpful for first responders, and for decision makers where to go and deploy assets So we took this grid and we deployed it to the Google Crisis Map And in so doing, we almost started to impact our server Because Schuyler was putting out a geoJSON feed of this entire grid and we, basically, were going and scraping it to put it into our servers, on an update cycle And we handled all the caching once it got our servers, but that direct request, every single time I was going to do an update, was starting to impact the main purpose of that site So because Schuyler knows what he’s doing He’s been doing GIS for a long time and web development for a long time He said, this is a simple solution Let me dump this out as a cache, periodically, and then you can hit that cache and not take down our site It’s a good thing he did it because the site, not just through Crisis Maps but through lots of other press releases, became very popular A lot of people got involved in this activity In fact, we said traffic records, on the Crisis Map, for the amount of traffic coming in, with people in the Northeast wondering what was going on, how were their homes, were homes gone? And so, it’s a good thing, that simple little change, of rather than having a dynamic request for millions of people, to having a simple cache, that we then cached for them made all the difference JOSH LIVNI: Yeah, so why would you care about this? You might be thinking your application is not going to necessarily be front page of reddit, hopefully it is, or on our Crisis Map feature there There’s all kinds of applications though that the same set of principles applies SEAN WOHLTMAN: That’s right, I mean what if you were building a really important local crime map and you wanted to get that information out there because it’s important to people in your community JOSH LIVNI: Yeah, or a totally different real estate site We talked about some of these last year Where you’ve got lots of dynamic data updating, to tons of people visiting on lots of different platforms SEAN WOHLTMAN: Or what if you have an incredibly popular game like “Ingress.” And you’ve got millions of spatial points of interest that you’re trying to keep track of, and how people interact with them JOSH LIVNI: Yup, or perhaps, you’re a local municipality trying to put out some spatial data It’s relatively static, to a relatively small number of people but you need to make sure it’s interactive in the appropriate way SEAN WOHLTMAN: Or you’re relying on a very dynamic feed, like an earthquake feed And you have different visualizations that people can come and customize, to try to make a map that makes the most sense to them JOSH LIVNI: Or a super immersive application, where you’re really taking advantage of some of the data, and imagery, and terrain matches that we have, to really get people in and add a little bit of your own magic on top SEAN WOHLTMAN: So there’s various tools that you might do for this And there’s various options here at Google that you can use to make these applications work You can use the Google Maps API and just push data into the client JOSH LIVNI: Yup, so a little bit of JavaScript there or perhaps some KML? Maybe you’re writing your own KML? Maybe you’re using it as it transport across different types of interoperable GeoServers SEAN WOHLTMAN: Or maybe you have a little application that you’re hosting on Drive that spits out geoJSON Or you’re pulling geoJSON feeds, like we were, from an external site JOSH LIVNI: So, we have a ton of different ways and services that help you build mapping applications And we see developers, oftentimes, pick and choose a few different tools and run into similar problems And what we want to do is show you, through a few different stories we’ll get to in a minute, some common patterns we’ve seen that might help you think about, in advance, what might affect your application as it grows So before we get into that though, I wanted to highlight something that Brian mentioned on stage earlier today, which is our Maps Engine platform

So this is pretty cool I’m not sure if all of you have the chance to play with it What we have here is the Maps Engine layer in Maps API Just loading some different layers I loaded parcels from San Francisco and trees from San Francisco And you can go ahead and scroll around and click on them And the thing I want to mention, just upfront– we’ll talk about some more of the stuff later– is that all of you are welcome, now to go ahead and get an account on Maps Engine You can head over to the Sandbox, the Google Maps Sandbox right after this. talk Give us your email and you’ll within a few minutes be signed up to a test account, where you can play around with this So throughout this session we are referring to some of the things that, hopefully, you guys will be playing with in a few hours SEAN WOHLTMAN: And if you want to know more about it, there’s also a session tomorrow called “Our Maps Your Business.” And so if you want to get a more in-depth look into Google Maps Engine than we’re going to do today, I highly suggest attending that session JOSH LIVNI: Yeah OK, so as we mentioned, there’s a lot of common patterns You look at all different kinds of applications– and we just showed a small subset of the kind you might be building on the previous animation– and we see very similar patterns So we’re going to go ahead and start out with some crowdsourcing example to show you what we mean SEAN WOHLTMAN: So also during Hurricane Sandy, we had another very popular layer that made its way onto the Crisis Map Because of the extent of damage to trees and power lines that were down, a lot of the residents had no power for days and weeks on end So a lot of folks actually did prepare, did have generators And there was, all of a sudden, a 1970s gas shortage inside of a New Jersey and New York So, some enterprising young students decided, hey, why don’t we get people, feet on the ground, to try to make a map of this So, let’s define a problem here All right, we need to map which gas stations have gas That’s a fairly simple problem statement How do you go about approaching that? So you might think, well, let’s go ahead and create a forum We’ll get a forum out there What’s the name of the gas station? What’s the address? Is there gas available? And you’re probably going to host that forum somewhere on the internet And then the idea is that you’re going to make this as viral as possible Hopefully get some press behind it And people are going to come in and submit that type of information But you can’t really trust everybody There’s most of the times people try to do good during crisis But there’s always going to somebody that’s going to come in there and troll So, you’re going to have some sort of moderation capability in there as well Somebody that can be able to approve things Somebody you trust, that’s been working on the project for a while and you trust their opinion And then you have to be able to make this spatial So you’re going to want to export that to some sort of format, whether it’s KML, or geoJSON, or whatever the geo-flavor du jour is And then, like we saw with Schuyler, you really want to make sure that if you have a database or if you have some service that’s generating these feeds, that you’re caching out the results, right? You don’t want people hitting your live service And then, and only then, once you’re up and set up and running, that’s when you want to put this data on a map JOSH LIVNI: Right So, we’ll make a brief segue here into something we see happen a lot, when people start collecting information that’s spatially related You come up with this type of forum we mentioned earlier You’ve got all the information you think you need; the address; even the latitude, longitude I’ve seen forms out there, just like this in the wild And I just want to say that this is not a very good form So part of best practices for your architecture, is taking advantage of other people’s architecture So this took me two or three minutes to modify this form, that I created in Google Spreadsheets under the form, and added an interactive map and a little drop down So if I’m looking for a gas station, say Exxon and it’ll pop it right up And this is just cut and paste code, directly from our example, super easy And if I happen to know the gas station’s there, I can kind of drag it over and update the latitude, longitude Those fields should be hidden Users should never know about that Anyway, you go ahead You say there’s gas there You submit the form And what you should see, there it is, is the crowdsourced information coming into your spreadsheet And we’ve got an extra column for confirmed, where a smaller select group of people can modify that Now, once you’re at this stage you have a whole bunch of data We want to get it out onto the map This is pretty easy The form took just a few minutes to create I could go here to tools and use Apps Script Now, if you’re interested in creating really amazing forms with really powerful Apps Script integration, you just missed a talk about that But you can catch it on video I think they’re all going live in the next day or two So the talk for that is called Forms, Use Apps Script to Create Dynamic Forms So, pretty cool talk where they go over the power of Apps Script, and all the kinds of things you could do For example, if something new comes in, you could send that data to a different API or service that you have to run some computations in it And all kinds of dynamic interaction you can do with Apps Script The easiest way to go from here is just a File, Publish to Web And then you get the power of the spreadsheets API The data comes out A RESTful request that you can make Make it public People can hit that feed It scales reasonably well There’s a little extra work though involved in sort of transforming that data feed in XML or JSON to something more map friendly Because this will not necessarily have that in mind So keep in mind those tradeoffs, if you’re just going this really simple way And also keep in mind the tradeoff of, this is not designed for tens of thousands of points Of course, spreadsheets can handle that It’s too much data to send over the wire to the client,

on all these dynamic requests So, a super simple way to get up and running Just think about the way you’re bring the data in and how you want to publish it out Oh yeah, and then a lot of people, when we mention scalability, they think immediately QPS What is the queries per second I don’t know how many of you have noticed the little sensors we have around Google I/0 Hopefully, you guys have seen that We have hundreds of those There’s one right on stage, tracking me as I talk And tracking you, if you walk by it And tracking five or six other different things, including temperature and all kinds of readings There’s a great talk on that, the Data Sensing Labs, tomorrow evening Check out the architecture there My point is here, we have a whole bunch of data This is also crowdsourcing The issue here isn’t tons of people hitting the site, but rather tons of data Every few seconds they are sending a variety of different signals back to a server With upwards of the thousands of these sensors, you’re talking tens, hundreds of millions of data points, over the course of days or weeks So that’s a lot of data and it’s more than most people have It kind of gets in the realm of big data And so, think about that If you’re going to have a big data set, that maybe the standard tools that you’re using for crowdsourcing, or even maps, aren’t going to apply Now, luckily, it turns out almost nobody has big data Everyone wishes they did and big data conferences are fun, but when it comes to mapping and GIS, most people have medium-sized data SEAN WOHLTMAN: So what do we mean by medium-sized data? This is a class of data, like Josh said, that most people actually have This is the data that’s too big to send the browser, but not quite big data We’re talking greater than 5 megs, probably up to a couple million rows at max So what are the strategies and what are the patterns are you want to employ when you’re dealing with this type of data? This is important since it is what most of you have So, if you have a small area in San Francisco, you could probably get away with pushing all the geometries in the Google Maps API and actually drawing them as an overlay here But what if you wanted to take that layer and push it out to all of San Francisco? We’re now talking about a couple hundred megabyte file that you have to be able to try to push into the client And that’s not going to be a happy day for anybody, right? Because you want to keep that density You want to make that map interactive and have all the data there So what’s the easiest strategy? Well, it’s nothing really novel What you really want to employ is take that data and then you want to break it up into a server-side rendering of that data You want to be able to just make a simple overlay of 256 by 256 pixels But it’s not just a rasterization process Because people are going to expect that these parcels are interactive So you have to have a strategy for being able to do some sort of interaction Whether that means you’re using some sort of an UTF grid or using some other technology, that you’re making that click actually respond and get data back So if we take a look at the diagram from the architecture standpoint, it’s not too different from the crowdsource You’ve got one or maybe a couple people collaborating on uploading a big data set You probably still have some moderation piece If you’re a county and you’re doing a tax assessment, and you’re going out and you’re changing polygons, you need to have some sort of moderation piece But then you need to have the server-side rendering piece that’s going to take that vector data, rasterize it and also keep track of the [? action ?] data for you Again, you’re going to want to have a cache The nice thing about having static tiles is there’s kind of cache built into that, in the sense that the tiles are the cache But you also want to make sure that it’s load balanced, of course If this becomes a very popular site, where your map goes viral, you’re going to want to make sure that you could handle the traffic load for it And then you publish it out to the map JOSH LIVNI: Yeah, so I saw this really innovative site and it’s almost sad that it’s innovative It’s a GIS portal for the City of Palo Alto I think they launched about a month or two ago And it kind of stood out to me because I go to a lot of GIS data portals And it’s usually, after going through some click-agreement, just a list of downloads Hopefully, in a format that your standard GIS software can read Often not And so you have no idea what the data is until you download it and put it into some sort of GIS system What I liked about the site, in particular, is you can kind of scroll down, play with the map Click on things actually directly See that you can have some fairly complex data Here’s some land use data, for example It looks pretty good You have an immediate idea of what it is, before you jump in And then potentially choose to download or otherwise play with the data Pretty cool So, let’s go ahead and see how you might go about building that type of application, and just a simple way to do it So, just as we walked you through it in the crowdsource example using Google Spreadsheets We’ll talk about our Maps Engine platform here So again, you guys can get some test accounts by going the Sandbox, if you haven’t signed up already Let’s go ahead and dive in So the first thing you’re going to do is you’re going to have some data, hopefully It could be a KML It could be a shape file, a variety of other formats are supported You can upload those files individually Or what I like to do is click the Upload button and choose a folder And that folder could just be full of all different kinds of GIS data And then you just batch upload it all with one click So you’ve uploaded what we call data sources Now you want to make a map And we do that by creating what we call a layer And that is, basically, some styles associated with your data So we have some pretty nice styling widgets here on the left You can go ahead and set filters, based on zoom level or attributes So if I want to style all the parcels that are on a certain type of street, [INAUDIBLE] will drop down auto-populated based on your data

We won’t go into that too much The next point here is you can also upload custom icons So, I brought in some trees earlier, as an example We have a whole bunch of nice icons But if you don’t like them, you can upload your own icons according to your brand or whatever icons you like, and style with those And then the other key piece here that it adds is really robust and simple ways to share the data out So it uses the exact same sharing dialogue you might be familiar with from Google Docs, and Spreadsheets, and so forth Where you just select the people that have access to the data, access to the published or unpublished versions And I won’t get into that too much now either But it’s a kind of a key principle It’s kind of that moderation flow You can have a public person that everyone sees And then some along the stages of creating layers maps Versions that only people that you trust or within your organization can play with So, the most exciting thing that I want to talk about is our relatively new API, which we made public just last week So I’m not sure many of you have had a chance to check out the documentation We’ll point out a couple things Those of you who are familiar with geoJSON might recognize it That’s the format that we spit out by default, if you make a query for features in your data set So that’s kind of nice It’s an interoperable standard with lots of other software And then what can you do with that? So let’s take a look at some of the other end points the API To walk you through some of the things that the API offers I put together this pretty straightforward application If you could see the URL, and I’m sure you can’t, the URL is So, if you go to GitHub and look at our Google Maps account, you’ll see a project under there called Galley This is the Galley application It basically is a proof of concept for working with the Maps Engine API So I’m going to go ahead and authorize it What’s nice about this, it’s using the GitHub pages, is there’s no server in the background I use the JavaScript OAuth library to authenticate in So that’s kind of nice You can build just a straight web app, use OAuth, authenticate in securely, and you have access to all the stuff that users should have access to In this case, I have access to two projects; signed up for one of those test accounts, so I have access to that; and I have access to my Josh Livni account We have access to all of the data and end points Some of the things I mentioned before are the maps, the layers I can render the layers if I click, for example, on the trees and zoom in You should be able to see all of the trees that we rendered according to the styles that we have We also have access to the tables This is kind of your data sources Just going to give an example of how we can interact with those So I downloaded the parks from City of West Sacramento And I’m going to go ahead and zoom-in to some parks here Click on this park and I can just go ahead and edit things And the way this is doing this– I’ll get rid of that exclamation mark I added earlier, send it back to the server And it makes a POST, a RESTful interface And now that data is live So the next people who view the published maps are going to be able to see my updates Oh, yeah Let’s see some satellite data And I can also go ahead and edit the geometries It’s a very simple tool Just using the Maps API geometry editing, I can go ahead and drag this around Add a little park space It’s probably not legal Hit Save And there we go Again, posted back So this is a simple way to play with the application Feel free to check out the code on GitHub I tried to document it nicely So you get an idea of how we’re interacting with the API, how you can build similar applications You could imagine, for example, that crowdsourcing app, it starts to need a little bit more robustness to it You could go ahead use Apps Script to push the data to Maps Engine on certain things occurring in the database, like things got confirmed And now you can render those out, again, like Brian mentioned earlier, using the same power that we have behind Google Maps itself So all of this data is sent out to all of our data center super duper fast Rendering using the same technology we use to render the base map So there’s a great way to publish your apps SEAN WOHLTMAN: How many lines of code, is that? JOSH LIVNI: The grand total? It’s about 600 lines of code, I think, including the documentations It’s a super simple app I think Twitter Bootstrap helped me out a little bit As did, obviously, the OOP JavaScript library I didn’t have to pay attention to UI or authentication But it’s really, really simple to build an application like this So, if you need a map editing tool, feel free to fork it and have fun Oh yes, thank you Sean So, we also have assets Assets are what we call all the stuff you put into Maps Engine So it includes– if you scroll down the left hand panel– all of our tables, and maps, and so forth But also a whole bunch of other stuff over on the East Coast Maybe, Sean, you could tell us a little bit more about what those things are SEAN WOHLTMAN: Yeah, so the nice thing about Maps Engine is it’s not just about vector data that we’ve been talking about And that’s where we’re going to pick up our story with the next type of data that you might be coming across, which would be imagery So imagery, we’re probably familiar with it Here’s a shot of us here in the Moscone Center, nice 15 centimeter resolution imagery But it’s not just photographs It can also be charts You might have charting data, like this NOAA RNC of Oahu And the same strategy applies that we talked about with the

medium-sized data This is where Google Maps got its start, like Brian mentioned, is actually doing the tiling and creating a static tile cache So the same principle applies You need to have some server in place that’s going to take that image, chop it up into 256 by 256 pixel images, and then serve them out Now, it’s really simple once you have those tiles, to overlay them with the Google Maps API So if I have a Maps API site, I can just make sure that the tiles are exported into the web Mercator, which is a standard for all slippy maps This got a EPSG code of 3785 And then if I want to play with the 3D, I want to make sure that I also want to put out tiles in the plaque array or 4356 In this case, this is really nice because I’ve got the bathymetry from Google Earth And I’m overlaying this nautical chart with all these soundings And the soundings are lining up under water So this is an option If you’re going to serve a 3D application, you want to make sure that you have tiles in that format as well So, just to review you’ve got your image You’ve got some sort of server running a tile rendering Once again, your cache is kind of already built for you because it is static tiles But you do have to think about if you’re going to put it out to two different projections There are a couple of categories, I would say, and different patterns that evolve when you’re taking a look at what the use case of the imagery is And the first one that I want to talk about is what I would call fast response imagery This could be– if you followed Public Laboratory and funded this Kickstarter– a great project where you take a cellphone or camera, a point and shoot camera, and tie it from a weather balloon Or you go and follow DIY Drones and build your own drone and put and Android phone there And a little Arduino to pilot it Or it could be actually formal imagery, such as NOAA NOAA goes those out and collects imagery after hurricanes And they actually fly with really nice commercial cameras in real planes But the key here is if it’s fast response imagery, it needs to be fast, right? Time is of the essence Here’s an example from Hurricane Sandy This is Mantoloking, New Jersey This is the first eyes on that we had of the area And it’s pretty dramatic if you take a look at what Mantoloking looked like before See the bridge coming into these beautiful homes, beautiful little beach town And then after Sandy rolled through, the dramatic change there We have all of a sudden, a new natural-made inlet and houses have been wiped out Josh is going to tell us a little bit about one strategy that we could use to get that imagery out as fast as possible JOSH LIVNI: Yeah, again, so we’ll take this super simple case first You have a little bit of imagery You want to get it out to people as quickly as possible There’s tons are proprietary and open source tools and workflows available for this I’m going to show you one example using the GDAL Library and Google Compute Engine Now, I’m not sure if you noticed any of the talks from the cloud track this morning, but Google Compute Engine, I believe, is available to everyone tonight And also they’ve announced a bunch of other great stuff, like [? sub-hour ?] billing, and you can boot up lost instances, and turn it down as needed, and so forth So the way to get access to this is you go to the Developer console, which is the same place you might, for example, get an API key for Google Maps You go to your Services tab and just enable Google Compute Engine So once you’ve done that, you’ll see a Google Compute Engine logo You can go ahead and click on that Go to the Instances tab And then there’s a variety of options I like to just choose a standard Ubuntu instance Don’t worry too much about the other parameters, booted up We’re going to show you how easy this is So you boot up the server You SSH into this this virtual server that runs in our data centers And then you just run two command lines to get up going with GIS in our server Install Apache2, so that I can have web requests coming in We’re going to serve out some tiles And then go ahead and install UbuntuGIS, which installs a whole bunch of software, including this GDAL Library After we’ve run those we’re going to go ahead and download some imagery data And then we’re going to use that GDAL tiles library Which is a little Python script, basically, that will mosaic that data together And create all of the appropriate tiles in all the appropriate projections for you So, it took about two minutes, I think, for the data we downloaded And rather than waiting two minutes, we made a video I think it just finished So that was it Download the data, created some tiles, done Now we have the tiles in a folder, and let’s take a look at the results SEAN WOHLTMAN: Great, so the results of that We have a nice little embedded Google Maps V3 API site here And I can zoom-in to that area that looked really damaged And I have a nice little custom overlay here, so I can do a little transparency slider And you can just really start to see the total destruction Now getting this out to people, especially people whose homes those were, first responders and rescue folks coming into the area, is critical So this is a perfectly fine way to address a small area, get that data out there as quick as possible with minimum friction As long as you understand a little bit about the GIS tools behind the scenes But, what if it’s not a small area? What if it’s a really big area America, the United States is a pretty big area It’s about eight million square kilometers and what if you wanted to do imagery on that scale?

And what if it was fairly high resolution? I’m not talking high resolution like we showed earlier, 15 centimeters But the USDA NAIP program, every three years basically, recovers the United States in one meter imagery So you can take a look at the Moscone Center here Obviously, it’s not going to be totally exciting in urban areas But it’s not really designed for urban areas It’s really designed to go out and survey agriculture They fly it in four bands, so they can also do some photogrammetery telling the different crops that are being grown So, this imagery, if you had it for the entire United States, which we do, for the past three years would be about 80 terabytes of imagery So my first problem is now I need to upload 80 terabytes of imagery somewhere And then, it’s not just that imagery Now, I actually we need to make tiles with it So take that same three years of data, I’m going to tile it out It comes to about 1.3 billion tiles that I need to write to disk And that in itself is about another 30 terabytes So, now I’ve got to worry about how many disks I need, how much CPU and RAM I need to spin up, how long this is going to take And there are a lot of really smart people working on how to do this with big map reduce processes My recommendation being not try to reinvent the wheel So we took all this and put it into Google Maps Engine And we blended and served redundantly over thousands of cores in a few hours And when I say we served it redundantly, we’re writing this to multiple data centers at once, for you with Google Maps Engine So we’re actually taking all that data We only tile it out to not just one data center, but across multiple data centers And it’s ready in a few hours So take a look at a quick preview of this Here we have a 2D map And this is the default map that you would get with Google Maps Engine This is built for you This is the Google Maps view And you can come in and see it’s just as performing as the underlying tiles that are there The one meter can go back and forth, on and off And the nice thing about Google Maps Engine, if you remember I brought up before that I wanted to serve both 2D and 3D Google Maps Engine doesn’t charging the overhead for also doing that in 3d as well So you get a lot of bang for your buck, running on the same servers that are running the rest of Google properties And there’s another thing that we should talk about as well There’s a security piece here I could share that map I can make it private Maybe it’s licensed data Maybe I can’t put it out to the entire public So I have the security things that we talked about with Josh But we’ve only talked about, so far, about putting this data out onto Google properties, Google Maps and Google Earth And if you work with anybody the GIS community, they loved it It’s nice, but they also want to be able to use it in their analytic applications So if you were going to do this yourself you now have to think about how am I going to implement things like OGC standards How am I going to put out a WMS feed? Am I going to have to have the overhead of returning WMS requests? Google Maps Engine does take care of that for you And so you can put it out into OGC compliant feeds that can work with other applications, outside of the Google framework JOSH LIVNI: Thanks, John Yes, so hopefully you can see that even from a little bit of data– maybe it’s vector data you’re crowdsourcing, maybe it’s a little bit of imagery, many of the same things apply– if you just have to serve it up a few people, it’s static, you’re not worried about a huge influx of data or dynamic requests taking you down, then there’s a ton of tools and options out there As people start adding complexity– and this is inevitable for most popular applications– and they need to think about things like the moderation piece, the simple sharing piece, the permissions, and so forth, the different projections, the OGC standards, make sure that you’re using a framework and thinking about that in advance So when you do get popular one day and someone starts slamming you, you have a cache ready When someone says, we need to take care some moderation issues, you have a method in place to do that So, hopefully, we showed you through some different stories that many of the same pattern arise We tried really hard to put out some software that makes this as easy as possible for you guys Hopefully, you go out and try some of that software, including Google Maps Engine Let us know you think And also let us know if you have any questions I think we have about 10 minutes Yeah, thank you very much If you do have questions, please come up to the mics And speak into the mics and we’ll do our best AUDIENCE: Hello, my name’s Pat I work for a high tech agriculture firm based in Des Moines, Iowa And I know that was interesting SEAN WOHLTMAN: Not a plant AUDIENCE: We have a problem And that is that a lot of people in the agricultural industry live and work in areas where they don’t have good access to the internet And so, I’m curious if you have any suggestions for patterns and practices that would help people when they’re wanting to look at map data, and they do not have an internet connection? SEAN WOHLTMAN: Yes, in fact we have a session tomorrow [? Sean Maday ?] and [? Harsh Bhatnagar ?] are doing, basically, what happens when you have to go off the grid I’d highly recommend taking a look at that There’s commercial applications, but there’s also grants that we do that, basically, allow cutting from the Google Maps Engine platforms And you can actually take portables of the maps that you create And take them with you in a portable cache that you could run on a PC, Mac, or Linux server

AUDIENCE: I want to question using Google Maps data Can you take the different scenarios for the Bermuda Triangle Because people who go there, they don’t come back, I heard Right? So Bermuda triangle SEAN WOHLTMAN: I’m sorry AUDIENCE: There’s something called Bermuda triangle, JOSH LIVNI: Oh the Ber– yeah, sure AUDIENCE: So, where if a ship goes, it doesn’t come back Or if people going, they won’t come back So maybe using this Maps data you can find out what is the geographical forces, which is making it so dangerous And get some conclusion That’s an interesting idea I’m, actually, surprised somebody hasn’t made some cool mashups of– I’m sure the satellites have lots of imagery SEAN WOHLTMAN: But actually there’s a talk tomorrow The AIS talk– JOSH LIVNI: Setting us up, too SEAN WOHLTMAN: –about tracking all the ships in the world And using bit query and Google Maps on doing that JOSH LIVNI: Good call Yeah, in fact we have a few– AUDIENCE: Find out what is happening there There is much energy, wind energy So they can also use that wind energy for this Google [INAUDIBLE] and then you can put a [? Cart ?] there and see what is happening JOSH LIVNI: It’s a good question Yeah, I’m certainly no expert on ship data But we happen to have some experts on ship data giving talks at different tracks So check out the descriptions I think we have like two or three talks And these guys really know about ships and where they go And we’re going to be showing that, so that’s a better question for someone there AUDIENCE: — there is so much of energy there, wind energy there, you know So all that energy you could use it for something else right? maybe using those analytics and the data of Google Maps you could use [? it for something. ?] SEAN WOHLTMAN: Very possibly Thank you AUDIENCE: Hi, you kind of wowed me with the forms example and the map to the side You said that it was pretty easy produce, but my experience with forms is it’s quite limiting And I was just curious if you go into a little bit of the details on that? I mean, did you to create a sidebar with the map and that’s how you got it to record lat, longs in the form? JOSH LIVNI: Sure, the way that I made that form was what I’d call the old school way, that I don’t recommend But what I did is I created the form using the forms widget You hit forms and spreadsheets And now of course, you could create easily your own form If you have an open source, CMS, Django based or something to create a web form It’s just a web form You’re trying to get it somewhere But in any case, what I did is I modified the Google Spreadsheets form to have all the fields I needed Then I downloaded it And then I opened up the HTML and I added a tiny bit of JavaScript to attach the fields Took me about five minutes Went into our examples for auto complete, and I just cut and pasted some code, changed the input field It’s done So it was pretty easy But, as I mentioned, there was a talk that was, I think, just last session, where they go over how to do that correctly So Apps Script is super powerful and you can do all kinds of dynamic things with the forms now And so, that talk will be on video and that would be a better way to approach the problem SEAN WOHLTMAN: But we will also have this entire talk posted, so you’ll have the slides to go in and take a look at the source, as well JOSH LIVNI: Yep I’m scared now AUDIENCE: Hi, I’m Andy I work for a company that provides solutions that overlay map data on top of Google Maps And one question I had was, in your best practices recommended, you talked about meshing up large imagery in tiles and serving that out But I also heard, in the keynote this morning, about the vector maps and how that’s the way of the future So what do you recommend best practices on serving up vector maps? SEAN WOHLTMAN: So, I mean, one thing to say is that when you have imagery, unless somebody has a really amazing algorithm, that that’s not going to get vectorized any time soon I think we’re still going to be in that world for a long time But yeah, do you want to take the vector– JOSH LIVNI: Sure, yeah So, today we try to serve out the tiles in the most appropriate format So, the hope is you upload your data vector, in this case, to Google Maps Engine And we’ll serve it out appropriately If one day it turns out that the appropriate way to serve that data is in vectors, that’s the way it’ll be served out So, the idea is you don’t have to concern yourself with that Now, there are some talks, I think [? Brandon Candy ?] is giving a talk on some amazing things you can do with WebGL in creating your own vector data that gets displayed on the clients And that’s a good talk to go to I think it’s called HTMl 5 A Love Affair and Maps a Love Affair And there’s a bunch of ways you can do it yourself But yeah, obviously we have a lot of technology and we believe that is a really great way to go And we’ll make sure that your data is served up when it makes sense, in the right way AUDIENCE: Thank you JOSH LIVNI: No problem AUDIENCE: Good day, I have one question Please tell me, I want to keep a road graph Can I keep a road graph by tools, by storage if I have maybe 10,000 or 100,000 vertices is in my graph And I need in calculation maybe, brought in compilation maybe by the extra algorithm or another Or I must keep it separate Mainly vertices on my server for calculation and some vertices for visualization by your Maps Engine

So what about performance? SEAN WOHLTMAN: Yes, so right now doing Dijkstra or A-Star or anything like that over a vector network is not something that Maps Engine does But when we talked about Compute Engine, you should think about that as a possibility To be able to run the traditional PostGIS or pgRouting types of applications in the database in appropriate form, until we can offer that as a direct offering in Google Maps Engine AUDIENCE: But if people create some vertices of raw graph by themselves then we have a [INAUDIBLE] or an other site Can we use your storage as a local cache, as front end server for our backends, in order to send it backend? JOSH LIVNI: Yeah, so the general pattern there is, if you would like us to render it out, we have the API So I showed a really simple example of editing some geometries, but, of course, you can batch post a whole bunch of features into an existing table, modify, or delete them So, if you need to do some type of synchronization or adding new things to data that’s in Google Maps Engine, you can do that using the API AUDIENCE: Sorry, do we need an Engine server for balancing between my backend and your storage? JOSH LIVNI: I’m not sure I understood SEAN WOHLTMAN: You’re asking how to connect between your backend database, to what it would be in ours AUDIENCE: I don’t know about average number of requests to your storage? What about it? JOSH LIVNI: You mean if you’re doing like a whole bunch of requests, very frequently, what are our limits? So, we have a few different limits I think right now, if you purchase Google Maps Engine– I’m not sure exactly offhand what the queries per second is that you can post and get It’s pretty high and I think you can, as many of our offerings, purchase more quota as you need I’m not sure what it is offhand, by default But it should be plenty I mean it’s many a second And you can batch post a large number of features that once So I think it should suffice SEAN WOHLTMAN: If there is a use case you really want to dig down into, please come by the Sandbox JOSH LIVNI: Yeah, we can talk a little more detail, something specific AUDIENCE: Quick question, in your imagery example, how was that 30 terabytes of data delivered? SEAN WOHLTMAN: How is it delivered? As tiles AUDIENCE: I’m sorry, delivered to the Google App Engine SEAN WOHLTMAN: Oh, how do we upload the initial? AUDIENCE: How do the– Yeah SEAN WOHLTMAN: OK, so there’s a technology that Google has called Xenon Which, like Josh said, there’s one way in which you could sit there and try to bulk upload of all that For the 80 terabyte source, you probably don’t want to do that Xenon is a technology that Google uses to monitor remote web servers, where we have a lot of partners, such as VEVO that puts out a lot of music videos And we monitor either a http or a ftp site, where we will actually pull that data across data center to data center And we can ingest that for you into a particular account So, it needs to be on a server somewhere Or, if it’s a really big delivery like that, we can actually have customer agreements through some of our partners that will help get that data online for you Because, yes, getting a large amount of data into the cloud is a barrier to entry JOSH LIVNI: OK, well thank you guys very much for coming Sean and I will be in the Google Maps Sandbox for the next hour or so Look forward to chatting with you guys SEAN WOHLTMAN: Thank you