Build a Geospatial App with Redis 3.2- Andrew Bass, Coding House

I think we’re about to get started here if you had a chance to visit that link we really appreciate it we are gonna use that data there for our demo today this is the Redis 3.2 geospatial talk my name is Andrew bass what are we gonna cover today is a few basics of the geo commands that are new as of May 6th in the stable branch of Redis we’re also gonna tell you about a few of the gotchas we found while we were building an app over the last month a team of people at friends I have here and we’re going to cover the basics of the geo commands which are pretty simple they were really well documented we’ll cover those gotchas like I said and we have a short demo app that we built a node which will use that demo that data that you put in for us on that demo we’re gonna do and then we also have a full scale app that we built over the last month so you can see a little bit more what that might look like in a real app ok so who are we my name is Andrew Bostick I said today I’ll be presenting with Deanna and Sean their classmates of mine at coding house it’s a boot camp down in Fremont about an hour from here we all come from different backgrounds and we’re trying to get into software development and that’s how we met Dave Nielsen from Redis he actually came down and gave his talk about a month ago and he mentioned these geo commands it were on the testing branch that we’re gonna be made stable pretty soon and we thought that it might be neat to try to build an app with that and see if we could play around with a little bit so we did and when we were kind of keeping it just with him he said why do you guys come down and give a talk about that he actually told us that like last week so here we are Sergio is not with us here today but he did build the app that we’re gonna show at the very end of the talk and it if you want to check it out it space meet us we’ll have that that’s well show at the end but if you want to kind of show off to you to some other people there’s a really easy guest view to see so you don’t have to login or anything ok so right is 3.2 as I mentioned stable very recently it includes these new geo commands which is for geo indexing the most basic way to look at it is you’re just gonna be saving those locations in some kind of key and you’re gonna compare them Redis is gonna take care of all the comparison for you makes it very simple it’s a special kind of sorted set so all of the sorted set commands that you’re already familiar with can be used in this case okay so the first couple of commands we want to show you Geo ad and geo hash geo ad very simple just like it says you’re just adding a location to your set so you’re gonna pass in a key longitude and latitude that’s our first gotcha we have here today basically when people talk about latitude and longitude they always say latitude and longitude right but if you’re actually working with geo information systems you’ll it’s more typical to say it the other way it’s the XY coordinates so you’re going to talk about longitude then latitude so if you’re not very familiar with the world of geo that might catch you off guard like I did us at the beginning it also uses the earth model as a sphere so you do have a small amount of error about 0.5% depends on which part of the globe here and how far north or south is gonna affect that okay and when we were passing in those longitudes and latitudes you need to know that we’re going to be using positive and negative so if you’re going to be west of the Prime Meridian those will be negative longitudes and if you’re south of the equator those will be negative latitudes all right so I do want to demo a little bit of the command line for you that’s what Dave said we might like to see that and if I can just hit the right buttons here there we go so the first command is geo ad I wanted to say that when someone had to type those numbers in oh I’m sorry I have to hit one more button thank you would have been really exciting for anyone is it big enough we’re not at all okay hold on just make a little bit bigger for you better it yes in the back okay I’m getting some thumbs up so if not you better yell at me so here we’re doing a Geo add the first parameter we’re passing into the redness conference attendees that is the key we’re passing in the longitude first then the latitude and we’re gonna give it the Twitter handle at Redis conference which is the official Twitter handle for the conference and we’re also going to pass in Salvatori as the location in Sicily there it’s just a generic location all right so now we’ve added those and we’re also going to demo the geo hash which I’m going to explain what it does but I just want you to see

that if we do a search for Redis conference attendees and we’ll just look at Redis conference so we get this really strange string back it doesn’t really mean anything to us it’s kind of nonsensical that’s the geo hash so we’d like to see there we go okay what is a geo hash all right so the way that kind of works it’s a little bit of a multi-step process but you’re going to take the world you’re going to divide it in half and one side will be a zero one side will be a one we’re gonna use binary and we’re gonna save that digit for example San Francisco is on the left hand side that’s going to be the zero digit we’re gonna save that and we’re gonna keep dividing so it might start to look like this so the first digit was a longitude and the second digit is gonna be a latitude that’s gonna alternate back and forth so you’ll take this latitude and longitude two separate numbers and you’re gonna kind of combine those into one long binary string which for San Francisco where we’re at is something like this 122 West 37.8 north and that’s going to be this long binary string but that gets converted into this geo hash now this isn’t something that Redis invented it’s a standard that’s used if you go that website spy little small for you sorry geohash.org or if you just google it it’s one of the first things to pop up if you put in any longitude and latitude in there it’s gonna return the string format it in the same way and why do people want to use a geo hash okay one reason we’re using it actually it’s being stored in a little bit different way but we’re gonna talk about it this way for now when we store this geocache we can compare those strings and if you look at the first three cities I have their San Francisco San Jose in Los Angeles they’re a little bit closer together so the first few digits in the string the first few characters in the string are similar so we know that those are a little bit closer together and as we get further away the similarities go away so we know that that’s a lot further away it’s actually being stored is like a 52 bit integer it means nothing unless Salvatori nods no that that’s not what’s happening at me so we want to be able to compare those one reason you’ll use a geo hash you might want to return the geohashes if you’re making a URL and you want a unique ID so you want to say this restaurant is at this location and it’s just gonna be slash 9q 8yy and you know exactly where that’s gonna be for when it loads into your website you can also use it for making unique IDs something like that since it’s gonna be different for every place all right the next command we’re going to talk about is a little bit easier to understand than geo hash a Geo position just going to return the position of the member that you already passed in and that’s going to give you the longitude and latitude back but we do want to notice one little gotcha actually I want to say this is a great opportunity for you to use pub/sub we’re not going to dim a lot today but if you have a position that’s constantly updating you could pub/sub and see where the like a car is moving or where are the city buses or maybe uber could track where their cars are at at that time so that would be a useful case for this maybe alright and we’re gonna demo it now yeah so we’re gonna do just yo position yeah I’m gonna hit that I’m gonna forget that but every time sorry that’s the problem with practicing on your own monitor at home you just see it there and you don’t even think about how it’s actually gonna be here right so we have Redis conference attendees and we’re gonna pass in at Redis comp and we’ll go ahead and pass in Salvatori as well so we get both of those back and you’ll notice that it’s a lot different than the original number we passed in so if you can still see it up there yeah if you see those numbers at the very of the screen I passed in a much shorter amount of information but it’s returning this very very specific data back to me okay that’s because we’re converting that fifty-two bit integer and we’re looking only at squares so it’s defaulting to some point in that square because no matter how small you go you’re still looking at a square with a 1 or 0 that you could cut in half right so it’s picking a certain point and kind of defining for you what that accuracy actually looks like and that’s how it’s getting stored in reference so it’s a little bit different than what you passed in but that’s still okay I’m gonna talk about that now all right if I can hit the right commands again okay good so what kind of accuracy do we need when we’re using geo and how much of those digits gonna affect us now this is just off Wikipedia you can go on there and read it I just cut off the table a little bit don’t worry about all the data that’s up there but essentially every decimal place you go in you’re gonna be going from one kilometer to 0.1 kilometers to 10 meters to 1 meter to 0.1 meters so on and so forth down down down and down for each decimal place so what is going to be useful for us using Redis like we heard yesterday it’s a

great application for a cell phone a user is standing in a city they want to see restaurants near them they’re gonna pull out their cell phone well hacker do they need to be able to find that restaurant right if they if they get within a kilometer that’s not going to be helpful but if they get within 100 yards or hundred meters that’s fine not helpful but maybe 10 meters or a meter they’re gonna find it they’re gonna see a sign and they’re gonna find what they’re looking for so for us that’s gonna be about four or five decimals for decimals is going to be about 10 meters of accuracy five is going to be one meter so you saw that string was like 10 decimals that’s like microns that’s like the hair on your head as the wind blows it’s not going to be accurate for really locating something so those numbers don’t matter for us all right and there’s also that error from the way we’re storing the data the very small error so it becomes insignificant all right you want to think four or five digits is going to get you close enough for our use cases and that accuracy is gonna change based on where you’re at as you go further north or south you’re gonna get a little bit more distortion or a little bit less but that’s a 10 just an easy number to remember okay all right so now is when we get into we’ve seen how to set up users how to save those geo locations but we want to know what is Redis really doing for us I mean I what good does it do me to save these locations well geo distance is the first one we see that’s gonna be really useful you’re gonna pass in two different members and see how far apart they are so I’m gonna try to do this a little bit better this time there we go yes and we want to pass in we’re gonna get the Geo distance for our Redis conference attendees and we’re gonna check how far did Salvatori come this morning or well yesterday or the last few days to present to us and tell us all about this and share our joy Redis with everyone right so oh wow that’s a really weird number it turns out that geo distance is defaulted to meters so that’s not very helpful so we’ll try that again and we’ll pass in mi4 miles again this is all in the documentation it’s very easy to follow so it came about sixty-six hundred miles to be here with this but you know since he’s from Italy we’ll probably want to see that in kilometers okay and one thing I wanted to point out it’s really exciting if you’re trying to learn this stuff with the new 3.2 if you have it installed on your computer it does this it might be hard to see because it actually you see this member one member two key it’s telling you what parameters to pass in and I like when I first saw it I thought it was an error I was if there’s I didn’t type that what is that but anyways I was really excited about that so and see if I can get there we go so that’s geo distance now geo radius and the next one we’re gonna talk about geo radius by member they’re pretty much the same command one you’re going to be looking at a latitude and longitude and geo radius by memory we’ll be searching by that member we have a lot of these flag options on here like with coordinates with distance with hash count ascending or descending and Deana is going to be demoing most of those for you do want to say one thing when you do the geo radius if you pass in the with hash this is one of those gotchas it doesn’t return that geo ass I showed you it returns that 52 bit integer which doesn’t have much of a use case you can use it like for a little bit of debugging and things it’s hard to find a lot of information about it so hopefully someone will figure out more about it then I understand and be able to explain it if you need it okay so don’t use with hash expecting to get that unique hash ID that I showed you at geo hash is that all right just looking for a nod or two yeah and let’s go ahead and try to do this command as well so I’ll just sample one and indiana is going to come up here and show you another command so if we just type geo radius the Redis conference attendees again passing a latitude and longitude well searches within 50 miles now I have the data that you guys put in but it was only as of like nine o’clock this morning Dan I’ll have the most recent data so we can see that these people are from the Bay Area so this might be the start of like a useful app to say pretend you’re in Denver Colorado you want to find some people near you well you come on here and say oh these are already users who are at the conference I remember meeting Nicholas Dunham number eight there I should get in touch with him maybe we can meet up and he can help me through my problems so again this is just a sample but that’s an idea of how you might use it or you’re gonna be looking for restaurants nearby on a cell phone right we’re gonna say this is the point of the user it’s passed up the longitude and latitude and you’re gonna search nearby them for restaurants set or places for coffee or whatever that set of data is that you want to demo for them all right and the last one is geo radius by member so this one instead of passing in that

longitude and latitude you’re actually passing in the member name for example at writers conference and searching a radius from their from their points so maybe that user is logged in and they want to see what people are nearby them right so instead of connecting a person’s cell phone with locations around them you want to connect a person with other people who are moving around and maybe live updating their locations they’re gonna pull out their phone hit a button and it’s using their stored location and looking for other people’s stored locations in that area but Deanna is going to demo that for us with our little demo app we had here that that’s basically what it does you’re finding people near you who are at there who were at the Redis conference based on the zip code and again that data is passed in by zip code so it’s actually a Google API we use to get back just a generic centered up latchet longitude and latitude for that so there will be some error but also we didn’t want to do like really specific information on people yeah Deana hey everyone my name is Deanna I also worked with Andrew and Sean and Sergey who’s not here with us today on this app that we built using – his new geo commands right now first I’m just going to start off with a simple app that we put together that andrew was just explaining prior just to see the commands put into practice so one second let me just get this all pulled up so this is a display was all messed up ok so it’s just a demo demo app and I’m gonna be showing you more notably the geo add and geo radius by member commands just to see you know which users how far they travel to get here to read his conference and you see what that looks like here so really quickly we did a we pulled in this get request just makes a request to the forum that many of you should have already filled out prior and it uses the geo add command right here you can see that the the syntax used we actually used a Redis NPM package for this the syntax for the geo add here is exactly as you would type it in the command line with the difference of you know the commas instead of spaces but it would be in the command line it would be geo add the key right here which in our case is Redis conference attendees as Andrew demoed and the latitude and longitude and latitude which here we have stored as variables and the member which in our case would be user dot a Twitter handle we have a you know you guys filled in your Twitter handles and that just adds it to our local database I’m going to be demoing it locally and registers default port is six three seven nine so so let’s see who traveled to Redis conference within a ten-mile radius start off with that okay we see DFO day right here he’s four miles away Redis conference that’s you know this place we’re here zero miles and we have a lot of people just within the general area and if you want it to pass in another parameter for example like another radius I mean 50 miles you get more but you see now that they’re not in really any logical order so another parameter that you could pass to it right here Sony close this so actually yeah what this does is it passes in the Twitter handle and the radius as a parameter and the Twitter handle here is the member we store those as variables so now we use the geo radius by member command and again it would be geo radius by member key which is the reddest conference attendees the member and the radius which in the last thing I just demoed was 50 miles there again as Andrew mentioned it defaults to meters and you could pass another options like kilometers as well but for the purposes of our app it has to be miles and with distance which is one of the options but let’s say you wanted some sort of order like who came the nearest or you want to see those displayed first you could put in ascending so let’s do that so now you see they’re not in any order but if I did the same thing again Redis con 50 miles away so now they’re in order we know that and done Nicholas who is also from coding house he actually lives in San Francisco he’s so he’s like right around the block and then all these people further away from like within 50 miles I know that Salvatore came from Italy so if we wanted to demo that actually it would be nice to see that in descending order to see who traveled the farthest you would just pass in this parameter sorry one second

descending and let’s just for the sake of this try like 8,000 miles just to see everybody in our database stored right now so we see that Salvatori did win for furthest traveled six thousand five hundred sixty seven miles away and we seems like a lot of people traveled pretty far to I don’t know if any of you guys see yourselves up here but register man six thousand four hundred ninety two miles away that’s pretty far if you’re here just let us know where you where you came from yeah and then you can go all the way down right here right his Kampf another option that you could pass through to the geo radius by member command is with cord so that would just also return the longitude and latitude if for whatever reason you would need it for example to use it to generate a map you can use that so let’s do that I know I keep demoing red as comfortable demo someone else later let’s see miles away so that as you can see also displays the longitude and latitude if you pass in the with cord command and another option that you could add is with hash but as Andrew mentioned before it just returns a 52 bit unsigned integer it’s of no practical value to the user so we won’t be demoing that so yeah those are the options you can pass and let’s just do one more thing so let’s see just for demonstration purposes whoo how far everyone is from Salvatori in italy so yes someone’s at Barbie I don’t know who fill that out oh yes so we see everyone so relative to Salvatori everybody here or most of us is actually really far away from him all within like 5,000 upwards of five four thousand miles away from him so that’s how you you know those are some sample use cases and right now I’m just going to also show you really quickly the app that we built which is called pace me to see a more practical use what this app does is you want to go find a running buddy near you but you don’t have anybody near near you that you know to run with or who runs your pace this app would solve that problem for you so I’m actually logged in but let’s show the register view if you were a guest I mean the guest view not logged in so what you can do is you can just go here find a buddy and you could type in a zip code so let’s say people who are here at the Mission Bay conference center one second was oh nine four one five eight my bed yes so you can see the results are immediate these users are already in our database Redis and the people in the top four actually the four of us who worked on this app and then a lot of other people have also registered for you know this and a lot of these people are actually pre generated dummy users so that’s why a lot of the data is slightly nonsensical but for the most part up here on the the first couple rows they’re actually real people who registered for this demo app also not actually deployed or anything and you can also change radius so another San Francisco radius so yeah again you can see the results are you know immediate and one more we actually came up from Fremont so that zip code is nine four five three nine so you know a lot of users but again with this guest view the options are limited we only limit it to a default of ten miles so you don’t see that many people but if you were logged in so if you were logged in so this is my profile you have a lot more options here this is relative relative to me I signed up with the zip code of this area which is nine four one five eight and all these other filters like sort by gender age range or you know ascending descending mile pace that’s actually front end filter but the miles that you’re passing in that’s all making calls to read us in the backend so I’m so here these are the people who are within 10 miles of mate runners within 10 miles of me in this database but let’s say I wanted to do let’s say 50 miles you can see almost instant whoa sorry I didn’t mean to click on his profile you can see almost instantly that change it returns back other users so the code really quickly I won’t I want to show you too much of the code that we did if you want it to see it and also for the demo app we will stick around if you just wanted to see the code that we actually use but we used a lot of like uh you know the geo radius commands H set H get all which we can go into later and yeah that’s that’s it does anyone have any

questions yeah well well take some questions we’ll try to answer them it looks like we finished a few minutes early and if anyone has any questions yes I actually just have some some observations after using reddit for doing geo and Xing for a couple years so before this is my first real exposure to the geo commands so there are some difficulties I guess with with this approach once you reach scale so this would work great once you’re on a single Redis but once you have to shard you’re gonna run into a lot of trouble because you just can move between so what we do at lift actually we track a ton of data and I guess the first difficulty that you have is that you want to be able to expire data out of the set so if we had every driver that had ever kind of driven around in the data set and then we tried to pull that data back every time you were looking for a nearby car there would be a tremendous amount of data so we have to have a way to kind of have recency for that data so you could go and clean it up manually I suppose so that’s that’s one thing to consider but yeah then the other thing is once you want to shard it onto multiple instances I the Geo I don’t believe you’re gonna have to build a lot of custom stuff like we’ve done so if anyone is interested I can talk more at like about it but the geo commands are really cool if you’re just getting started definitely yeah thank you that’s great because our experience is very limited with geo so maybe he’ll be able help you with some more difficult questions he saved us I think any other questions yes the code for paste me yeah we can yeah we could push the I’ll put the link up if you is it on yours or Sergio’s yeah if you go to the pace move so we just put our contact info here at the bottom instead of trying to list all of it out for everybody and if you click on Sergio he has it and then I’ll just tweet out a link for it yeah we can just open it there he is and join a burger and it’s gonna be one of these ones that he just did read his conference sample app so it’s he has this weird username c ze CH h at Redis code on it at read his comp sample alright yet / furnace comp sample sorry yeah so it’s a pretty simple app and again that was just to kind of show you the basics of how it looks different in node versus how it looks on the command line and I’m sure if you built anything and notifi used Redis NPM package or Rhetta say oh I think we used Redis I’m trying to remember now the Redis Iowan wasn’t supporting the geo commands when we first tried to use it because it was still in the testing branch and so actually we had like a lot of problems building our app and deploying it because there wasn’t any like AWS didn’t have just a package you could just use version 3.2 or you couldn’t just use it on the cloud on Redis labs cloud so we had to like manually install it on a table us on an ec2 instance so I’m sure that people with more experience could have done it in a few hours and figured it out but you know it took me a second amount of time together to figure all that out but yeah it was a great learning experience for us anyways and I’m sure that all that’s gonna be much simpler for everyone going forward if you check out my side I have a blog post about how to install it on ec2 instance so if that’s something that appeals to you or you really want to launch something that’s available for people to use instead of localhost it’s not too many steps but it’s very clear and I also had a teammate test it all the commands on a fresh instance to make sure we didn’t miss any steps for you which seems like when I look for documentation there’s always this step that they didn’t tell you that I should have known but as a beginner it was hard for us are there any other yeah there we go can you come up to the mic or no you can come up to your point about installing it I was to be able to do it in about four minutes with the docker container yeah see so somebody knows what they’re doing like it’s just so doctor yeah and I installed it like a month ago so there wasn’t maybe it’s something they’ve already been on top of it and if you have it installed on your local machine yet it’s super easy just homebrew will do it that’s what we used or the directions are on the red style I Oh site and if you haven’t seen actually they made another change here which I was really excited about this one as well since we have some extra time when you look at the documentation on Redis IO commands and then you’re at a specific command down here used to just show you the data but now you can actually interact with those commands so we could like run the same one they just ran if we wanted or we could use like zero move and actually delete somebody out of there and run as easy range and see what members are within a certain distance so that’s pretty cool if you’re trying to learn this stuff you can play with it right there instead of having to go and play in your command line and figure out how it works any more

questions I see one sorry is the distance between two points or also including the angle is how you asked me my understanding is that it’s a straight-line distance like as the crow flies is we would say I don’t know he this guy knows this is why we need the advance people here saving us oh yeah yeah right yeah straight line on a sphere it’s the it’s what’s that word hammer sign right yeah Hammerson model of the earth and some of that is in the documentation I don’t know from a geo command here it doesn’t mention on this one but you can find that on the documentation it’ll mention it it’s like a sentence it says that’s the model they use that was one of the little gotcha so we try to cover it okay nice any oh there’s another hand yes yeah the zip codes I don’t know if she has that data up here it it’s just the a Google API we used so when you typed your information in on the little demo app we had the link up to at the beginning that was actually making a it was actually making a Google API call and using your zip code and we were storing the latitude and longitude so we weren’t storing your zip code were storing the latitude and longitude we got back when you typed the information in originally and in the app we actually demoed just made a call of that database and wrote it on to our localhost because of those kind of problems that apparently aren’t as hard as we made them out to be but we didn’t want to have to like deploy to AWS and set up the ec2 instance and everything for this demo app so we just did it on localhost yeah yeah my understanding is it’s just trying to do some sort of geometric centering of the zip code and again for this demo we weren’t too concerned with the like really specific accuracy and with the paste me app too we’re also just using the zip codes passed in so that’s like a an improvement we could make to the paste me app is actually like using your phone and refreshing that location each time but we built this for a part of our class so it was more about learning all these different parts that how they kind of fit together you can see we used a lot of different technologies which we’d use node before we hadn’t used oh and angular sorry we used angular node so all the other stuff was kind of learning for us I just I’ll try to do most people here that in the back or I can sort of try to summarize it in a sentence or two do you all here back here okay so the first question was is there a plan to allow you to define what accuracy you want to use and Salvatori said that there isn’t really because you because it’s being stored as that sorted set you need to define what accuracy are gonna use so that it can compare them you can’t have one that’s like one decimal place and one that’s ten and try to rank them it’s gonna be a little bit harder your start having things tie you want them to be unique more or less okay yeah he’s like well yeah right it’s the 52 bit integer I talked about that I didn’t really show is what the score is in the sorted set so it needs to have those scores so you can see like who’s winning or losing or who’s that far away it’s gonna look in that list and say these people are that far ahead or that far away and that’s how it gets the distance or the radius and then the second question was is there a plan to use something other than the sphere model the hammer sign I’ve read it but I never heard it said before I Oh other than radius for circles I’m sorry yeah okay so if you want to use like a polygon like a triangle or an octagon or something and he said that’s something that they’re working on maybe thinking about doing because you can start to get some really interesting models yeah all right yeah I think we’re we’ve got a couple of minutes left if there any other questions and we’ll have this little sample app up if you want to just come and type in your Twitter handle and see if someone was near you or ask us some other questions that’s fine as well thank you very much