Michael E Anderson – extending android via external microprocessors | ELC 2014

so how many of you this is your first trip to embedded Linux conference or Android builder summit most of you Wow okay good yeah I’ve been speaking it embedded Linux conference now since 2007 and Android builder summit when they started having those I started speaking at those as well spoken on a lot of different topics most notably I was a keynote guess was two years ago on talking about the Internet of Things what Tim was talking about and we’re seeing a lot of changes that are happening especially to some of the mainline operating systems like Android and how it plays with the real world I mean it’s really interesting and that’s probably what’s going to be the topic of this particular discussion we’ve still got another couple minutes for people come in but when we take a look at how much progress Android has been able to make over the past a relatively few years I mean we’re only talking what’s for year well 2007 was when google acquired the android folks from what used to be danger inc and they’ve made some pretty big strides they’ve had their hiccups of course we all know about the problem that came about when google decided they were going to go off and do their kernel and not interface with the rest of the real Linux community I was very fortunate to have gotten a google nexus one as a we’re sorry gift from android from the google folks they showed up at the embedded linux conference and said they had two pallets of cell phones and they go we’re really sorry we screwed up the colonel community would here take this and maybe this will help us ways your ruffled feathers it did for a little while at least till it was till I couldn’t upgrade the phone anymore and then it’s like okay well it’s a nice little handset but not much else all right well we are currently on time so let’s get working here my name is mike anderson and i’m the chief sign scientists for the PTR group we’re a small 35 person engineering firm we do a lot of custom work for not only commercial organizations but also the government in related in relation to porting Android porting Linux to new platforms so one of the major things that we’re starting to see is that as Linux starts to become an android of course obviously starts to become so pervasive we’re now in a position where management is coming into a new project and saying what I really want is I want to be able to have this really cool GUI sitting on top of everything and I still need to be able to do all my real time control I still need to be able to do all my embedded systems work but I want to use this Android front end and it makes a lot of sense when you think about it from the perspective of how much does it cost to actually deploy something in particular training and in many cases we tend to neglect training as an aspect of all of this because training doesn’t really factor into the calculations all that much until you really start thinking about how am I going to ship a product what do I have to do in order to be able to get the user experience in such a way that I don’t have to have a support desk to set there and answer phone call saying I don’t know how this works so fortunately one of the things that’s happened for us and I really have the guys at Apple I really say thanks to those guys because they did a lot to establish the whole concept of the swipe and the pinch and zoom and everything in everybody’s mind so that now if you take a look at the install base for smartphones and tablets it’s billions of devices we see that now actually here in the United States the carriers where you see t-mobile for instance getting ready to buy basically your contract out from verizon or somebody else in order to get you as a subscriber the reason they’re willing to do that is because they’ve hit ninety-five percent saturation of the market ninety-five percent of all of the people who could buy smartphones have them there is no more market to be had there is no new niche to push into if you’re a smartphone vendor however as Tim indicated in the discussion that we just had in the keynote there are new

markets that we can push into with Android and that’s the key of what we’re going to be talking about here in this presentation so we’ll take a quick look about Android at Android in the outside world and how we deal with devices in the outside world we’ll look at strategies for adding new sensors now the traditional Android approach and then the approach that we’re going to talk about in more detail here which is adding external micro processors will briefly touch on what does it mean if we would to say that we had a real time Android what exactly would that be and why leading from the real time Android why we need to seriously consider the use of external micro processors for doing most of the work then we’ll take a look at some interesting approaches one is to actually write code for the microprocessor or the microcontroller the other is this kind of weird approach called fermata that you basically turn the external microprocessor into a dumb peripheral versus using it to actually do some intelligent processing a couple different strategies there so let’s kind of move forward here yeah yes they should be I I gave them to the I gave them to the Linux Foundation let’s put it that way you should be able to download these if not now shortly but and of course you can always send me email Mike at the PTR group com if for whatever reason you can’t find it then by all means send me an email be happy to send you a copy of it all right so how does android deal with devices today that is physical sensors and things of that sort well Android in lib sensors has got an implementation for many of the most common sensors that will encounter in a cell phone in a smartphone we’ve got gyros and accelerometers and a compass and a GPS and all that sort of stuff all of that’s integrated in through lib sensors and the sensor how so the hardware abstraction layer is there specifically to try and abstract the details of the Linux hardware driver implementation from how the user will see this device in the Java framework adding a new sensor to the platform let’s say we’ve picked a sensor that does not exist we want to do something through I squared C and there is no existing sensor that uses that particular interface that’s available on our phone how would we go about adding that well normally that would mean that if we were to going to pick a sensor say a heart rate sensor for heartbeats and heartbeat monitoring I mean we’re starting to see a lot of this stuff happening now for Android in the medical community where they want to have the equivalent of a halter monitor for those of you who ever had heart monitoring done for you well the holter monitor they put all kinds of little electrodes on you and then they have a little recorder which is literally a cassette tape recorder and it sets there and it records your heartbeat over multiple days and then they play that back through and they kind of digitize it and look at it go well he’s got a problem with a rythme or whatever well if we were to have something like that that would be implanted inside of someone we’re interested in monitoring 24-7 and had that then tied to his smartphone and I say he in the sense of the generic third-person sense of the word not male versus female but have it key to their smartphones so that if we think about how to resuscitate someone who’s having a heart attack there is this thing called the golden hour and if you can get medical care within the golden hour the first hour after the event actually occurs chances of survival are outstanding every minute past the end of the golden hour things start to deteriorate pretty quickly so if we had a way of adding a sensor to android so that when that heart attack first occurs it immediately reaches out and contacts the doctor it determines based on the GPS location the closest hospital and contacts emergency care facilities so that they can then reach out and have somebody there within that first hour that would be a tremendously powerful tool however in order to add such a sensor to Android we would have to download the AOSP sources we would have to go in and modify lib sensors we’d have to add an a Linux driver to the colonel we would then have to physically figure out how we’re going to connect it all of those things come into play here where we might be able to do it for a single phone or a single platform but the chances of us being

able to do that for every phone very slim so how do we then say all right how do we add a sensor outside of the android open source platform and how do we get it integrated in in such a way that we don’t have to make major modifications to the actual handset or the tablet in order to extend it for this new capability so it turns out that the real world is filled with this kind of stuff we’ve got canned bus we’ve got a 2ds digital-to-analog zwi got i squared C motor controllers a lot of those use PWM pulse width modulation so I squared C of course obviously spy Twi lots of different connectors lots of different interfaces out there and the question is how do I get those things wired to the physical sensor and then how do I get the android platform to be able to support them well we could create custom hardware I mean I could make a connector that’s going to wire the I squared C out in some reasonable fashion so that you could then plug in a new sensor that was based on I squared C technology and be able to attach it the Android platform physically but once I get past the physical attachment then I need to go in and make some modifications to the Android colonel and I need to make modifications to in droids how as well as lib sensors in order to make all this stuff function unfortunately the Linux kernel that they use an android isn’t really tuned even for soft real-time we’re not running the preempt desktop the desktop preemption a model we were using voluntary preemption for the most part so that’s not going to give me terribly good even soft real-time performance so if I’m trying to sense things in the real world in real time I’m not going to be able to get there with the out-of-the-box android platform so we have some other options here android of course natively supports bluetooth Wi-Fi USB and NFC in some instances depending on whether you got the NFC hardware out there so using one of these connections we could potentially interface an external device and then offload all the time sensitive work offload all of the expensive real time constraint kinds of work off to the external microprocessor and then simply use Android as a front-end a data collection platform a user interface something that would be a communications interface for instance it might then reach out to the service provider so that we can then have medical community dispatched so we have some kind of conflicting goals here though we would love to be able to keep our costs down we want to be able to add this new feature add this new sensor whatever to an Android platform but not end up extremely it making the Android platform or expensive we don’t want to have to redesign the hardware we don’t want to have to spend thousands of man-hours going in and well hopefully not thousands but at least hundreds of man-hours going in modifying the android open source project writing the driver the drivers modifying the hal getting all that stuff built rebuild the OS etc etc we’d like to be able to reduce the cost but still be able to guarantee service and that’s where we really find and this was the thing that Tim was alluding to in the keynote having two big processors sitting out there that is having an arm cortex a15 quad core and an arm cortex a8 sitting out there in the sensor that’s a lot of money those even the cortex a8 SAR still pretty expensive processors and they have fairly large Ram requirements cooling requirements power requirements the whole swap thing sighs wait power all come into play so we really want to be able to do something with a very small controller that can provide us the real-time control provide us the ability to do what we need to do but not duplicate everything that’s in the Android platform we want to use android for what it’s made for not necessarily for doing all this external control so there have been Oh power oh yeah he gets down here they have it squirreled away there you go so we have a couple of things that we

could do we could if we’re really trying to do some real-time embedded kinds of things for Android we could say well let’s extend Android and there’s actually some folks at University of Buffalo edu who are working on how you would make Android real-time capable now making a real-time capable Android is not a trivial thing even with all the work that we’ve done in Linux with a preempt RT colonel that’s not enough in order to make it play in some reasonable way for Android we have really you know when we ask somebody what is real time and I actually have seen a video go where they went through and they kind of interviewed a whole bunch of people what is real time what is embedded nobody could really come up with a satisfactory answers it turns out because it depends on who you talk to but the reality is that real time basically means we have computing with a deadline and the question is what happens if we miss the deadline if we just miss the deadline and we go on to the next sample period okay that’s soft real time if we miss the deadline and the wings fell off the airplane that’s hard real-time so you know we have to understand what the consequences are of missing the deadline and there have been a lot of attempts that have been made for android trying to make it real time capable so there have been folks that have gone through rebuilt the kernel for Android turn on preempt RT turn on all the wonderful features that are part of the real-time patch and be able to support soft real-time with reasonable efficiency inside of the Android kernel itself if we were using an indie k application native development kit application in C or C++ then we actually might be able to achieve some reasonable level of service yeah are there some parts of the advocacy the kernel of which don’t take into account free and dirty nice stuff fortunately most of the things that are in the kernel right now the ones that you be most concerned with would be a shmem and the binder those seem to behave okay with preempt RT turned on but I’ll be honest with you I have not set there and stressed them to any significant level it’s been like a turn Amman oh wow the colonel doesn’t crash that’s good you know will say that’s success but that’s unfortunately not enough the big problem the kernel itself doing real-time that’s really not the big issue here the big issue is the entire android framework the android framework in spite of all the fact that all the people that have looked at this i mean we’ve had some really talented developers looking at the android framework quite a long time now but the problem is that the lvm is still a vm it is optimized for particular types of behavior it’s optimized to make a handset a smartphone a tablet behave well for user interaction but anytime there’s a human in the loop we immediately have soft real-time at best because I don’t care how fast you blink the the planes going to fall apart light if the user is drinking a cup of coffee and they don’t notice it the plane falls apart you know there’s nothing there that we can use in order to enforce the dalvik VM to do what it’s supposed to do I mean we have the advantages that the dalvik VM is not a stack-based vm it’s a register-based vm there’s a lot of things that they’ve done the garbage collection has been made better than we had in a typical Java but realize that the real time job of community when they say real-time their target is one millisecond that’s their target and one millisecond is good for a lot of problems don’t get me wrong but there are a lot of things like motor control that it’s not good for because at motor control we start talking about 400 kilohertz and faster refresh rates in order to keep the motor spinning correctly so there has actually been these guys at University of Buffalo have looked at replacing dalvik and putting in a real-time java implementation but they quickly found that that was not a viable option the amount of effort that they were going to have to go through in order to make this work was just two owners so they may may revisit it don’t I mean they’re a university and you know university you have lots of cheap late i’m een slave leia cheap labor you have graduate students and so you can use that in order to research things that would normally not be economically

feasible in the real world but whether they’ll come back to that or not I don’t know they’re certainly looking at it yeah what about androids new arg that port forward in the place dalek with parity right so that does a lot more for you whether or not you know I haven’t had an opportunity to sit and actually characterize the performance of that because you have to do it in light of having the preempt RT colonel in place so it’s not just oh I’m going to turn on a RT and it’s going to solve my problems it’s no no I have to turn on the preempt RT colonel I got to rebuild the colonel I gotta turn on a RT I got to see how they interact with each other a lot of stuff has to happen in order for us to even come close to being able to say we’ve even tested it sufficiently yeah yeah okay leaving that part aside if you’re not using solid p.m. you’re basically ignoring all the Android user applications all right in if you’re doing that then why not just use an ordinary that it was oh absolutely well the reason that you still use the Android piece of it is because what you would like to do is you’d like to be able to get the whole pinch zoom sweep swipe all that sort of user interface the user experience part you run the NDK underneath it and it turns out that they’ve made a lot of extensions starting with ice cream sandwich they started making more and more extensions for the NDK so that now when you’re looking at games and a lot of the really high speed applications under android they’re really not running in dalvik yeah so we have to really understand when we talk about embedded and real time they are not one in the same we have to realize that an embedded computer is basically anything that you inherently know there must be a computer in there somewhere but you’re just not quite sure where that’s an embedded platform that camera right there we have TV sets we have printers we have routers all these devices we inherently know there’s a computer in there somewhere but there’s no keyboard mouse monitor interface to that thing now on my TV there’s a little Jack on the back of it that says console and if you understand how to hook up your bus pirate you’re going to figure out that it’s a serial port console that gets into the TV set but my wife takes a very dim view of me hacking our TV set while she’s watching it especially oh this won’t do anything oh sorry i’ll have to reboot you know TV sent where’s the control out the lead on the TV set anyway oh yeah alright so the embedded space is actually quite a bit larger than the real time space and the real time space just simply means we’re focusing on deterministic behavior now determinism isn’t necessarily faster and that’s one of the kind of the interesting problems that we run into with the preempt RT patch because we can get very deterministic behavior out of preempt RT with a little bit of tuning but it doesn’t necessarily mean that it’s faster than the non preempt RT patched kernel and turns out in many cases it’s just deterministic it just takes the same amount of time every time and we’re willing to trade performance against determinism in the real-time world so then if we have the kind of intersection between real-time and embedded now we have airbag deployment any lock brakes those kinds of applications fall into this well it’s an embedded application certainly there’s no keyboard mouse and monitor on your airbag deployment it has to be real time because it turns out that there was a issue many many years ago where airbags on the on the passenger seat were killing people because it turns out that the airbag on the passenger seat was a little bit further away than it was on the driver side and they were using the same timing constraints so you were a lot further forward when the air bag deployed and it cuts your neck and snapped it bad problem so uh we see those kinds of applications are really the intersection of real-time and embedded well it turns out there’s a whole bunch of those kinds of problems that exist out in the world and we would like to be able to leverage what Android does but still be able to guarantee t that we can meet our performance requirements so if we take a look at an out-of-the-box android device it really isn’t capable of doing deadlines based computing without significant modifications and even if we did do the modifications we’ve modified one Android platform not all Android platforms so that’s a big problem for us if we’re trying to turn this into a marketplace especially in the medical community so

we’d like to be able to offload all the real time constraints we’d like to be able to offload all the weird electrical interfaces out of Android to something else and then let Android provide the UI and the user experience this is where the microcontroller start coming in so what are we talking about with an external microcontroller or a microprocessor depending on where where you went to school there are a lot of very popular ones that are out there they’re either 8 16 or 32-bit depending on which versions we pick some of them may have a real-time operating system on them a lot of them don’t a lot of them are running bare metal so when you’re running bare metal on top of a platform you really have to understand how that platform the boot sequence you have to understand how it initializes code you have to understand what it’s going to run does it have multitasking is it interrupt driven is it a super loop what is it so it definitely implies some issues in terms of understanding how the platform runs but there are a lot of really popular ones out there we see the Arduino this is the atmel AVR processors the pics pick 24 pic32 s msp430 s there are a lot of these little micro controllers out there and they are selling for dirt cheap when Tim was talking about not having a fifty-dollar cpu but having a ten-cent cpu these are the kinds of CPUs he’s talking about Tencent cpus and less so some of these are really really cheap you can buy you know full up development platforms for msp430 for five dollars that’s the board the chip the IDE tools everything for five bucks launch pad launch pad yep I’ve got three or four of them when they come out that cheap I got i’ll buy them and then i’ll give them to my students I i’m a software mentor for a robotics team on a high school robotics team so i’ll buy a bunch of them and I’ll just kind of give them to the students to say here’s what I want you to do I want you to make the robot have blinky lights on it here’s some LEDs here’s some resistors here’s some wires here’s the controller here’s the development environment go for it and we see how many of them are successful and those that are successful I want on my software squad those that are unsuccessful I send them to frame and bumper you should really help these mechanical guys over here all right so the problem is that each one of these things has got its own kind of development environment along with it here’s some examples this is the launch pad from TI we have the arduino of course this one is an arm cortex m3 that one’s from nxp and then we have the yo-yo board here some of these have actually been modified specifically to deal and work with android and that’s really one of the cool things because we really have two different approaches to the problem here one is we can write code to run on the microcode on the microprocessor that means we have to have the compiler we have to understand the development environment we have to understand the boot sequence and things of that sort and then we write the code and we run it on the microprocessor and the microprocessor handles the real time constraints for us it handles the i/o the weird funky little I squared C and those other kinds of iOS that are coming off the boards in many cases and does so in its own unique API so you’re not going to be able to leverage your Linux experience here it’s going to be something completely different this gives us the ability to offload work and use the microprocessor as an intelligent buffer and then simply give commands or give information give data back over the Android side so the Android can make command and control decisions whereas all the real work is being done in the microprocessor yeah a question so are you talking about the ATK partially yes and we’ll get to that just second because the adk was something that Google did that unfortunately it really hasn’t taken off and that’s that is really unfortunate because it had a lot of potential and a lot of it was because well it will get to the mountain will get the reason why in a second all right there is an alternative approach and there’s this thing called fermata in the ant in the Arduino world they call it fermata basically it’s a special type of firmware and we’ll get to what that is in just a second so here we’ve got some example development environments this is and this is the Arduino development environment over here we have the launchpad development environment this is based on Eclipse so there’s a lot of different IDEs a lot of different gooeys they can run in windows they can run in OS 10 they can run in Linux in many cases it just depends on you basically

you buy into a religion here if you’re going to use our dwie knows you’re buying into the Arduino approach if you’re going to use pic 24s or pic 32s you’re buying into microchips approach going to use the msp430 you’re buying into TI’s approach so you end up having to kind of say hey this looks like an easy one for me to work with I’m going to go this direction maybe it’s the right direction maybe it isn’t maybe you have to change a little bit later but at least you got something started so I mentioned this thing called fremada it’s actually there’s an open website out there it’s an open standard the microcontroller world especially the Arduino guys support this thing called fermata and what it does is it’s a firmware that has a serial port interface on it and it simply waits for a command and the command says which port you want to talk to and what value you want to read or write to that port so it basically it takes the microcontroller and turns it into a dumb peripheral everything is done over on the Android side the Android sends the command to it the peripheral just simply does what Android told it to do the pluses and minuses of this our first of all with fermata and of course the yo yo board which comes specifically for the ADK uses uses this basic same approach it’s not the same code unfortunately it’s not compatible but same concept the advantage of doing something like this is that I now don’t have to write code for the microcontroller basically the microcontroller has one firmware load and one firmware load only and all of the real work has to be done over on the Android side of course that doesn’t solve our problem of real time constraints because Android is still decidedly non real-time but at least it solves our electrical interface problem how do i generate a PWM signal how do I deal with a 12-bit a 2d those kinds of things can be done with fermata and I don’t have to write any special code on the Android side to make that happen I just need to be able to communicate to the device that’s running Fermata and there are a lot of them out there if you go out onto the google play store you’ll find a lot of these applications that are built specifically to deal with this kind of interface so it’s popular enough that there are probably 30 different apps out on the App Store that all work this way yeah so it’s a little unclear to me so are these all going over the USB communication or it can be different kind and be different a lot of them actually go through bluetooth okay and that’s because that’s kind of the easiest thing to get running with fermata the the yo-yo board can use either USB or bluetooth we’ll talk about connections here in a second yeah it is fermanagh intended to be a specific implementation or a generic term well it is a specific implementation that has become generic it’s like Xerox we used to say we’re going to Xerox something now it’s just kind of become a generic term where or it may think you remember what many things can be fermata firmware it’s an open source project so you can actually download the source code of fremada and port it to another platform I’ve seen somebody who’s ported it to the Raspberry Pi for instance there’s a speck yep there’s a speck for the protocol and it’s a serial type protocol and this is one of the things we start to see more and more often even when we’re dealing with things like USB in the android accessory development kit it’s all about serial type communications so for those of you who have never dealt with serial ports before well you’re going to have to learn about serial ports because whether they’re bluetooth or their USB they’re going to look like a serial port and that’s why in Bluetooth it’s actually called spp the serial port profile in bluetooth because it basically takes everything makes it look like a serial port so should you program the microcontroller or should you use for Mata well there are a couple of rules in embedded development work number one rule embedded in real time is never trust the user always assume that the user will do the worst possible thing to you at the worst possible time and program that way and that will minimize not eliminate but minimize the chances of you having a catastrophic failure in the system the second rule of real-time development is no matter what your management asks you the answer is always well it depends management hates this answer but unfortunately it’s really the only answer we can give when we’re dealing with embedded in real time applications because it almost always depends on something do we have enough memory do we have enough bandwidth and CPU is the bus working correctly there are so many factors that come into play in trying able to meet deadlines or constraint oriented computing that

there’s no guarantee until we try it and then we may find that we need to tweak this bus setting somewhere maybe the read-modify-write cycle isn’t working correctly whatever crazy stuff happens using the from auto approach is a quick and easy quick and dirty way of doing something you don’t have to worry about the microcontroller you basically abstract it and make it a dumb peripheral but that means that now you’re going to be doing a lot more programming on the Java side or on the Android side whether you’re using the NDK or you’re using Java and you’re going to have the electrical interface but you’re not going to have the real time capabilities that you would have if you program the microcontroller it we program the microcontroller then we can use the microcontroller for interrupts we can use the microcontroller for all kinds of things but it means that we have to learn a new environment and then the interface between the microcontroller environment and the Android environment so how do i connect to the microcontroller well serial ports are definitely there almost all these little micro controller boards have got a serial port on them whether one of the things we need to watch out for these serial ports is there often not level shifted that is they are TTL level signals they are not compatible with rs232 voltages so you cannot simply plug the wires into your 9 port serial board and expect it to not blow the microprocessor up so you go to places like sparkfun Adafruit a lot of these vendors out on the web sell little level shifter boards that will go from TTL level voltages to normal rs-232 voltages yeah this Q much to things like 3.3 volt yes and that’s the other tricky part about it and actually this afternoon i’ll be doing a presentation on how you build a linux-based robot and it’s that 3.3 vs. 5 volt thing that came back and bit me nastily head because it has to do with the BeagleBoard and the BeagleBoard likes 3.3 volt and my interfaces were 5 volt so i had to interpose an Arduino between the two in order to get them to play nicely we’ll talk about that this afternoon though so we’ve see a lot of things 80 to 250 not for that’s zigbee quote unquote actually today it’s becoming 6lowpan now 6lowpan is ipv6 over low performance wide area networks which is why it’s 6lowpan with 6lowpan we’re starting to see this show up in skata implementations now skate is supervisory control and data acquisition it’s basically in the industrial process control business so if you make petroleum if you do chemicals if you do anything that requires process control of any significant way you run into skate assist so we’re starting to see 80 to 15 for show up in SCADA USB of course ethernet Wi-Fi all of these have some native connection to a lot of these micro controller boards and regardless of the transport layer whether we’re using bluetooth or cereal or USB they all boil down to serial communications it’s always I’m going to format a message i’m going to send it across the serial link it’s going to be intercepted by the microcontroller interpreted the microcontroller will do something with that message and then send you a message back that you then have to decode there’s no standard in the way those messages are written you decide what the standard is for your application now oh one other thing some of these devices have Wi-Fi interfaces and Ethernet civ ailable for them in those cases they look like sockets so that’s another option a lot of people pick because sockets are easy to understand and I can go out on the web i can see billions of examples of how it’s done we end up with something that’s basically a nearly universal connection and its really driven by the size and pervasiveness of the Arduino ecosystem there are over a billion Arduino ‘he’s in place today they have been around for a long time and many of these third-party boards have actually adopted the Arduino pin out as a way of being able to leverage the Arduino mezzanine cards called shields so we actually end up there a little over 300 different shields that are available and that’s a real working system by the way it’s like a really easy one for a similar idea yeah yeah it’s a it’s except these are much smaller than PC 104 and we don’t have to pay any licensing royalty rights because this is all open source hardware so the Arduino is open source the shields are open source everything is open source you can get download the Gerber’s you can make it yourself it’s great so there are a lot of shields that are available

midi SD card lcds you name it it’s probably got a shield available for it and there’s some really cool things that are happening that use the shield pin out so i’ll talk about that in a second so here’s the typical Arduino pin out we have power over on this side so there’s 3.3 volt there’s 5 volt there’s a couple of grounds I have a 2ds here these are 10 bit a 2ds I have on the Arduino do a which is the new cortex-m based platform it has a 12-bit a 2d i have p WMS everyone that has this little dash on it that’s a pwm capable line we can use pwm is an analog output so if you understand how voltages work if I could have a voltage at zero volts and five volts and if I switch that voltage on and off fast enough I can average to any voltage in between just by changing the duration of how long I have it at 5 volts versus how long I have it at zero volts so this is how we approximate analog output using PWM signals so if you’ve ever seen an LED that kind of goes bright and dim bright and damn thats PWM so you’ve seen it you just may not notice that you knew what you were looking at these are digital i/os on this side they can be either inputs or outputs these this particular platform has to interrupts coming in and we’re starting to see this particular pen out being used in a lot of different places so here we have the Arduino tray board this hasn’t been released yet I’m on the beta for this we have a this is a BeagleBone black right here and that’s an arduino leonardo chip so this is the atmel 32 you for processor on one board so it’s got ethernet it’s got USB you can then plug in shields into it or you can plug in capes if you want to use capes on that here’s the you do board so that’s a 1 gigahertz satara this ooh do board uses the imx6 so freescale imx6 you can get it in a single dual or quad core version and right in there is a cortex m3 that happens to run our dwie nose software so you have a serial connection from the cortex this one the imx6 into the cortex m3 and then we have the Arduino pin outs this by the way this extra stuff here this is the Arduino mega pan out so it uses the Arduino standard pin out and then adds a whole bunch of extra circuits to it has 54 digital i/os instead of 14 we see the Intel Galileo board this one is a quark processor it runs linux natively it does not run the arduino code but they have an arduino ide that intel has ported to it so it uses the same pen out so we can use the same shields this one I think is 5 volt compatible a couple of others this one is a an embedded x86 with an arduino mega pin out there’s the gert we know board so this is a standard arduino chip with a plug / module that plugs into the Raspberry Pi which is a nice little alternative again it has the same pin outs and then there’s the Arduino do a which is an Omega pan out plus the cortex m3 nice processors that give you a lot of capability and we’re seeing a lot more of this okay nobody can really out do what the ecosystem what the Arduino ecosystem has been able to generate so people are basically saying screw it I’ll just combine our dwie nose with my platform and we’ll go from there so this is the basic layout we’ve got 14 digital i/os on this board 6 analog inputs 1 uart jtag and two 8-bit timers one 16-bit timer we also have some interrupts associated with this the mega goes from 14 digital i/os 254 digital i/os again all of this is open source you can download the Gerber’s make changes and send it off to a fab house and within two days you’ll have new boards back based on all of this circuitry that you didn’t really have to do anything in order to make a new board we had of course they do have the plug overs for both ethernet and Wi-Fi so we can get some pretty decent connectivity out of this the Android adk which is the accessory development kit this was introduced by google in 2011 and it was originally designed as a USB connection between an arduino and it turns out a yoyo board a pic 24 board into android

and we actually see it it’s actually being deployed today in a lot of exercise equipment where you plug your smartphone into the exercise machine and then it keeps track of where you are in San Francisco so you can actually ride you can ride the tour de france for instance with one of these things and it changes the elevation of the bike as you’re going up down up and down hill it changes the amount of tension that it takes in order to pedal all that’s being done by an external microprocessor is not being done by the Android side but the Android side is being used to visualize what’s happening they did release in 2012 ATK to it added support for the cortex m3 so that’s a fairly powerful processor in its own right but it turns out the ATK is really just a protocol specification every Android device since gingerbread has shipped with the ATK libraries so it’s something that’s available in any Android device if you just simply plug it in it’ll automatically detect the device and say hey you’ve just plugged in an accessory would you like for me to download the apk and you can actually specify a website that it will go to the website download the apk and side loaded into the system if you’ve allowed for external downloads it has been ported to the Raspberry Pi by the way that was actually at embedded Linux confer actually the Android builder summit last year this individual Gary he announced what he’d done he had ported the ADK over to the Raspberry Pi so now you have Android and you have a Linux front end which I don’t seems a little counterproductive but nonetheless it was kind of a cool thing so android of course has an issue when we deal with USB because android devices are expected to be devices not hosts so when we plug in an external microprocessor the microprocessor has to behave as a USB host and that normally requires some additional circuitry in order to make that happen so the USB on the other hand once you’re plugged in and you’ve got the established connection it just looks like a serial port stream so you’re just simply writing messages back and forth it just happens to be packetized messages over the USB it does not run USB isochronous mode it runs normal USB transfer mode Bluetooth of course they’re very very popular and you’ll see that most of the connections that exist right now between Android and Arduino ‘he’s almost always done with bluetooth that’s the one thing that everybody could guarantee was always going to be there yeah with this concept is there a provision for being able to pull at a bday over that’s USB interface let’s say that you’re in an environment where you don’t have Wi-Fi or cellular not that I’m aware of but it certainly wouldn’t be a hard extension so it’s a good idea i mean we see that in some cases with USB modems where you plug the USB modem in and it becomes a storage device temporarily so that you can then install things so it would have to be something like that but yeah it’s an interesting concept and I think it wouldn’t be too difficult to make it happen but I haven’t seen anybody who’s done it when they came out with the ATK version too they also added audio support the a2dp profile again works just like a serial port behaves like a serial port it just happens to be Wireless oh they also support bluetooth low energy which is the now now called Bluetooth smart because Bluetooth Low Energy wasn’t descriptive enough now it’s Bluetooth smart which is more descriptive losses for know your way out of date bluetooth for became bluetooth low energy because for wasn’t impressive enough and then they renamed it the Bluetooth smart because Bluetooth Low Energy told people too much marketing operations marketing upgrades exactly Wi-Fi if we happen to have the Wi-Fi plug over the shield it actually has its own little web server built into it so all the tcp/ip and UDP traffic is actually offloaded to the Wi-Fi interface so the little 8-bit microprocessor doesn’t really do anything except talk to the Wi-Fi interface across the I squared C interface so here’s an example of what that code would look like most of this is print lines we just set it your cereal I peed your IP address and then down here we’re doing a UDP begin packet and then we’re writing the UDP packet we’re saying to finish the packet and then the UDP packet goes out so it’s pretty easy to work with if you decided you wanted to use sockets and the same thing between Wi-Fi and ethernet interfaces they both have the same basic source so we find ourselves in a

situation where there are a lot of options that are available to us the biggest problem that we’re going to have to address here is we have to be able to partition our problem space so that we understand what parts have timing constraints and or electrical constraints and what parts are we trying to get and use android 4 if we’re trying to use android for the user interface we’re trying to use android to print pretty pictures we want to be able to have a service guy walk up to an industrial control device plug his smartphone into the industrial control device and have the industrial control then tell the smartphone hey I’m fine although this PWM seems to be acting up a little bit those kinds of diagnostic messages and things of that sort that’s easily done using Android as the front end and then one of these other microcontrollers have been talking about is the back end allowing those to do the real-time work and simply offload all of the real time constraints and the electrical constraints off of the Android handset but by adding this microcontroller now we have the decisions that we need to make are we going to use fermata are we going to use are we going to actually program the microcontroller we’re going to program the microcontroller we have to buy into that religion and fit you’re out exactly how that’s going to be done with the Ides etc but fortunately we have a bajillion and that’s a technical term connectivity options that are available to us whether it’s Bluetooth serial USB you know we have mesh networks we have you know the attitude of 15 dot for basically whatever connectivity we want to use is driven by our application and its communication constraints so if for those of you who are interested the SEF ternoon i have my android I mean excuse me building a linux-based robot which uses a lot of this stuff and then tonight at the at the birds of a feather demo showcase I’ve got my little robot will actually try and drive it around if everything works ok my eye I want only the freshest code for you guys so I’m still working on the app any questions yeah we know that’s a tiny movie staying patient right pop if you use it this is great yeah you know actually turns out that atmel bought the arduino line from nordic semiconductor and so it was actually a couple of guys at nordic that had created it and it’s actually it’s it’s really been a tremendous success the arduino community was actually built specifically to enable non-engineers to do engineering kinds of things so will i have another presentation later in the week where we get into the details of arduino and how it interfaces to linux and we actually take a look at some of those characteristics and how you talk to the between the devices what the code looks like it’s an incredibly simple interface that’s one of the reasons why our dwie nose are such a popular processor type they’re dirt cheap and you can get them very very small means smaller than the size of your pinky for a surface mount part and it’s an 8-bit processor it’s got its limitations but if you need interrupt driven you need pwm you need digital i/os it does all that stuff and there’s a simple environment and ecosystem with lots of examples out there on the internet to help you and that’s really the key that’s why arduino continues to exist in light of boards like the Raspberry Pi and the BeagleBone black those boards are actually cheaper than some of the Arduino but the Arduino exists because of the ecosystem yeah the Senators kind of didn’t mention but it says anybody hmm you don’t have to know anything yes to get them in the browser yeah that’s it that’s another one the embed processors that’s mbed the embed processors there’s also a bunch of applications out on the Play Store for embed so that’s another that’s another complete almost completely separate track on microcontrollers so yeah that’s also out there mm-hmm now that one well actually I think they do have something similar to Fermata for embed yeah it’s more I believe me arduino zar not where they are because of their capabilities they’re there because they are easy to understand and there’s an ecosystem that supports it and the ecosystem kicks ass both in hardware and software yeah and

they were there first I mean you know this is kind of at the heart of the maker movement so a lot of your your 3d printers and things of that sort will use Arduino xin side of them to control the 3d printers all the stepper motors and everything yeah great me if I’m wrong first of them what gather so far is that this will not have one so in some B’s is made of photos those the captain they bet is coming you think that yeah all the command and control has moved over to the Android side with fermata so many of them oh c’mon control it’s a fail I mean how do you need help in solving the bank oh you’re not you’re only solving the connectivity problem because we can’t wire a PWM out of an Android handset we’re just getting past the electrical problem it does not solve the real time constraint problem that’s correct are you alone yes well that’s what we were talking about back up here where we actually program the the micro well should we program are used for Mata and the real time constraints if you have real time constraints you have to program the microcontroller no question about that okay that’s it thank you