MobBob - Mobile-Phone Bob

MobBob - Mobile-Phone Bob

Shared on https://letsmakerobots.com/robot/project/mobbob-mobile-phone-bob by Kevin Chan

2015-03-03
This project is an attempt to combine my recent (healthy) obsession with using phones and Bluetooth LE to control robots (with apps written with the Unity game engine), combined with my (yes, still healthy) obsession with little bipeds. (Hey, if legs were good enough for my father and his father before him, I don't see any reason why my "offspring" should get all high and mighty with their new fangled "wheels".)

So far I have just designed the 3D printed parts. I went for a Bob-like aesthetic, but designed the parts to work with the Arcbotics micro-sized servos with metal gears and the analog feedback wire. These servos also have a hole in the bottom for an axel to go in, so I've made my servo braces use that.

I've also designed a holder on the front for my Nexus 5, and built a bracket to hold a phone battery extender which I'm hoping to use as the power source for the robot itself. I placed the battery and the phone in a way that they balanced the robot out. So, he's not too front heavy (from the phone). Because I wanted to position them to balance, I just hot glued the parts into the positions I wanted. (At some point, now that I know where I want them, I might redesign the parts so that they don't need hot glue.)

I have NOT connected the microcontroller, and I have NOT checked that the servos will be able to walk with the phone... So this might all end in failure (and tears... lots of tears).

However, as my philosophy is hope for the best, but plan for the worst... I designed the top part with the phone holder and the battery rack to be removeable. So, if the weight doesn't work out, I'll design a new part to fit there so that the robot will only have a microcontroller and not have the phone.

I will try to get the microcontroller connected and some test programs happening next weekend... So this is all for now...

So, in the best case, I'll have a phone controlled Bob. I can use the phone screen/speakers to emote, and also play with OpenCV.
In the worst case, I've just made a cute robot-shaped phone holder for my desk.
 
So I started with the servo frames... Since I'm building everything around the legs.
Putting it together to check the fit

Then I worked on the base plate that holds the 2 legs together and also the 2 feet. Note that I made a brace plate that goes on top of the servos. It's designed to hold the servos down, and also I plan to use it to mount the phone holder, battery holder, etc. (Because this part is removable, I can remove this if the whole phone idea doesn't work out...)
Next, I made the phone holder... It balanced better than I expected.

Next I made the battery rack and fitted both it and the phone holder to my servo-brace piece. I put the phone and the battery in to check the positions where it would all balance. Then I hot glued the pieces into place.
And then I put in the phone and battery!
Next thing is to wire it up and see if I made a robot or a fancy phone stand. Thank you for reading!
 
2015-03-04
My current STL files are now uploaded for anyone who wants to have a look. This is a work in progress... so some parts will need more work/tweaking!
 
2015-03-07
I've added a video of the test of getting MobBob to walk! I think it's all going to work. :D
So, based on this test, I think the servos are able to move the weight of MobBob including the phone and battery. Also, I think the phone battery is able to supply enough current to power him.
In the video, I'm just running a very basic Arduino program to create a rough walking gait. There is no communication with the phone yet.
With more confidence that this will all work, I'll keep working on the Arduino code, and then work on the phone app after that's done.
Having lots of fun with this project! :D
2015-03-08
New video demonstrating smooth servo animations and serial commands! (The Arduino code is pretty much done!)
The Arduino sketch includes my servo animation code that can play back servo animation keyframes pretty smoothly on MobBob. The code will automatically work out tween servo positions between the provided keyframes. You can see that his movement is much better than my first test video with the rough walking gait.
I've also integrated my serial command parser code (this is the 3rd project I've used it). So that MobBob will now receive and execute commands received over serial (both from direct USB connection and via Bluno's Bluetooth LE).
I've tried to make the code as configurable, robust and easy to use as possible! (I also have a tendency to write essays of documentation... So you can ready more about the code in the comments.)
This code can be used with any Bob-style robot and will allow you to control the robot using serial commands. It should also be pretty easy to create new animations using the keyframe arrays.
The next step is to make a phone app that will control MobBob using serial commands.
Here's the code! (It is also attached as a file to this post.)
[Code removed from here and put on GitHub ]
 
2015-03-09
Sneaky update. I tweaked a few things in the code below in the 2015-04-08 entry. :)
 
2015-03-09b
I've "assembled" MobBob. I've now attached the microcontroller onto him so that he doesn't have to drag it around with him.
The attachment could certainly be tidier... (I'm a software guy, and that's the excuse I'm sticking to.) However, it's tidy enough that it doesn't effect his movements, and gives me easy access to both the battery recharge port and the Bluno serial port.
With MobBob assembled, I can now focus on making some fun phone apps.
(I may still trim/solder the wires and tidy him up at some point. I ordered a Bluno Beetle this weekend for that purpose. It looks tiny! When that arrives, I might redo this part and move away from my plume-of-wires look.)
Here are some photos!
   
2015-03-09c
Sorry for updating so much recently. I promise this is the final update until next weekend.
The first demo app is done. It doesn't do anything smart or autonomous, but demonstrates how the phone will communicate with MobBob. :D A video has been added to this post.
I've put the app onto DropBox in case anyone wants to try it.
The app was written in C# using the Unity game engine. I've written some util classes to make it easy for the app to connect to and control MobBob. So, the next thing is to work on some fun apps for MobBob! :D
  
2015-03-10
So I lied that there would be no more updates until next weekend... However, I agree with byerley that the code was causing too much scrolling. So I have removed it from the post below and have put it on GitHub here. (The most recent version is also an attachment in this post.)
https://github.com/cevinius/MobBob.git
Big thanks to everyone for your kind comments, suggestions, and offers to collaborate on and to further this project! LMR-ians rule! :D
 
2015-03-14
I will be making are a real update (new app!) this weekend, but for now, I just added the license since all the smart kids seem to be doing it.
Also, I fixed all the post dates so that I'm living in the present again. (I think bdk pointed this out to me days ago, but I was too dumb to understand. :P)
 
2015-03-16
Two updates!
First, jinx did some tweaks on the parts. There is a cooler version of the leg part from jinx that has wider holes for the servo screws (and other tweaks). There is a also a split version of the phone holder that should make it easier to print. (I think Craig may have printed versions of these parts??) These have been uploaded and should be viewable in the STL viewer.
Second, I did some work on the app. The app now includes functionality for an animated face, and includes a basic UI system for enabling new control panels to be added easily. The app is written using the Unity game engine.
The Face can play back predefined animations, and can also be programmatically controlled. It is setup like a game character, and is quite easy to use. Animations can be added and setup easily using Unity's Mecanim animation system.
The UI system is fairly basic. I've added a pop-up menu that can be shown/hidden using a vertical swipe gesture on the touch screen. Within the pop-up menu, there can be any number of pages. You can switch between them using the "Prev" and "Next" buttons at the bottom. Currently, I have separate pages for walking and for playing back pre-defined animations. I'm planning to add custom pages for different MobBob functions. E.g. For computer vision, there may be a page that has a camera view, etc.
The Face continues to animate and work even when the pop-up menu is visible. So MobBob can continue to emote even when the menu is showing. I was originally thinking that I'd want to hide the menu as much as possible so that MobBob can use the full screen to emote... However, now that I've been looking at it for a while, I kind of like how it looks when the menu is showing...
So, this updated app doesn't do whole lot more than the last app! However, I think it is now a much better framework to build on, and I should be able to add more functionality to this as MobBob develops. :D
2015-03-25
I was hoping to add more features and to tidy a few more things up before posting this, but I didn't get a chance this past weekend. Here is the current version of the app which includes MobBob's face
As I did it in Unity, the app is cross platform. I've also tested an iOS version (running on my iPad) and it works well. At some point, I'll borrow an iPhone and do a video of MobBob working with an iPhone. :)
I'm looking forward to getting some time to work on this next weekend. Thanks for reading.
  
2015-03-28
Just a quick note about the app!! I noticed that more people are downloading the app and quite a few people are watching the video, so I wanted to provide some more info for anyone who wants to try the app.
(markcra told me that MobBob is featured as a "Design of the Week" in this week's MyMiniFactory newsletter. So, if you're here because of that, a HUGE thank you for your interest in my project!)
The current app is setup to work with DFRobot Bluno boards. It should work with any of them (there are a bunch of different ones...). The app connects using the Bluno's Bluetooth LE GATT profile, so I'm searching for and connecting to the Service/Characteristic offered by the Bluno board.
The setup is an updated version of what I did on Hackneyed, and I wrote more info about the Bluetooth LE connection

I wanted to note this in case anyone was trying to use the app and wondering how to make it connect.
I will update the app at some point to support regular Bluetooth. I went with Bluetooth LE as I like that you don't need to formally pair the devices. Right now any tablet/phone with the app can connect to MobBob, and one phone can connect to any MobBob. So, it keeps setup and connections easy. That being said, the app is super naive at the moment and assumes you only have 1 Bluno within range to connect to.
So I will need to add a new UI for selecting which device you connect to. When I'm doing that, I'll also look at adding support for regular paired Bluetooth communication as well, since I think a lot of people may want to do that.
Apologies for not being clearer on this part. And I'm hoping to squeeze in some time to update the app this weekend, so hopefully, I can show some new features the next time I post!


2015-07-19
Not a proper update this time, but there is one coming. I have this coming week off work, and I'm planning to spend a lot my holiday time doing more work on this project.
However, I was recently in touch with Roxanna who said that the app wasn't working for her. I had also noticed that the app does not "connect" on my Samsung S6 phone (which I hate and have given up on - stock Android all the way! :P). I used the S6 as a test device, and found a potential cause of the problem! I've updated the app
(For anyone interested in what the problem was...I found that if I scan for a specific Bluetooth LE service (in my case, just the Bluno's service), it returns nothing on the Samsung. (On my Nexus 5/6/7, however, scanning for just the Bluno's service will return the ID for the Bluno.)
If I scan for any services, then the Samsung (like the Nexus 5/6/7) will return all the services it sees, including the Bluno one.
So, I've changed the code to scan for all services, check each one to see if it's Bluno and then connect/subscribe to the Bluno services when it finds them.
With this change it's now working on the Samsung S6. I've also passed this to Rox to see if it helps her connect.)


2015-07-23
The app has been updated!! Yes, I know I'm a slacker!
See the top video for a demo of the latest app.
I've now integrated OpenCV into the app. I'm currently using a face detector, but have plans to use more of the features. (E.g. Blobdetection and have him try to follow a ball.)
For the camera feature, I currently have a checkbox on the front page to flip the image if it's upside down. I noticed that it's different between my Nexus 5 and Nexus 6. (I'll figure out a way to do this automatically at some point.)
Also, I've added an option on the front menu to "Skip Connection". This enables users to get past the front page without actually connecting to a Bluno. This means you can try the app out even if you don't have a MobBob body to use it with.
I'm excited now that OpenCV is integrated! I can start playing with more of the algorithms and adding more fun features. :D
Even though the current vision interactions are quite basic, he already seems a lot more alive now that he can see me!

Thanks for reading!
Kevin


2015-07-28
MobBob now has speech recognition integrated as well! I just posted a video with a demo of it (see above).
I've only programmed a few commands/key-words that he'll recognise, and he'll just repeat what you say if it's something he doesn't understand.
Now that thi feature is in, I'm planning to add more speech/conversation functionality. (And I need to add more facial animations so that he can emote better too.)
MobBob can now walk, talk, see, hear! I'm having so much fun working on this little guy! :D
 
2015-08-10
This release includes the voice recognition functionality seen in the previous update, and also includes the new features seen in this update.
So, onto the details of this update...
I've added a new feature to MobBob that enables him to track and walk towards colored balls. You can see a demonstration of this feature in the latest video (at the top).
Besides adding this feature, I also spent a lot of time tidying up the Bluetooth connection/disconnection stuff. The app should now correctly handle when you leave the app by pressing the Home button, and should also resume correctly. (If resuming, it will bounce you back to the Connection page if it no longer has a connection.) I've tested this a bit, and it seems to be working well, but if I was at work, I would definitely still get QA to put it through its paces... :P
I've also added some code to the app to take advantage of the "Response" messages that my Arduino code sends back after it completes playing each movement animation. I had coded this feature into the Arduino code with the intention that it would allow the app to know when MobBob's body was mid-animation and when he was idle and ready for a new command. I hadn't used this up until now and just used delay timers to wait for moves to finish. I've now added code that will track the response codes and keep track of whether MobBob is idle or currently moving. Having this makes it much easier to program behaviours, as I can wait for MobBob to finish its current move before issuing the next move. Again, I've tested this a bit and it seems OK, but I need to test this more. (If it goes wrong, MobBob can potentially think he's still making a move when he's finished... and the app won't send new commands.)
Also, when I was first learning to use the Bluno, I wrote a little utilility app for sending text to the Bluno and printing the responses that I get back. When I was testing the connection and response code stuff, I found it useful... so instead of keeping it as a separate app, I've now integrated this into MobBob's app. It can be accessed from the front page. I will more than likely be adding/tweaking the debug features here as I add more to MobBob's app.
Tidying up the connection and response code stuff is not particularly glamorous, but I think this makes makes the codebase more solid and will make it easier for me to add more functionality.
Thanks for reading.
Kevin 
 

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
 

2015-08-11

I've put up a new version of the app with some minor tweaks (no new functionality) (This update was done primarily for Roxanna's MobBob!)

As I mentioned in the last update the current app is now making use of the "Response Messages" that the Arduino code sends back to tell the app when it has completed an animation.

I had a message from Rox that the new app would stop accepting new commands after a while. I don't know for sure, but I'm guessing that it is due to unreliable Bluetooth causing MobBob to miss Response Messages. This would lead to a "stuck" state where he just sits there waiting for a Response code (that got lost somewhere). I'm suspecting this since Rox is using a Samsung phone, and I've personally had a LOT of bluetooth issues with my Samsung S5. (Not just with my robots, but also with my bluetooth headset and with Sphero/Ollie.)

So, I've made a couple of changes to the app to help address this...

On the front page, there is now a new option "Wait for Responses".

If this option is ticked, the app will try to use the new functionality to wait for responses. To further improve this functionality, I added a time-out, so that if it doesn't receive a response in 10 seconds, it will assume some sort of error and stop waiting for a response to the last command. I've also went through the code and did a few changes that should make the response handling code more robust overall.

If this option is NOT ticked, the app won't wait for the response messages.

This means that there is a potential to issue commands before the previous one has completed. The Arduino code was designed to handle this situation, so there should be no major issues, but it will be less tidy (e.g. Voice may say the next command, but he's still finishing the last command.)

For the Colored Ball tracking feature, I had MobBob decide on his next step when his previous step finished. This relied on the response notifications. So, for this case, I added some extra logic so that if the app is NOT using reponse messages, MobBob will use a timer instead to determine when he should take his next step. Since the timer is approximate... he will pause for a little bit between steps. So, again, this should work but it is less tidy than when the app can rely on response codes.

So, please tick the "Wait for Responses" check box if it works for you. The app will work better. However if you have problems with receiving response codes, you can untick this as a workaround.

If you want to check whether you are receiving response codes, you can do that with the Connection Utility feature I mentioned in the last update! That will show you when the app receives response codes.

Have fun!!

Kevin