Video Making a Birdhouse is Like 'Hello World' for a Versatile Factory Robot (2 Videos) 24
Benjamin Cohen is a Ph.D candidate at the University of Pennsylvania working under adviser Maxim Likhachev with a real-world, cheap way to make robots to accomplish a multi-step project with minimal human intervention, which he calls "autonomous robotic assembly." Project Birdhouse -- part of his Ph.D. work, along with teammates Mike Phillips and Ellis Ranter -- is Cohen's effort to create a sort of "Hello, World" for robots. With a combination of a research-platform robot base, off-the-shelf parts, like a nail gun (read: "One not built for robot use"), and software to squeeze greater accuracy out of the system as a whole, he and his colleagues have come up with a robot that can grab a selection of parts, align them properly, and assemble them with nails into a functional birdhouse. QR codes let the robot give the robot a sort of recipe to follow, and the system is smart enough to squawk if it doesn't have the right parts to complete the task. (Check out more video with the robot in action, and a great many photos, sketches, and diagrams illustrating the project's evolution.)
NOTE: We split today's video in half, with both halves running right here, today. This way, if you watch the first video and and want to learn more, you can move on to the second one. And the transcript not only covers both videos, but has "bonus" material that isn't in either one.
Timothy Lord for Slashdot : Ben, what can you tell us about Project birdhouse? Start out with... What is Project birdhouse?
Ben: lProject birdhouse is a demonstration in flexible automation. So right now in manufacturing in the world, if you want to manufacture a new product, you have to reconfigure an entire factory, such that you have all these highly specialized machines, and it's assembly aligned, pushing materials down the line very precisely, doing very precise motions and each one of those machines is extremely specialized and also extremely expensive. But for that expensive price tag, you get incredible speed and great performance and robustness. So that means there is a lot of overhead to create a new product. So if you design a product and want to have it made, you are going to have to order a really large volume of that product for it to be worth it to you. You got to order a really large volume of that product for it to be worth it to you to make it, because the price tag to actually reconfigure the factory will be way too high otherwise. So for example, I noticed that in the automobile industry, every car manufacturer uses the same body for five to seven years. And maybe in 1995 the Pontiac Bonneville looks one way, and then 1996 to 2000 looks another way. And the reason why we use the same body is not because designing it is a super difficult thing, but actually it takes a manufacturer about a year to move around old machinery and reconfigure old machinery in that factory to make that new body, so they just use that for five years.
Slashdot: What are some examples? I mean there is tooling, and there are extruded parts, what are some of the expensive techniques or manufacturing aspects?
Ben: Sure. So a good example is let’s say inserting a rivet into a car door. So right now in factories we actually do have robotic arms that are actually doing the motion to insert the rivet to the same exact spot every time. So now you could think of that, it looks like a robot, it’s a robot arm, but it’s like when you go to Disneyland and you get there and this robotic Mickey Mouse is waving to you. Maybe it has a sensor which could see something is there, so it notices and starts waving. Or maybe it hears you, it doesn’t know what you are saying, it just hears the sounds, so it starts waving. So let’s say this arm, it does the same exact motion over and over again and maybe it’s a little smart. So it waves till something is there. It doesn’t know if it’s a car door, it doesn’t know if it’s a ball, it doesn’t know what it is, but it knows something is there so it should do its motion. That’s how manufacturing is done now. So we actually do have robotic looking things, they could be robots if we gave them sensors, and we gave them software to be able to see some input and then act based on that input.
And that brings us to where we want manufacturing to go from now on. We want to move towards flexible automation. And by that I mean the whole world wants to have smarter atomic arms so that their robots, they have sensors, they have brains, and with that they can do more tasks, so each specific machine is more flexible now. You don’t actually have to move that car door to the same exact space every time, because if that car door is brought to a location that’s slightly over, the arm can actually see it and adapt to it. So now we can move away from this really old manufacturing model in which we reconfigure factory such that all the materials are brought to the machines. And now let’s start bringing machines to the materials, like before we had machines at all, we had people walking around a factory doing different tasks at different stations. Let’s go back to that. And then if you could just add the workstations within a factory, and you can have robots doing tasks, that are smart, that can move around things and not collide, that can more their arms to any arbitrary position safely and to do a complicated dynamic task.
So this is where we would like to go. If we move from the fixed automation model to the flexible automation model, then we can have cheaper hardware, now these machines are not as specialized, they could actually do a few tasks in the same day, okay, and work on a few different products. They also should be cheaper because they are smarter, they will have brains now, and they will be able to compensate, they will look at what they are doing and use feedback from their sensors and the lasers or the cameras to adjust to do things more properly. So they don’t actually have this open loop motion anymore, they can actually fix what they are doing and react accordingly.
So now we don’t actually have that huge overhead cost in manufacturing anymore if we were to have this. So let’s say I am an Etsy company owner and I want to make 5000 of something. Then now I can put an order into this factory and say, I want to make 5000 of these things, and as a factory owner I will say, “It should take my engineers two full eight-hour days to get our machines to be able to make that. Sure. I will make that 5000 item order and then ship it out to you,” and that’s what we want to be. We want to be able to have factories be flexible and not have such large overhead costs as they have right now.
Slashdot: With all that as background, you are starting out with something that’s much smaller, obviously, than the manufacture of a car. You selected a birdhouse as sort of an exemplar of what can be done with this robotic automated assembly. What’s the basis for that?
Ben: So the reason why we chose a birdhouse is because – a few reasons. One is because in your shop class, in your home ec class in sixth or seventh grade across America, a birdhouse is the thing that every kid makes as their – the first thing that they assemble, either that or sometimes it's just tool box. But we think it’s mostly a birdhouse for the last 60 years. So we thought if that’s going to be the first thing that a kid should build, it makes perfect sense that a birdhouse should be the first thing that a single robot builds.
Slashdot: It’s kind of a hello world for robots.
Ben: Exactly. It’s a hello world for people as well. So we also chose birdhouses for a second reason. Like I said, we want to show that our system is flexible. We are not just producing one single birdhouse over and over again. Birdhouses can be designed in many different ways, so we want to choose something that is flexible in nature. So, we chose birdhouses because they have all these different type of pieces, you could put them together in different ways, you could have a two-room, a three-room birdhouse or a double-decker or whatever you can think of, using these pieces our robots should be able to build and they could be very different from the previous birdhouses that are built. But that just shows the ability of our software.
Slashdot: You mentioned safety a few minutes ago. And yet you also talk about robots with nail guns and there seems to be something of a risk there. What kind of safety concerns are you thinking of either long-term, and specifically with this building a birdhouse project? Do you try to sense people around? Do you turn off power tools if there's a warm human hand in the way? How do you do that?
Ben: Well, the nail gun is wired to the robot and it can't actually shoot unless it's pushing down on something. Actually right now, if it's pushing down on a human, it can actually shoot at them, but the nail gun is extremely weak. I shouldn’t tell this today, but I had the robot shooting the nail gun on me from five feet away and it bounced of off me. The nail gun that we choose is weak in general because actually our robot is weak. The robot that we use is PR2, it’s a research platform, it's meant for mobile manipulation tasks that you'd find in your household like picking up items off of the floor and putting them back where they belong or cleaning up a table, putting dishes from the dishwasher into the cabinet.
It has a maximum payload of like four pounds. So, that means when its arm is fully extended, it can really only pick up four pounds and our nail gun was 3.5 pounds and it actually can really pick it up. So, the payload is actually smaller than advertised and this caused a problem. We had to find a nail gun within the max payload of the robot and that the robot can actually press and shoot, and so we had to choose a weak nail gun. And so that's why, so safety was less of a concern because the robot was using the weak nail gun and it had to be pushing on you. Even a tired grad student knows to stay away from that. So, that wasn’t a concern.
Slashdot: Can you talk about the rest of the hardware platform? What do you need besides your base platform and a nail gun attached, what else do you have in terms of sensors right now as far as your operating robot?
Ben: Sure. So first I should say that we have this PR2 and the PR2 has pinchers, twofinger pinchers and you can't go to Home Depot or Lowe’s and buy tools for pinchers.
Slashdot: Okay.
Ben: So, we started this project and we are like, we need tools, but we can't go to Home Depot and buy them because everything is made to be ergonomically comfortable for human beings, with five-fingered hands. And so we actually had to team up with Professor Kern Maass from the Industrial Design Department at Appalachian State University to actually create or modify human tools in such a way that they could be used by the robot, and that takes two things. One, they have to be manufactured so they could be held by the robot and two, manufactured so that they can be controlled by the robot.
And so in the case of the nail gun, we took the guts out of a human nail gun, put into enclosure that they designed for us, that is holdable, graspable by the robot’s pincher. And then same thing with the vacuum gripper. It’s a suction cup and it can pick up pieces, it’s very strong, it’s a very good tool and same thing with this turntable that behaves as the third hand of the robot. So this turntable in front of the robot, like all the other tools, could be controlled by wire. Okay, so the robot, it shoots by wire, it’s turned on by wire, it’s rotated by wire. The robot could send the right signals to all three of its tools so it’ll make it do things, instead of pushing a button or turning a dial.
So that turntable behaves as the third arm. And by that I mean, let's say left hand, we are holding up a piece to attach. Right hand, we're holding the nail gun to actually attach that piece to the base of the birdhouse and then be able to rotate the birdhouse into the proper position, so that the piece could be attached correctly. And so all three of those tools make up the tool center of the robot and now in terms of actual sensors that the robot can use, well we have a connect on the robot’s head. The connect is RGB-D camera, not only can it sense color, it can also sense depth and so with the camera, we can sense a bunch of things in the environment.
And the primary thing is we could sense the fiducial markers, the bar-code looking things that you see taped to every piece that the robot will be working with and also you can see that those fiducial markers are actually attached to the worktable in front of the robot and the parts table to the left of the robot, because the dream that we have in mind is that the robot wakes up at 6AM and goes to some workstation from some resting place where it’s charging and it doesn’t have to be fixed at that location. And so the robot can sense where all of the work surfaces that it's going to be using are. So it uses those fiducial markers, those bar-code looking things to do that.
So, taking that we could actually sense where all the furniture is that the robot is expecting. And the robot can adjust itself to be able to adapt, to move furniture and the robot can also see all the pieces with the connect. In terms of additional sensors, we also have positioned sensors and all the motors in the robot arms. So when you expect to go somewhere, if we don’t get there, we know we didn't get there and so that helps out when we need to align two pieces together and our visual sensor, the connect, isn't accurate enough, isn't calibrated well enough such that it could tell us, oh, you are lining up two pieces correctly. It will actually have to feel for lining up the pieces correctly. So in addition to the connect, the position sensors in the motors, we also have the push button on the nailer. So that’s some feedback that we can use to tell if we're actually pushing on the surface or not.
Slashdot: Will the robot refuse to make a birdhouse if it doesn't have all the parts it needs to start?
Ben: Yes. The robot starts out by looking at the inventory table. And so – well at the start the robot is given an inventory what pieces to expect, a CAD model, what the birdhouse should look like; and a set of instructions which it's not really a detail set, it’s just an order of assembly: Do this piece first, do the base piece and do the left wall, and do the right wall. So it will look at the inventory, if it doesn't see the pieces there and if it doesn't see that it can pick up all the pieces the way they are currently oriented, it won't start the task.
Slashdot: The software you’re using to control this, are you publishing the software, are you making a great deal of stuff that's available for people to watch your progress? Can people also look at your software and adapt it themselves?
Ben: So in our lab at the storage-based planning lab at CMU, all of our software is released OpenSource and actually software that we built upon in this project is also there. We definitely plan on releasing everything Open Source once we submit a paper to be published, but it is not yet Open Source. But I can tell you, it will be; that’s the way we do things and, yeah, we have some good building blocks we think that other people can use, we also have the PR2, it won't be something similar.
Slashdot: Speaking of that and of sharing information, you mentioned how difficult it was to find tools that were graspable and manipulable or rather graspable and useable I should say by the robot hands. It seems like there are a lot of labs, a lot of people experimenting with robots. Is there not enough infrastructure in that, do you share the designs that you made to adapt the tools with your robot, is there sort of an infrastructure out there for sharing that kind of information?
Ben: I don't think there's an infrastructure out there for sharing robot enabled anything. So Robot enabled frying pan, something like that. I think this is going to have to be something that we put together. As of now, if anybody wants to 3D print any of our tools on their own they’re more than welcome to, we can send them our designs.
Slashdot: Can you put them on repositories like ThingiVerse as well?
Ben: Yeah, I'm sorry, what is that?
Slashdot: ThingiVerse is a place where lot of 3D objects are sort of archived and there are some others as well.
Ben: Yeah, no, we definitely could do that. We plan on hosting them on our lab website, where we can also put them on an online repository where people know to go there for it.
Slashdot: One thing that’s interesting there are a lot of purpose adapted tools for people who have use of only one hand or limited mobility in general. And a lot of that stuff – it looked adaptable to robots because it’ll be simplified motions or it will be a smaller range of motion to activate whatever its final output is, everything from pepper grinders to special things to hold telephones. A lot of that stuff seems utterly adaptable in a context like this.
Ben: I think that's a good place to look, also you know a few years ago for a project I actually designed and built a robot enabled kitchen. I had to make sure that all the handles and all the cabinets were ADA compliant handles. And they had a large gap between the handle and the cabinet wall, so that the robot can put its clunky gripper with poor accuracy in there and actually be able to open it.
Slashdot: A lot in common there I think between even just for instance clumsy people like me. But the things that robots need to grasp like a nice wide area. Ben, what's the next phase, do you intend to develop this further or is this going to remain sort of a cool final project in a PhD. You want to go into designing this sort of autonomy as a long term part of your life.
Ben: Yeah. I expect to graduate this summer, I'm up for [parole] in July. So I would love to continue along this road into factory automation when I graduate. As far as, Project birdhouse, I'm sure that my professor [Max Lekachev] will continue the project with other students in the lab. And the next step would be to actually remove that list of instructions that I said. So, now, the robot will just need an inventory and it will just need a CAD model and then get rid of that list of instructions, let the robot figure out the order of assembly itself, so our lab is a motion finding lab, we do a lot of hierarchical planning too. Let’s have the robot figure out or given what it should look like at the end and I'm sure that’s where our lab will go with it.
Slashdot: I want to ask you one more thing about the software you’re using to control it. For instance with 3D printing there are some very standard file formats that people know they can export on to a USB drive, or send down the wire via USB connection to their machine. And those are pretty well documented formats. It seems like you're doing something different enough and more complicated enough. Have you considered regularizing the type of instructions you’re sending to the robot. Can somebody build a compatible birdhouse maker and follow the same recipe?
Ben: Are you saying like given them having the same robot or no?
Slashdot: Not even, I’m asking could somebody take your recipe for making a birdhouse and adapt it to other hardware?
Ben: I think they can adapt it to other robots or other platforms with two arms and a connect, I think the requirement of the PR2, it’s not a real requirement. We just have a PR2, but actually joining the construction, we don't move it around at all, we don't raise and lower its spine, the arms are already fixed tight, if you have a torso, if you have two arms fixed to a table and you have a set of tools, you should be able to use our code with some modifications obviously and build the birdhouse.
Slashdot: The actual list of instructions that results in a birdhouse, is it stored in an XML file, what is the active recipe form if that's a sensible question?
Ben: So it’s actually a YML file, and the YML file, it says, the order of assembly like I said and it also has a description of where the nails go. And each piece is given a list of nails and then the robot has to figure out the order of the nails and so which nail to put in first and then also, imagine you’re using a screw gun, drill, you could drill at any roll around where that nail has to go in. So to make sure that the robot doesn’t collide its hand with each other, it’s a big problem because the robot has really clunky arms and we’re working in this really tight workspace in front of the birdhouse. So we bring over the piece with one hand, and first they have to think about which nail to put in first, so remember this has to be rolled such that, it won’t collide with the nailer and the nail location that we chose. And then the roll angle of the nailer also has to be chosen. So the list of instructions it has is written in YML and it says which piece goes first, which pieces goes next, and also has a set of nails for each piece. So it’s order of assembly and a set of nails and then the robot uses that and makes sure that it’s only going to do that in set motions, so it doesn’t hurt itself.
Slashdot: All right. Ben, is there anything else that you think we should talk about? It sounds like you gave me a good overview of the software itself. I’m glad I asked you about Open Source because that's one of our favorite things to learn about, especially with hardware because that gets ever more interesting as actuators, platforms, motors, all become more widely available and there’s lot more tweaking going on, and more people have access to hacker spaces, so that’s always really cool to see. Is there anything else that we think we should have touched on, but didn’t.
Ben: Yeah, I guess, I just want to say something about the hardest part of this project.
Slashdot: Sure, that's actually a great thing to know about. So what is the hardest part of building a robot like this?
Ben: So, the hardest part of this project was I mentioned before that we're using a research robot, this research platform that for the last five years you can look on YouTube and see amazing video, people got it to pick up things, clean up after themselves, get themselves a beer, and clean off dishes off the table, and all of those tasks have something in common, and that's that they are not extremely precise tasks. They're very forgiving. If the robot's pincher goes over to pick up a soda can, it has a lot of area, a lot of a loud error built into the task.
And all of my work until now has been in manipulation in pick and place, and this is the first time, when we ran into this project we knew what we were trying to do. We were trying to use a research platform which has up to 3 centimeters of position error in its motion vector. So that means that when my eyes see something and I say, go to it, my hand can be anywhere within 3 centimeters of that actual thing. And we are trying to build a birdhouse where we have a 1 centimeter nailing surface at the bottom of the birdhouse, at the bottom of each wall or on each side, along every edge we have 1 centimeter in which we can joint two pieces of wood together. So to actually hold up each piece to the base or to another wall and actually line it up correctly was a huge challenge, and also actually getting the tip of a nail gun to the edge of a piece that has to be nailed in is also a huge challenge, because we're using a research robot when we should be using an industrial robot that has one millimeter or
Slashdot: So Ben, what's the biggest challenge that you found in actually making this robot work?
Ben: The biggest challenge that we found in the project is that we're working on manufacturing and it is a very precise task. So Mike, Ellis and I spent about 6 weeks figuring out a way to account for this 3 centimeter error, and still be able to line up two pieces of wood accurately, so with a vacuum gripper we can actually bring in a piece and hold it at the proper 90 degree angle, and also bring in the nail gun, so that we don't miss when we shoot a nail, we actually hit the bottom edge or the vertical edge such that it goes into the edge and pierces through one piece into the other piece, so when we have 1 centimeter slice around every edge of a piece to insert a nail or to actually line up the two pieces, we have to get down our error from 3 centimeters or more sometimes to 1 centimeter to be able to nail into this 1 centimeter really 9 millimeter zone, where if you insert a nail there, you will be able to attach the two pieces.
So, actually Mike, Philips and Ellis, my teammates on this project, they focused in the last few weeks while I was doing other things to create this alignment behavior, now imagine you close your eyes and you know approximately where something is. I call it the Helen Keller approach. And let's say I know that the base of the birdhouse is somewhere in front of me in this – I could have 3 centimeter error then I take a piece that I want to bring over to it, start out further down from where I think it is, start out further back from where I think it is, and take baby steps and keep moving towards it, and once you hit something you try to go past it, but the robot is like oh, I can’t go past it, this must be the edge and now it knows exactly where the edge is, and that it can move back, move up and move forward, and keep repeating that motion until the edge actually is lined up with the base of birdhouse.
Same thing with the nail gun. We know approximately where it is, and we have a push button at the tip of our nail gun. So it's actually easier on the nailer side, we could actually take the nail gun, start out two rows, start out too far back and keep taking baby steps until we – I’m sorry we’ll start out too high with the nail gun, keep taking baby steps until we hit a surface and then now just lower and lower and lower until the button doesn’t push any more. And now we know where the edge is, I mean we could raise it back up and hit the edge.
So these alignment behaviors compensating for poor sensing and an uncalibrated robot, that were the biggest challenge of this project. And we think that if we had industrial strength manipulators with 2 millimeter accuracy, actually the assembly time would be reduced from 20 minutes to 4 minutes, and that was the challenge I guess, using a research platform to do a precise task.
Now comes the hard part (Score:2)
Re: (Score:2)
I thought the spice rack was the first carpentry project to make or break a would-be carpenter? Honestly, it doesn't take a robot to make a birdhouse. My guess is someone could do it with all mechanical means with little problem. You're talking about six to eight pieces of wood, only some of which need to be cut other than to length, and one hole.
I figure all those cheap birdhouses in places like Christmas Tree Shops are made by machines in China. But that isn't the point here -- this robot is supposed to know how to do it, not just repeat predefined steps in a highly controlled environment. Robots already do that on a much more complex level - like assembling parts of cars.
Robots with nail guns vs. sharks with lasers (Score:1)
Is it too late to put this thing together as a 2015 summer blockbuster?
Re: (Score:2)
They gotta' hire Adam Corolla to be the consultant who advises against arming the robots.
Only two videos? (Score:2)
NOTE: We split today's video in half, with both halves running right here, today. This way, if you watch the first video and and want to learn more, you can move on to the second one.
Why not split the video into four? Or eight? In fact, why not split it up into individual frames, each one 1/30th or so of a second long and autoplay them one after the other, then I can exercise my free will and stop watching any time I want to because I'm not a gibbering idiot?
They'll take yer jurb! (Score:2)
"When the robots come, they will replace all workers. And humans will be free to pursue other interests, like art and culture."
"How will the humans eat and pay rent?"
"The robots will provide all their needs"
"But how do they pay the robots for this?"
"The robots will also provide them with money."
"Are you sure you've thought this through?"
"No, not really. But look, isn't this robot COOL?"
Re: (Score:2)
To paraphrase, how will people pay for things?
Why does anything need to be paid for?
Re: (Score:2)
Why does anything need to be paid for?
Because communism, while sounding great, doesn't work in the real world. It's great to think that without money, we would all just share and share alike. But unless you find a way to fundamentally alter human nature at the DNA level, that system just doesn't work for long in the real world.
"Hey come fix this broken robot."
"No, I don't feel like it. I'm going to sleep in and play videogames today."
"But you're the only one who can fix it!"
"Are you going to force me to or compensate me somehow for my effort?"
"
I wonder (Score:1)
If someday instead of looking for a "Made in America" sticker I'll be looking for a "Made By A Human" sticker
Re: (Score:2)
Ah, /., I knew you when... (Score:2)
Evolution (Score:1)
Great research that builds on the author's previous work:
Autonomous Poop Scooping https://www.youtube.com/watch?v=NkcaivxHZnY
Worst Birdhouse Ever (Score:1)
Arming robots with nail guns is the first step (Score:2)
I look forward to our birdhouse-building robotic overlords
A story about robots making birdhouses (Score:1)
You know what would have made this story really really cool? Hmmm, how about the two main videos actually showing a robot making a birdhouse. After watching the videos I am confused as to which guy was the robot; the one with the dark hair or the blond hair.