Cornell Researchers Taught a Robot To Take Airbnb Photos (engadget.com) 8
A team of researchers from Cornell University used a computational aesthetic system to teach an AI robot "to not only determine the most pleasing picture in a given dataset, but capture new, original -- and most importantly, good -- shots on its own," writes Engadget's A. Tarantola. The project is called AutoPhoto and was presented last fall at the International Conference on Intelligent Robots and Systems. From the report: This robo-photographer consists of three parts: the image evaluation algorithm, which evaluates a presented image and issues an aesthetic score; a Clearpath Jackal wheeled robot upon which the camera is affixed; and the AutoPhoto algorithm itself, which serves as a sort of firmware, translating the results from the image grading process into drive commands for the physical robot and effectively automating the optimized image capture process.
For its image evaluation algorithm, the Cornell team led by second year Masters student Hadi AlZayer, leveraged an existing learned aesthetic estimation model, which had been trained on a dataset of more than a million human-ranked photographs. AutoPhoto itself was virtually trained on dozens of 3D images of interior room scenes to spot the optimally composed angle before the team attached it to the Jackal. When let loose in a building on campus, as you can see in the video above, the robot starts off with a slew of bad takes, but as the AutoPhoto algorithm gains its bearings, its shot selection steadily improves until the images rival those of local Zillow listings. On average it took about a dozen iterations to optimize each shot and the whole process takes just a few minutes to complete.
"You can essentially take incremental improvements to the current commands," AlZayer told Engadget. "You can do it one step at a time, meaning you can formulate it as a reinforcement learning problem." This way, the algorithm doesn't have to conform to traditional heuristics like the rule of thirds because it already knows what people will like as it was taught to match the look and feel of the shots it takes with the highest-ranked pictures from its training data, AlZayer explained. "The most challenging part was the fact there was no existing baseline number we were trying to improve," AlZayer noted to the Cornell Press. "We had to define the entire process and the problem." Looking ahead, AlZayer hopes to adapt the AutoPhoto system for outdoor use, potentially swapping out the terrestrial Jackal for a UAV. "Simulating high quality realistic outdoor scenes is very hard," AlZayer said, "just because it's harder to perform reconstruction of a controlled scene." To get around that issue, he and his team are currently investigating whether the AutoPhoto model can be trained on video or still images rather than 3D scenes.
For its image evaluation algorithm, the Cornell team led by second year Masters student Hadi AlZayer, leveraged an existing learned aesthetic estimation model, which had been trained on a dataset of more than a million human-ranked photographs. AutoPhoto itself was virtually trained on dozens of 3D images of interior room scenes to spot the optimally composed angle before the team attached it to the Jackal. When let loose in a building on campus, as you can see in the video above, the robot starts off with a slew of bad takes, but as the AutoPhoto algorithm gains its bearings, its shot selection steadily improves until the images rival those of local Zillow listings. On average it took about a dozen iterations to optimize each shot and the whole process takes just a few minutes to complete.
"You can essentially take incremental improvements to the current commands," AlZayer told Engadget. "You can do it one step at a time, meaning you can formulate it as a reinforcement learning problem." This way, the algorithm doesn't have to conform to traditional heuristics like the rule of thirds because it already knows what people will like as it was taught to match the look and feel of the shots it takes with the highest-ranked pictures from its training data, AlZayer explained. "The most challenging part was the fact there was no existing baseline number we were trying to improve," AlZayer noted to the Cornell Press. "We had to define the entire process and the problem." Looking ahead, AlZayer hopes to adapt the AutoPhoto system for outdoor use, potentially swapping out the terrestrial Jackal for a UAV. "Simulating high quality realistic outdoor scenes is very hard," AlZayer said, "just because it's harder to perform reconstruction of a controlled scene." To get around that issue, he and his team are currently investigating whether the AutoPhoto model can be trained on video or still images rather than 3D scenes.
Photographic feedback loop? (Score:3)
Lately there has been a general globalization of aesthetics, driven by social networks and apps. Everywhere in the world people tend to take pictures that "look like" the expected aesthetics of such-and-such application, a process further helped by the set of photographic filters offered by each app. I would only be half surprised if internally the apps used similar AI-driven "aesthetic scores" to give more exposure to the media in line with their graphic codes, leading to more likes, thus more incentive to take further pictures using the same aesthetic, etc.
I wonder, in this new context, how new aesthetics will emerge? If robots start to take pictures based on an AI "aesthetic evaluation", and that the pictures are subsequently used to train the AI doing the evaluation, it seems to me that there is a feedback loop that will only reinforce the same kind of aesthetic.
Re: (Score:1)
News and Parks (Score:2)
Queue the AI home designs (Score:2)
Queue the AI home designs where houses are built in such a way as to be pleasing to the algorithm. (Assuming we aren't there already?)
Yes, the age-old problem (Score:2)
"How can I photograph this walk-in wardrobe as if it was a Duplex to bait people into hiring this dump?"
Has been solved by AI?
Oh goody one less job (Score:2)
It's one of those little bits of job killing automation nobody talks about. Not a lot of jobs, but they keep piling up. [businessinsider.com]
I'm not saying we stop new tech, but we better figur