These past couple of days, I’ve finally gotten some time to work on the tremendous backlog of photos that I have sitting around from a number of trips. Among those pictures are sets of photos in the hundreds destined for photosynth. A number of my friends have expressed interest in what the software is, what it does, how it works, and how to take photos best suited for processing. I think now is a great opportunity to go over the basics.

What Photosynth Does

First of all, what Photosynth does is create a 3D point cloud model/representation of an object or scene from a set of photos. Depending on the scene complexity, the number of photos might be in the tens, or hundreds for sufficiently complicated scenes. It all depends on the model and how much time you have on your hands.

Perhaps the best way to explain it, is to see it. The following is a synth of the Pantheon that I recently finished processing, constructed from photos taken by my brother and I from a D80 and D90:

How it does it

The software uses feature extraction to identify textures in parts of each image that are similar, then tries to fit each corresponding from each image together to create a perspective-correct view. The process is extremely computationally intensive, but only needs to be done on the initial set of images to determine position and location. The beauty, of course, is that this process requires no human input for reconstructing the scene; it’s entirely computationally derived.

I won’t claim to be the most qualified to talk about it, but it does use feature extraction and some fancy fitting to work. An important note is that the software works based on unique features in texture, not necessarily on structure. This is why synths with lots of unique patterns turn out extremely well, while others don’t.

Creation

Creating the actual Synth is actually the easiest step; just create an account, install the software, add your photos, and go.

The real work in that process is creating proper tags, descriptions, and then adding geotagging data from photos, or later on in the web interface. Doing so is a great way to get your synth recognized.

How to take the best shots

There’s a great how-to on the official Photosynth website that goes over how to take pictures optimally, but I’d like to share some of my own.

If I’m taking a photo of a single object, something like this column, for example, I’ll try to stay equal distance away from the object, and take photos in steady progression around the subject.

The important thing to keep in mind is that although Photosynth can extrapolate the point cloud from features, it still cannot extrapolate images that you haven’t given it. Simply put, if you want to get the nice scrubber bar to circle around an object, you’ll need to take the requisite photos to make it. I find that pacing steadily around while taking photos at regular intervals is the best way.

  1. Take photos of subjects from a variety of angles. If you can, from every angle possible in an equal manner.
  2. Take photos from a single perspective pointing in multiple directions. I find that spinning around taking photos from each corner of the room works marvelously well; even though you look slightly special in the process.
  3. The most important thing to keep in mind is that quantity is generally on your side; so long as there is variety in the shots, but overlap as well.
  4. Choose subjects that have a variety of textures and features. Things like the Sistine Chapel synth really well because unique texture is everywhere, while cars generally don’t because of their solid color.
  5. Take wide angle shots of the entire room from the four corners first, then one from the center. Afterwards, take photos of objects/features close up. These are things that people will want to focus on when viewing later; a good example are pictures on the wall in a museum or specific fresco sections on a large wall.

Shameless plug

Some of my favorite Photosynth creations are:

  • Piazza San Marco: Link
  • Sistine Chapel: Link
  • Artemision Bronze: Link
  • Salpointe Graduation 2009: Link
  • Replica of Equestrian Statue of Marcus Aurelius: Link
  • Library of Celsus: Link
  • Pantheon: Link

For equal comparison, here are some that didn’t turn out so well:

  • Parthenon: Link
  • Olympic Grounds: Link
  • Port of Dubrovnik: Link