Posts tagged Digital Camera

Photosynthing everything

These past couple of days, I’ve finally gotten some time to work on the tremendous backlog of photos that I have sitting around from a number of trips. Among those pictures are sets of photos in the hundreds destined for photosynth. A number of my friends have expressed interest in what the software is, what it does, how it works, and how to take photos best suited for processing. I think now is a great opportunity to go over the basics.

What Photosynth Does

First of all, what Photosynth does is create a 3D point cloud model/representation of an object or scene from a set of photos. Depending on the scene complexity, the number of photos might be in the tens, or hundreds for sufficiently complicated scenes. It all depends on the model and how much time you have on your hands.

Perhaps the best way to explain it, is to see it. The following is a synth of the Pantheon that I recently finished processing, constructed from photos taken by my brother and I from a D80 and D90:

How it does it

The software uses feature extraction to identify textures in parts of each image that are similar, then tries to fit each corresponding from each image together to create a perspective-correct view. The process is extremely computationally intensive, but only needs to be done on the initial set of images to determine position and location. The beauty, of course, is that this process requires no human input for reconstructing the scene; it’s entirely computationally derived.

I won’t claim to be the most qualified to talk about it, but it does use feature extraction and some fancy fitting to work. An important note is that the software works based on unique features in texture, not necessarily on structure. This is why synths with lots of unique patterns turn out extremely well, while others don’t.

Creation

Creating the actual Synth is actually the easiest step; just create an account, install the software, add your photos, and go.

The real work in that process is creating proper tags, descriptions, and then adding geotagging data from photos, or later on in the web interface. Doing so is a great way to get your synth recognized.

How to take the best shots

There’s a great how-to on the official Photosynth website that goes over how to take pictures optimally, but I’d like to share some of my own.

If I’m taking a photo of a single object, something like this column, for example, I’ll try to stay equal distance away from the object, and take photos in steady progression around the subject.

The important thing to keep in mind is that although Photosynth can extrapolate the point cloud from features, it still cannot extrapolate images that you haven’t given it. Simply put, if you want to get the nice scrubber bar to circle around an object, you’ll need to take the requisite photos to make it. I find that pacing steadily around while taking photos at regular intervals is the best way.

  1. Take photos of subjects from a variety of angles. If you can, from every angle possible in an equal manner.
  2. Take photos from a single perspective pointing in multiple directions. I find that spinning around taking photos from each corner of the room works marvelously well; even though you look slightly special in the process.
  3. The most important thing to keep in mind is that quantity is generally on your side; so long as there is variety in the shots, but overlap as well.
  4. Choose subjects that have a variety of textures and features. Things like the Sistine Chapel synth really well because unique texture is everywhere, while cars generally don’t because of their solid color.
  5. Take wide angle shots of the entire room from the four corners first, then one from the center. Afterwards, take photos of objects/features close up. These are things that people will want to focus on when viewing later; a good example are pictures on the wall in a museum or specific fresco sections on a large wall.

Shameless plug

Some of my favorite Photosynth creations are:

  • Piazza San Marco: Link
  • Sistine Chapel: Link
  • Artemision Bronze: Link
  • Salpointe Graduation 2009: Link
  • Replica of Equestrian Statue of Marcus Aurelius: Link
  • Library of Celsus: Link
  • Pantheon: Link

For equal comparison, here are some that didn’t turn out so well:

  • Parthenon: Link
  • Olympic Grounds: Link
  • Port of Dubrovnik: Link

Scanning Books – My Take

Why Scan Books?

With the prevalence of eBook readers like the Nook, Kindle, Spring Design Alex and others, comes the necessity of building and maintaining a vast digital library. There are more resources online than one can easily list for both purchasing (and downloading) books in a suite of electronic formats, from PDF to DJVU, but what if you already own a book of the traditional dead-tree sort? What if you aren’t willing to purchase it again just for the convenience and ease of reading it on your brand new eBook reader?

Scanning becomes your only option.

I’ll be honest, the process isn’t easy, quick or glamorous. But it beats spending a day craning over your flatbed scanner or cutting the spine out of your expensive book to feed it through an equally expensive loose-leaf scanner (speaking of which, what the heck is up with how expensive they are?!). If the book is sufficiently expensive, it becomes an economical prospect quickly given the few hours required from start to finish.

I’m not going to address the legal/ethical/moral considerations. You could argue that making a PDF copy for yourself constitutes Fair Use, but the law being what it is, who the heck knows? Regardless, just exercise some moral introspection and decide for yourself.

Equipment

  • A relatively decent Digital SLR with wide to normal focal length lens
  • Large sheet of black construction paper
  • Tripod/Monopod and a way to hold the camera
  • Snapter or other image processing software
  • Adobe Acrobat/other PDF creating utility
  • 2-4 hours of your time, depending on the book complexity

The specific equipment I use is:

  1. Nikon D80 with Tamron 17-50 f/2.8 lens
  2. Nikon SB600 flash (optional)
  3. Nikon remote shutter release (IR)
  4. Large piece of black construction paper from Michael’s
  5. Monopod, table, and a copy of my CRC Handbook (more on that weird combo later)
  6. Snapter for processing images
  7. Adobe Acrobat 9 Pro for making PDF and OCR

Software

I’ve already mentioned Snapter twice, and although they’re commercial software (with a very generous 15 day free trial that gives you all the functionality of the real book), don’t let that fool you. I’ve had a lot of success with their software just because of how easy and functional it’s been in my experience. So much so that I went ahead and got the paid version.

That said, there are a few open source alternatives that do a pretty good job and are worth mentioning:

Scan Tailor is pretty good, has a nice GUI, and is very active. Unpaper doesn’t have a GUI but offers a lot for a command line tool. There’s always the advantage both OSS solutions offer that you can either code/propose functionality changes in the software itself with the active developers.

Another relevant article with tips is from /. , which posted ironically the week after I had already embarked on and discovered the ins and outs of scanning with a digital camera myself.

Setup

My setup is simple: I mount the camera on the monopod, stick it on the table, and balance it there with my trusty CRC handbook and some other heavy books.

You might be wondering why I didn’t just use a tripod. The reason is that it’s a much more challenging prospect to carefully both tilt the tripod and balance it so the camera is completely perpendicular to the book’s surface. For the best photo quality, one needs the book to be as close to coplanar with the camera sensor as possible. It makes sense, otherwise we’ll have a more challenging time getting the book totally in focus (depth of field will come into play), and have a harder time flattening the book in software.

I generally tape the black paper down to the floor, snap photos of the cover and back cover, and then tape those down as well. More on positioning later.

The whole thing looks like the following:

Doing this in my office

I have the flash set to bounce from the ceiling, just because in practice this yields the most readable photos. I also use all the light I can from the room itself.

A difficult consideration is that sometimes the print/copy itself has glare. This seems a lot more common with newer books than older ones; it’s almost like the print has a layer of varnish atop it. Just make sure you preview a few images and can actually read the copy.

Positioning the book is the tricky part; it’s difficult to balance between filling the frame with the book (so you have good resolution), and leaving enough space at the edges so that your software can do edge detection. Leave too little space around, and you’ll have a nightmarish time trying to field flatten. Leave too much, and you’ll be throwing away a ton of your image. Even worse, if you don’t tape the book down, it will gradually creep out of the frame.

Another big consideration is rotation. I’ve discovered that Snapter doesn’t really account that well for material that has even subtle rotation. You end up with slight skew in the resulting images. It isn’t a big problem, but rotation will immediately cause you headaches.

I usually go for something like this:

Pretty good for taken with my 3GS

You could zoom in a bit more in this case if you wanted; in practice you’ll discover for yourself what works best.

I set the camera to use a relatively big F/# (in this case F/5.6) so there’s as much depth of field as possible. You want the whole book in focus.

Now just snap away

This is the grueling part, capture images of every page. Snag a friend or something as having two people makes this process go much faster. One can turn the page and crease stubborn ones into place, and the other can trigger the shutter with the remote and make sure the book isn’t creeping out of the frame.

I find this can take anywhere between a half hour to much longer, depending on how much trouble the book gives you. The most challenging parts are the very beginning and the end. At these points, the pages have the most curve to them, sometimes sticking up. This is where sometimes creasing them down or using some tape on the stubborn ones can make or break your day.

Eventually, you’ll have a directory full of images somewhere you need processed.

Processing

Snapter-ing?

At this point, you can use whatever tool suits your fancy, but if you’re using Snapter, read on.

Click Book, grab all your photos, and go make yourself a drink as you wait for it to do initial edge detection and processing on images. Nothing is being changed, it’s just generating the initial traces around the book it finds.

After this is done comes the only other bothersome part. It’s very worthwhile to manually go through each page and make sure you’re happy with the edge detection. Frequently, pages that have black or dark color at the edge cause headaches. Drag the handles around until they match closer. This can be grueling, but it’s important.

Click Input, change the background color to black (since we’re using a black piece of paper, or at least I did). Under Output, I also generally turn cropping each page off since I’d rather deal with a spread. Grayscale output will save on space later, and I keep the DPI the same since I’ll compress and downsample later in Acrobat. Now, you can click process and have yourself another drink.

Processing Images

After this is done, you can preview the results on the right. If everything is right, click Save and wait a little longer.

Saving flattened images

Now you should have a directory full of images waiting to be made into a PDF.

Adobe Acrobat

You can use whatever you’d like to make the PDF from the resulting JPEGs, however, I’ve had luck just using Acrobat.

Click Create -> Merge Files into a Single PDF, and then grab all those images you have.

Combine them, and you should now have a huge PDF. Save it, but you aren’t done yet. At this point, I generally take a look at the PDF Optimizer under the Advanced tab, and click Audit Space Usage. Yeah, it should be pretty huge.

I cheated; this is actually after compression

If you absolutely need color, just skip this. If your book is black and white, converting is going to save you a ton of space.

To convert pages to grayscale, under Advanced click Print Production -> Convert Colors. Check “Convert Colors to Output Intent” and select “Gray Gamma 1.8.” I usually then exclude the front and back covers from the page range, unless you don’t care about that pretty color you’ll be missing out on.

This will take a while...

This process also will take some time. Adobe is multithreaded, but still doesn’t use all my 8 logical cores on my i7 920. Just be patient.

After this finishes, you should now see a dramatic difference under the space audit report for Images. There might be a lot of document overhead, however. Don’t worry, this is normal.

At this point, it usually makes the most sense to do some OCR if you want, just to make the document searchable. Document -> OCR Text Recognition -> Recognize Text Using OCR does the trick.

Adobe's OCR works quite well

Click Edit and select Searchable Image (Exact). This won’t resize your images or do compression; we’ll do that later. Now, wait a long time while it consumes CPU cycles and hopefully makes your document so much more powerful and useful.

After this finishes, you’re ready to do some compression and hopefully make your document small enough to not be an embarrassment, you storage hog, you. I usually downsample to around 300 DPI, leave monochromatic images alone (since we don’t have any), and opt for JPEG2000. Check everything in the Discard Objects, Discard User Data, and Clean Up tabs.

JPEG 2000 is amazing

Click Ok, and now be prepared to wait the longest you have yet. Even on my rig, this takes an hour or two.

Check the space audit once more, and you should now have a reasonable sized, fully searchable, readable PDF, ready for your enjoyment.