Uncategorized

Fixing Dropbox’s Camera Upload Organization Mess – EXIFmover

First, I have to note that I love Dropbox. I’ve been using the service almost since the beginning to synchronize my work across multiple desktops, notebooks, and mobile devices. There’s really no alternative at this point that rivals Dropbox in my mind.

That said, the state of their camera upload organization is abysmal. This is a recent new emphasis for Dropbox, automatically uploading all the photos taken on a mobile device or onboard an SD card inserted into a notebook. The problem is that all of these photos are dumped into the same /Dropbox/Camera Uploads folder, so you end up with a huge list of photos. That’s not so bad, except that in my case I’m dealing with sample photos taken on multiple devices for testing results in a completely unwieldy directory with so many files that it often causes Finder on OS X or Explorer on Windows to stall for seconds upon opening.

Screen Shot 2012-12-15 at 3.06.23 PMThis isn’t that well organized…

I’ve pleaded my case for a simple feature to be added – per device folder creation. This didn’t make it into the last update, and about a week or so ago I wrote a script to do exactly that. It parses the EXIF data, and sorts photos into a folder with name /[type]+[model]. The end result looks like this:

Screen Shot 2012-12-15 at 2.28.35 PMFinally, some per-device sorting

It’s called EXIFmover and I stuck it up on Github as a Gist since it’s pretty simple. I use Python 2.7 personally, so that’s what this is tailored for. I’ve tested on OS X, Windows, and Ubuntu. Python’s OS interface is fairly platform agnostic. The one prerequesite is EXIF-Py for parsing the EXIF metadata from photos, which can be obtained from that project’s Github page.

Stick both EXIFmover.py and EXIF.py in the Camera Uploads directory, and run. As more files are uploaded, this can be run again and photos will be sorted once more.

Finally, some organization.

Spoiled by iMessage – Or Things I don’t miss about SMS

As the smartphone space becomes increasingly commoditized, the obvious next step is to make hardware choice a function of ecosystem choice. That’s already happened for some of the obvious features (marketplaces for music, video, and books), but the next step is delivering ecosystem-specific features for core phone functionality. Probably one of the features I use the most on a device is messaging, and since iMessage is nearly a year old, I couldn’t help but think more about why I feel a back-of-my-mind longing for it whenever I go back to a device that’s strictly SMS. Also, a recent influx of weird SMS issues I encountered while chatting with a friend prompted both a twitter rant and me deciding to finally get all of this down somewhere.

So here’s a simple bulleted list about some of the things I don’t miss about SMS:

  • Split messages at the 160 character limit
    • The caveat here is that some carriers do indeed properly concatenate messages, but this usually only works if both sender and recipient are on the same carrier. The Verizon -> AT&T exchange used to properly implement this, back when the CDMA iPhone 4 came out, but it later was broken and inexplicably never fixed. Part of me wonders whether iMessage wasn’t spawned partially because of this annoyance.
  • Messages arriving out of order, sometimes as a result of the split from the character limit
    • This is a continual annoyance, there’s no greater way to break the flow of a conversation than by having replies arrive completely out of order. Also SMS is predominantly circuit switched, and not an IP based protocol. Obviously BBM/iMessage just work over the internet, and aren’t tied to having a working cellular session.
  • No delivery guarantee, SMS is best effort delivery only
  • Duplicate SMSes as a result of carriers implementing proprietary systems to mitigate SMS’ best effort delivery system
    • Verizon seems to be particularly aggressive about this. I still would like to know more about just how their system determines when to re-send a particular SMS, but I sometimes get 3 or 4 of the same SMS from Verizon subs.
  • No read reports or delivery status reports (at least on most new smartphone OSes that don’t implement SMS delivery receipts)
    • Delivery reports are actually a pretty common feature for SMS (usually just uses a prefix), but implementation varies from carrier to carrier and is a big mess. I suspect this is the reason smartphone platforms tend to implement it at random, if at all.
  • Delivery time is sometimes very long depending on SMSC load
    • Try getting an SMS delivered on New Years, I dare you.
  • Image sizes are smaller and more compressed for MMS, different for each carrier and up to the exchange to accommodate and tailor to sizes
    • This is set on iOS in the carrier bundle
    • iMessage can deliver full quality images if both sender and recipient are on WiFi
  • The price of an SMS
    • $0.11 per message is downright absurd

RIM understood the importance of having a platform-specific messaging product with BBM, specifically in both the platform lock-in and loyalty that this kind of feature adds. There’s a certain amount of potential energy that it takes to overcome both this messaging tie-in when switching platforms, and users can readily identify the differences. Apple clearly got this and delivered with iMessage, and I expect that having a messaging platform that spans the smartphone, tablet, and desktop will make that potential energy to escape to another ecosystem even harder to overcome. There’s a certain message or conversational velocity that makes BBM and iMessage feel entirely different from traditional IM, yet faster than the sometimes minute-delayed SMS.

Currently Google’s messaging “product” is Google Chat, but the acquisition of Meebo and other internal rumblings make me strongly suspect that a more cohesive and mobile-oriented messaging product with the same BBM/iMessage like conversational velocity is coming, maybe even at Google I/O 2012. There’s so much at work here which both shifts the balance of power away from the carriers and also makes the whole messaging experience better by sharing it with the tablet and desktop.

A Quick Analysis of EXIF data from Apple’s iPhone 4S Camera Samples

Motivation

In the course of doing smartphone reviews for AnandTech, I’ve taken a lot of photos for the expressed purpose of comparing camera quality. I don’t have an exact number, but it’s an absurd number of images, and of those, maybe 20% or so actually get published. We reviewed the iPhone 4S and discussed its camera at length in our review, but one of the things that piqued my attention was Apple’s claim that they had were sharing untouched, direct-from-the-iPhone samples online (at the bottom). In case you want all of the images, I’ve uploaded a zip here locally.

What’s interesting is that by default, iOS captures geolocation data on each image capture unless you decline the location services request on initial launch. I was curious to find out just how untouched these images were and whether all the EXIF data was left intact, including the GPS location data. Clearly these photos were taken at places that are either some engineer’s favorite hideouts, or possibly much more. Also, there’s that ever looming question of when they were taken, and whether there’s any chance anyone could’ve seen Apple employees taking photos with an unreleased iDevice at some scenic location.

You can analyze EXIF data in any number of image manipulation packages, and increasingly desktop OSes are exposing this data directly (Finder, Explorer, e.t.c.) but for quick analysis I turned to exifdata.com which does a nice job visualizing everything. Alternatively one can just dump the EXIF data using libraries like the very popular exiftool.

Image 1 – IMG1031

Here’s the dump from one of Apple’s demo images, “IMG_1031.JPG” which is the squirrel photo shown during the announcement event. There’s all the standard data included in the full headers, and you can see specifications such as focal length, exposure time, ISO, model, and the software (iOS 5) used. There are some other fields as well.

SubjectArea I believe corresponds to either where the face detection routine selected an AE/AF target with highest confidence value (most likely to be a face), or where the image was focused using tap to focus. The iPhone 4S also includes some new fields at the very bottom such as 35mm equivalent focal length, field of view, hyperfocal distance (at half this distance, all objects and further meet image plane blur criterion and appear in focus – this half trips people up), and even circle of confusion diameter (blur size/circle of confusion). These values at the very end appear to be specific to the iPhone 4S (and possibly its H4 ISP) and aren’t part of the JEITA EXIF 2.2 specification.

ExifTool Version Number         : 8.68
File Name                       : IMG_1031.JPG
Directory                       : .
File Size                       : 3.1 MB
File Modification Date/Time     : 2011:10:05 01:43:44-07:00
File Permissions                : rw-r--r--
File Type                       : JPEG
MIME Type                       : image/jpeg
Exif Byte Order                 : Big-endian (Motorola, MM)
Make                            : Apple
Camera Model Name               : iPhone 4S
Orientation                     : Rotate 180
X Resolution                    : 72
Y Resolution                    : 72
Resolution Unit                 : inches
Software                        : 5.0
Modify Date                     : 2011:08:24 13:13:33
Y Cb Cr Positioning             : Centered
Exposure Time                   : 1/286
F Number                        : 2.4
Exposure Program                : Program AE
ISO                             : 64
Exif Version                    : 0221
Date/Time Original              : 2011:08:24 13:13:33
Create Date                     : 2011:08:24 13:13:33
Components Configuration        : Y, Cb, Cr, -
Shutter Speed Value             : 1/286
Aperture Value                  : 2.4
Brightness Value                : 6.992671928
Metering Mode                   : Multi-segment
Flash                           : Auto, Did not fire
Focal Length                    : 4.3 mm
Subject Area                    : 1631 1223 881 881
Flashpix Version                : 0100
Color Space                     : sRGB
Exif Image Width                : 3264
Exif Image Height               : 2448
Sensing Method                  : One-chip color area
Exposure Mode                   : Auto
White Balance                   : Auto
Focal Length In 35mm Format     : 35 mm
Scene Capture Type              : Standard
Sharpness                       : Normal
GPS Latitude Ref                : North
GPS Longitude Ref               : West
GPS Altitude Ref                : Above Sea Level
GPS Time Stamp                  : 21:08:30
GPS Img Direction Ref           : True North
GPS Img Direction               : 346.4727273
Compression                     : JPEG (old-style)
Thumbnail Offset                : 908
Thumbnail Length                : 12311
Image Width                     : 3264
Image Height                    : 2448
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Aperture                        : 2.4
GPS Altitude                    : 1222 m Above Sea Level
GPS Latitude                    : 37 deg 44' 10.80" N
GPS Longitude                   : 119 deg 35' 58.80" W
GPS Position                    : 37 deg 44' 10.80" N, 119 deg 35' 58.80" W
Image Size                      : 3264x2448
Scale Factor To 35 mm Equivalent: 8.2
Shutter Speed                   : 1/286
Thumbnail Image                 : (Binary data 12311 bytes, use -b option to extract)
Circle Of Confusion             : 0.004 mm
Field Of View                   : 54.4 deg
Focal Length                    : 4.3 mm (35 mm equivalent: 35.0 mm)
Hyperfocal Distance             : 2.08 m
Light Value                     : 11.3

 

If you look at this data a few things pop out. There really is GPS location data inside, and just like before the iDevice also encodes what direction the phone was pointing (compass data) when the image was captured.

This first photo was captured in Yosemite National Park on August 24th 2011, a full one month and 10 days before its announcement on October 4. Interesting.

Image 2 – IMG1664

This second image is of some waves breaking on a rocky beach and shows very nice detail. Just like the first image, it too includes GPS data and the creation date.

Make                            : Apple
Camera Model Name               : iPhone 4S
Orientation                     : Rotate 180
Date/Time Original              : 2011:08:29 18:50:23
Create Date                     : 2011:08:29 18:50:23
GPS Latitude Ref                : North
GPS Longitude Ref               : West
GPS Altitude Ref                : Below Sea Level
GPS Time Stamp                  : 02:27:00
GPS Img Direction Ref           : True North
GPS Img Direction               : 357.1590909
GPS Altitude                    : 0 m Above Sea Level
GPS Latitude                    : 38 deg 26' 28.20" N
GPS Longitude                   : 123 deg 7' 36.00" W
GPS Position                    : 38 deg 26' 28.20" N, 123 deg 7' 36.00" W

I’ve truncated the exported data for this image since the same format as the previous one. Note zero feet above sea level, indeed this checks out.

This photo was taken at goat rock beach on August 29th, five days after the first image, and one month 5 days before the announcement. What’s intense about this image is that if you study the google satellite view enough, then take into account the pointing direction, you can actually see the rock in the photograph.

Image 3 – IMG0032

This image is a landscape photo with a waterfall in the distance.

Make                            : Apple
Camera Model Name               : iPhone 4S
Orientation                     : Horizontal (normal)
Date/Time Original              : 2011:08:25 09:27:36
Create Date                     : 2011:08:25 09:27:36
GPS Latitude Ref                : North
GPS Longitude Ref               : West
GPS Altitude Ref                : Above Sea Level
GPS Time Stamp                  : 17:26:10
GPS Img Direction Ref           : True North
GPS Img Direction               : 254.5312024
GPS Altitude                    : 1192 m Above Sea Level
GPS Latitude                    : 37 deg 44' 38.40" N
GPS Longitude                   : 119 deg 35' 30.60" W
GPS Position                    : 37 deg 44' 38.40" N, 119 deg 35' 30.60" W

This one also includes the GPS location and pointing data.

Upon inspection, we can see this photo was taken also at Yosemite National Park and on August 25th 2011. It seems very likely this was taken by the same person who took the squirrel photo, given the fact that it’s a day later and in the same region in the park – perhaps a camping trip or something?

Image 4 – IMG1295

This image is of a person holding a flower up to the camera showing shallow depth of field and some serious bokeh. The subject is the same as shown in the 1080p video sample Apple provides, and appears to lack GPS data of any kind, but was taken on August 29th.

Make                            : Apple
Camera Model Name               : iPhone 4S
Orientation                     : Horizontal (normal)
Date/Time Original              : 2011:08:29 15:54:02
Create Date                     : 2011:08:29 15:54:02

Image 5 – IMG0945

This image looks almost like a test scene, and includes some potted herbs, drawers, fruits, lemons, and a bunch of different knick knacks. The EXIF data for this one includes GPS data.

Make                            : Apple
Camera Model Name               : iPhone 4S
Orientation                     : Horizontal (normal)
Date/Time Original              : 2011:08:30 14:01:03
Create Date                     : 2011:08:30 14:01:03
GPS Latitude Ref                : North
GPS Longitude Ref               : East
GPS Altitude Ref                : Above Sea Level
GPS Time Stamp                  : 13:25:21
GPS Img Direction Ref           : True North
GPS Img Direction               : 21.31747484
GPS Altitude                    : 342.1 m Above Sea Level
GPS Latitude                    : 50 deg 55' 6.60" N
GPS Longitude                   : 14 deg 3' 24.00" E
GPS Position                    : 50 deg 55' 6.60" N, 14 deg 3' 24.00" E

So far the images we’ve seen have been from just inside the US – this image is from the Königstein Fortress in Germany of all places, and was taken on August 30th.

This one is particularly interesting since it’s a great demonstration photo but also is the only sample image not taken in the US.

Image 6 – IMG1401

This image also has no GPS data, which means it possibly was taken by the same iPhone 4S as number 4 (and this user disabled location services for the camera app), or just someone else who was likewise cautious to not include position. This image is a macro of some flowers.

Make                            : Apple
Camera Model Name               : iPhone 4S
Orientation                     : Horizontal (normal)
Date/Time Original              : 2011:08:29 17:30:26
Create Date                     : 2011:08:29 17:30:26

This image was taken on August 29th, however.

Image 7 – IMG1720

Likewise, this one also lacks GPS data, but shows a beach at dusk (thus, west coast) and appears to have been taken likewise on August 29th in the evening.

Make                            : Apple
Camera Model Name               : iPhone 4S
Orientation                     : Horizontal (normal)
Date/Time Original              : 2011:08:29 19:28:56
Create Date                     : 2011:08:29 19:28:56

Image 8 – IMG0940

The last image shows the same hot air balloon as shown in the 1080p video sample, and includes GPS data. We can see where this video (and likely image 4) was taken.

Make                            : Apple
Camera Model Name               : iPhone 4S
Orientation                     : Rotate 180
Date/Time Original              : 2011:08:30 10:33:45
Create Date                     : 2011:08:30 10:33:45
GPS Latitude Ref                : North
GPS Longitude Ref               : West
GPS Altitude Ref                : Above Sea Level
GPS Time Stamp                  : 18:48:01
GPS Img Direction Ref           : True North
GPS Img Direction               : 270.9027778
GPS Altitude                    : 31 m Above Sea Level
GPS Latitude                    : 38 deg 30' 13.20" N
GPS Longitude                   : 122 deg 46' 21.00" W
GPS Position                    : 38 deg 30' 13.20" N, 122 deg 46' 21.00" W

Oddly enough image 4 and image 8 were taken on different days, so this possibly is a different location. That said, the location appears to be the Kendall-Jackson winery right off the 101.

Like the beach photo, this is just up the coast from San Francisco. The photo was taken August 30th as well.

Conclusions

Apple took sample images for its iPhone 4S presentation between the dates of August 24th and August 30th, a little over one month before the public unveiling. It’s interesting to note that had you been fortuitous enough to be in Yosemite National Park on the 24th or 25th, you might have by chance seen an iPhone 4S snapping photos of scenery and squirrels. Other locations near San Francisco seem logical given Apple’s location in Cupertino, CA, leaving the photo from Germany an odd outlier.

It seems that for all the scrutiny placed on bars surrounding 1 infinite loop, being in picturesque locales one month before an iPhone unveiling is quite possibly another logical strategy for spotting an unreleased iPhone.

I have to come full circle and say that I’m impressed Apple really didn’t scrub data from its sample images. This is something even I do to provide some location anonymity in the course of uploading sample images, though as of late I’ve become sloppy with scrubbing all the GPS data from EXIF for each sample image.

 

 

Arizona SB1070 – My Thoughts

Earlier today, I was reading yet another Digg article on Arizona’s immigration bill. For the large part, most of the articles and comments I’ve been reading have accused Arizonans of either being gun-toting crazies or racist white elites. I’m sure (read: certain) there’s some demographic of the state that probably is, but the entire state people? What a way to typecast.

Anyhow, something about what I was reading there finally compelled me to write a bit, and what started small quickly ballooned to a huge comment I left on the post. I’m reproducing it below:


Epic Long Post:

It’s time we settle this illegal immigration dispute once and for all, honestly. I’m a native Arizonan, and I can honestly attest to how completely out of hand this situation is getting, and how completely misunderstood and misconstrued the current state of affairs are down here.

First of all, the majority of Arizonans support this legislation. Now, before you write us all off as being racially insensitive bigots and crazies, ask yourself what the rational reasons could be for passing such a sweeping piece of legislation. I’m shocked at the fact that this discussion is almost entirely centered around racial profiling (do you not show your ID for everything else already? Being pulled over? Getting on a plane? Buying something?) and the economy (albeit very superficially). The problem has gotten so immense that it literally has effects on almost every major issue.

To be honest, I don’t know how I feel about the bill, I just think it’s time this issue gets the serious attention it’s been sorely lacking for the greater part of two decades now. If nothing else, Brewer should be applauded for finally getting the border states in the limelight and *some* debate going, even if it’s entirely misplaced.

So just bear with me, put aside your misconceptions about the issue (because odds are, you don’t live here, you don’t follow the issue, and you’re probably not aware of the scope of the problem), and think.

1. The environmental aspect is being completely downplayed. This is something that has even the most liberal of the liberals supporting drastic immigration reform down here in the Sonoran Desert; the long and short of it is that Mexicans and drug traffickers are literally shitting all over the desert. The sheer volume of people crossing through these corridors in the desert, and the trash they bring with them, is absolutely stunning.

Example photo of waste in the desert

Don’t believe me? Look: http://www.tucsonweekly.com/tucson/trashing-arizona/Content?oid=1168857 Some of the locations in here are barely a 10 minute drive from where I sit now. Talk to me about the environment, and then look at the kind of mess being left out there. I don’t care what the solution is, this kind of dumping/shedding of stuff/massive ecological disaster cannot continue. It can’t. It’s literally devastating.

2. Drug trafficking. Has anyone even talked about this? It isn’t just about arresting working Mexican families, it’s about combating the completely out of control drug trafficking problem going on in our backyards. In fact, I’d say that probably the main catalyst has to deal with security rather than economical drain – in fact, there’s no arguing the fact that the Mexicans living here are probably helping us out with their labor and efforts, rather than draining the local economy.

In case you haven’t been following, the drug cartels are now nearly out of control in Mexico, in fact, it’s a problem that’s of more immediate concern to us down here (in terms of security) than terrorism. In fact, screw terrorism, I’m more worried about my family being shot or killed in the crossfire of this ongoing drug battle than some terrorist setting a bomb off. Read about how insane this is: http://online.wsj.com/article/SB123518102536038463.html

“The U.S. Justice Department considers the Mexican drug cartels as the greatest organized crime threat to the United States.” http://en.wikipedia.org/wiki/Mexican_Drug_War You better believe it. People are being killed in Juarez, Nogales, everywhere. This is literally next door, folks! Not a continent away! Full scale political unrest! Talk about a threat to national security.

3. The murder of Rob Krentz has galvanized support for serious, strong, kid-gloves-off reform in the state. If you aren’t familiar, this super high profile incident involved the murder of a well liked, peaceful Arizona rancher on his own property some weeks ago. http://azstarnet.com/news/local/crime/article_db544bc6-3b5b-11df-843b-001cc4c03286.html It’s now been found that marijuana was found on the site, and there’s definite drug trafficking ties as the ranch lies one of the numerous well-known migration and trafficking corridors that dominate southern Arizona.

Robert Krentz

I think when the history books are written, this guy’s shooting will be a real inflection point you can point to as leading to this kind of legislation. The sentiment for structured amnesty or some other kind of reform almost completely disappeared after a few similar incidents. Violent, often fatal crime near the border is literally making it a physical hazard to be down here.

Want more proof? Look no further than the concealed carry legislation that also just passed. It isn’t that we’re all a bunch of friggin psychos, it’s that we’re honest-to-god scared of being shot in our homes or out in the desert. I know I sure as hell wouldn’t go walking around out there when even the border patrol is worried about some parts of the desert even just an hour away.

4. Sure the economy has something to do with it, absolutely. Hell, our economy is worse off than California’s by percentage and by capita: http://www.bizjournals.com/phoenix/stories/2008/02/25/daily29.html

The major public universities in the state are struggling for dollars to keep classes going, mandatory furloughs everywhere, and we’re paying for the rest in fees and still not going to break even. Hell yes, the economy has something to do with the perception that illegals are partly responsible. (however true or untrue that actually may be, since personally I’d wager Mexican migrant labor probably has a net positive effect on the local economy; let’s be honest, profiling them as lazy people really *is* racism)

So there are a few good arguments I don’t really feel have been emphasized enough online, anywhere. Sit around and discuss the finer points of constitutional law and whether this is “racial profiling,” honestly, that debate has already been beaten and played out enough already.

Meanwhile, the problems down here are getting worse, and worse, and worse, and the very real drug war raging in the desert just continues to get scarier. I think this will be a very interesting and potentially huge states rights issue. In the meantime, some of the points I touched on (I hope) are good food for thought if you think that Arizona suddenly just decided to “go insane” or “lose our collective shit.”

I promise you, it isn’t the case.