The Washington Post has apparently caused some confusion and consternation regarding a photograph that it posted on their front page on Friday.

The photograph in question is a High Dynamic Range (HDR) photo. HDR photos are indeed composites as the WaPo credit indicated, but the way that all HDR photos are taken is to quickly snap several photos (ranging from 2 or 3 all the way up to something ridiculous like 10+), and then to merge these images.

HDR photography’s entire purpose is due to the fact that a camera’s sensor often can’t capture the full range of light bouncing off a scene in the world into the camera. To compensate for that, taking several images that are targeted at several different ranges within a scene allows one to capture the full range of light, definition and detail in a scene in a single photographic representation.

This allows one to take photographs like this composite of three images I took in Chepstow Castle in Wales:

A single image taken from this vantage point would never be able to capture all three of the dark stairwell, the soft reflection off of the wood, and bright sunlight illuminating the ruins outside. If one were to capture the dark stairwell, the brightly lit exterior would be washed out, and likewise, capturing the brightly lit exterior would leave the stairwell in total darkness.

Similarly, looking at the alternative non-HDR image that the WaPo considered running, you will note the bright reflection of the sun off the side of the bridge smack dab in the middle of the photograph. An HDR image of the same scene could minimize or avoid the eye-catching glare. I can see why they chose to go with the HDR image.

Unfortunately, HDR photography is most often associated with the poor artistic choices many novices will make when testing out their compositing tools, which has resulted in a generally negative sentiment towards the technique (poking around flickr will reveal many odd, questionable, or overly artistic uses of the tool).

In spite of that, there should be no editorial controversy about legitimate uses of HDR. A photograph is simply a representation of a slice in time, captured on a single piece of film or image sensor. That slice of time could be a fraction of a second, or it could be minutes or even hours. An HDR composite is no different. It is a technique that allows a single slice of time to be captured in separate data files and stitched back together.

So, what’s the source of the controversy? My opinion is the use of the word “composite”. “Composite” is an awfully broad term, and one that is fraught with difficulties ranging from intentional deception to more innocent misrepresentations. The vital detail that any editor, journalist, or reader should ask is whether the files being composited are contiguous in time and space (would this even be a controversy if we were discussing a composited panorama?). If the answer is yes, they are contiguous, then, in my opinion, there is little room for misrepresentation. They are parts of a single whole divided into constituent components.

Chris quotes Theodore Roosevelt, and with that, it occurs to me that everything Chris says is just as applicable to places like the news world as much as it is the world of Javascript.

“It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood, who strives valiantly; who errs and comes short again and again; because there is not effort without error and shortcomings; but who does actually strive to do the deed; who knows the great enthusiasm, the great devotion, who spends himself in a worthy cause, who at the best knows in the end the triumph of high achievement and who at the worst, if he fails, at least he fails while daring greatly. So that his place shall never be with those cold and timid souls who know neither victory nor defeat.”

(via Chris Williams: An End to Negativity - JSConf.eu ☠ 2011)

Earlier generations have weathered recessions, of course; this stall we’re in has the look of something nastier. Social Security and Medicare are going to be diminished, at best. Hours worked are up even as hiring staggers along: Blood from a stone looks to be the normal order of things “going…

(Source: New York Magazine)

It brings me much excitement to announce that I’m joining Investigative Reporters & Editors (IRE for short) as their primary developer on DocumentCloud in September. As it may be apparent to followers of my blog, I have spent the past couple months diving headlong into the world of journalism and tech, particularly through the Knight-Mozilla Journalism Challenge. Through that exploration, I have been astonished to find how rich and deep a technical world has grown to push forward the causes of journalism.

DocumentCloud has been in the vanguard of journalists and software developers constructing this world. It was conceived of by three news editors, Eric Umansky, senior editor at ProPublica, Scott Klein, news apps editor at ProPublica, and Aron Pilhofer, news apps editor of the New York Times, built by an small but amazing team consisting of Jeremy Ashkenas (of CoffeeScript fame), Sam Clay (of newsblur and Tastylabs) and Amanda Hickman (organizer and tech geek extraordinaire), and funded by the John S. and James L. Knight Foundation’s News Challenge.

Many web developers know of DocumentCloud’s prolific open source software development, through Backbone.js, underscore.js and a number of others. What most are less familiar with are the causes for which DocumentCloud has built all of their great contributions to free and open source software. And as excited as I am about joining an organization of DocumentCloud’s technical prowess, I am even more excited to join IRE and DocumentCloud for the civic and public goods they aim to serve.

DocumentCloud’s goal of building a public online repository of primary source documents used in journalism sits happily at the juncture of great software development, computationally interesting problems, and civic good. In creating a public repository for journalistic source documents, the public can connect directly with the information that journalists cite. Journalists and the public in turn can refer to and communicate over the documents that represent the products and currencies of power in our society.

Aside from day to day work on the platform, I will be working on DocumentCloud’s 2011 News Challenge grant to build user annotations on documents. Journalists can already mark up documents (as you can see in the Guardian’s deciphering of James Murdoch’s closure announcement for the News of the World), and we want to provide a way that the people formerly known as the audience can participate in the journalistic process. I have some particular twists I want to pursue on this idea, but I will leave that for another blog post. :)

I’m also really excited to be joining IRE. IRE is a membership organization for journalists, and focuses heavily on journalism training, particularly technical training in recent years. IRE hosts two conferences a year as well, NICAR and IRE. I view software as a means to other ends, and so I am happy to join an organization that sees the importance of better digital literacy amongst journalists, and is now, through their adoption of DocumentCloud directly invested in the effort to make software tools for journalists.

I’m excited to see what we can accomplish. Interesting times lay ahead, and it’s up to us to shape them. I can’t wait to get started.

(via Before and After Shots of Joggers - My Modern Metropolis)

Week two of the Knight Mozilla News Lab featured lectures from Jesse James Garret, John Resig and Christian Heilmann.

@jeresig’s and @jjg’s lectures were especially interesting as they focused on what qualities make projects and products accessible to users. This is something that Aza Raskin has also contended with in his blogs regarding his new project Massive Health.

How can we incentivize, encourage or alter user behavior through application design?

There are a range of possible ways that users can use a particular application. One of the great parts of creating and on the flip side, hacking, is watching users figure out fun unintended uses for a piece of technology. That’s one of the cores of innovation. But there is a tension between providing tools that are powerful multi-function, and providing tools that users can actually understand.

The way that @jjg put this is that a lump of clay provides a much wider range of possibilities and creative choices, but creating with it requires some skill and perhaps training, while products like legos constrain the creative space but provide users an obvious place to start, and a coherent means to explore the creative space. Likewise, successful software provides users with an appropriate framework for exploring the conceptual space.

There are interesting subspaces within this domain. It is not always the case that when users have a range of behaviors available to them, they will choose beneficial or cooperative behaviors. Comment trolls are a perfect example of this. This is crucial for developers to consider, because any application where users have a range of opportunities to interact with others is a potential hole for bad behavior.

And the history of communication on the internet is basically a story of taking unrestricted messaging (email or AIM or ICQ) and adding constraints which hinder bad behavior without interfering with the majority of intended use. Spam filters, moderation systems, and facebook style walled gardens were all built with this in mind. None of these technologies have fundamentally changed the use of the systems they build upon. All of these technologies are systems designed to discourage behavior.

At the opposite end of the spectrum a variety of systems have sprung up to encourage a variety of behaviors from users. Whether they’re companies that pay for crowdsourced work, such as Amazon’s Mechanical Turk, Crowdflower, or Serv.io, or game companies like Zynga who’ve harnessed social pressure on facebook to drive user adoption of their games.

The world of journalism has explored this space some. Broadly speaking there’s been two efforts, gamification of the news, and crowdsourcing data challenges. the Guardian’s MP Expense app is one particularly interesting example of the latter. Unfortunately, the majority of the journalism related crowdsourcing apps don’t provide any reward or connection w/ the process of news production, or news consumption.

What I would like to see (and what i hope to address in one of my future projects) are better ways to provide leadership and encourage participation to users in the news making process.

I don’t like the assumption that lays under Tim Harford’s analysis. Why should we presume that twitter cascades are anything but rare and tiny? Retweeting and mass attention is weighted as the rarity that it is. How often, in a world controlled and directed by users, do you think it is possible to sync up the attention of a large number of people all at one time?

I can also tell you that Harford is wrong qualitatively. There are people who i follow and who follow me with larger follower counts. When those uberusers retweet things i’ve said, i immediately notice an uptick in participation. How much is that participation worth to me? Depends on the subject and who retweets as to what the nature of the response is.

But therein lies the difference. I use twitter as a medium for interacting with others, not as a medium to broadcast to the largest group of people. The way that people like @acarvin have managed to build and more importantly sustain such a large following is by interacting w/ users, and keeping open that possibility of interaction.

Analogously there was a dustup recently over @planetmoney’s coverage of jonathancoulton questioning whether JoCo was just a gimmick or not. JoCo pointed out that the sadsack analysis of what it is that he does makes incorrect fundamental assumptions as to what success is in the age of the internet, and importantly, what the progress of success is.

What does reaching all users on the internet actually get you? Can you handle all of that attention at once? Do you have the capacity for either monetizing or otherwise capitalizing on that attention? Because that’s the difference between 1 hit wonders w/ 15 minutes of fame, and people like @acarvin. And if you can’t, you may be better off increasing the quality of interactions with your users, not the quantity.

As I am auditing Mozilla News Lab, I’m going to deviate from the assignments a little bit. Later this week, or early next week I will explain why I am auditing instead of participating as a potential fellow, but for now I’d like to discuss two impressions that I have.

As an outsider, I’ve been quite frustrated at some of the opinions and debates that journalists have had about the way forward in the age of the internet. Because of that jaundiced perspective, the last month has been a bit of a revelation to me. My participation in MoJo has acted as a focusing lens for my interest in technology, investigation and journalism, and has provided the impetus for me to do something that I hadn’t cause to do much previously, that is, to talk to journalists interested in tech innovation.

I am currently more hopeful for the current progress and fate of journalism than I have been in a very long time. I have in the past followed along with some of the debates and opinions in journalism, but what was not apparent to me at the time was that there is a solid vanguard of tech-savvy journalists out there building tools to make journalism better. MoJo itself speaks to this point.

The first week of lectures for the Mozilla News Lab reinforced a common thread from my discussions. The first three lectures for me have really focused on the process of crafting a project, whether that means how to produce mockups to communicate your ideas (in Aza Raskin’s case), building an agile project (in Burt Herman’s case), or the importance of sketching ideas, and collaborating with others (in Amanda Cox’s case).

What is important to me about this is that news production is a parallel process. What has been impressed upon me over the past week by journalists such as Sarah Cohen and others is that news reporters aren’t conservative, or dismissive of new tools. Instead, their primary problem is that they are overworked, and uninterested in tools that will don’t help them accomplish their existing goals, e.g. producing news stories.

This is particularly relevant for people building new news tools with the goal of making Journalism better, because when someone tells you that they wish to make something better, the first question should always be “better for whom?” Journalism, as with any heterodox field has multiple constituencies, some of whom do not get along, and most of whom do not have the same motivations, objectives, or needs.

So, what I endeavor to do, and I hope that other MoJo participants do when developing their projects, in focusing on the process that they intend to use to develop their project, is ask specifically “what journalistic process am I trying to improve?” and just as importantly “am I making the process or lives of my users or participants more complicated?”

I’ve noticed an uptick in twitter spam recently, and knowing some people who work at twitter, i’ve been trying to keep an eye out for particularly egregious cases that i might be able to point their way. There’re a lot of spammers who are the equivalent of smash and grab artists. Spin up a new account, blast as many people as one can with links, as quickly as possible until you’re shut down. I’m sure you’ve seen lots of stuff like this:

These sorts of accounts almost always get banned after a couple hours, however, whatever process Twitter uses to ban these accounts takes painfully long. I’ve watched accounts continue to spam people for hours after i’ve hit the “report spam” button, blasting people with messages you could easily filter using a regular expression. I usually just use curl -I to figure out where they intend to redirect me

But what frustrates me more than the smash and grab @ replies, far more, are the spam account follows. I actually filter through all the people who follow me, partially out of curiosity, and to make sure that i’m not missing people of interest to me (not famous enough to warrant ignoring real people who follow me). So when I see patterns that again, one could catch with a fairly rudimentary filter, i usually take the 5 minutes it takes to suss out what the scam is.

So, when i noticed this in my inbox today, it was pretty easy to see which accounts were involved in the scam:

So what exactly is the scam? Oh… i see.

See, this is a particularly stupid scam, because in spite of what @SoaringVenture (no link, so no google juice) wants, tweets are all set so that search engines don’t follow their links.

So, all these jackasses are really succeeding at is annoying me (and presumably others), and demonstrating once again that not only is SEO bullshit, but SEO purveyors are idiots on top of it. If you want a higher pagerank, make content that people want and care about. There’s no dearth of interesting ways to improve the world. SEO is not one of them.