Until now, I have ignored Jimmy’s pleas, taking for granted my frequent access to Wikipedia. But this Thanksgiving weekend, as I was cleaning out my home office and attempting to cull my book collection to make some room on the shelves, I realized something that convinced me to make a donation.

If you have ever tried to get rid of some of your old books, you know how hard it can be to pull the trigger.

Some books, like my copy of The Great Gatsby, one of the great novels of the 20th century, are clearly keepers even though it cost just $1.00 at the used book store back when it had an intact cover.

Others books are wrenching decisions. How can I possibly toss out the TED Book Club selection from 2009? I still haven’t read it!

Some books, on the other hand, are easy decisions, headed for the recycle bin because even the library won’t take them. I found a whole shelf of books that I haven’t touched in years. Technology books, software books, reference books.

Information storage, not stories.

I realized that one of the biggest reasons I haven’t touched these reference books in a while, and certainly haven’t bought a new one in years, is Wikipedia.

The information in Wikipedia is fresher, well-written for the most part, and far more extensive than the best reference library. So why buy books that are just information stores when Wikipedia has so much more to offer?

The footnotes on Wikipedia are one of the best parts of the service. With every article on Wikipedia you are one click away from the best bibliography on the web for any topic.

Crazy as it sounds, my kids tell me that their teachers don’t allow them to cite Wikipedia in their research papers, even though it is the first place the go for any new project. Even so, Wikipedia is an invaluable research tool for students because they can go to the footnotes and find original sources that no one argues with.

To my surprise, my kids also had contributed to Wikipedia. What might a grade school kid add to the greatest encyclopedia on the planet? Adding information about the latest MMORPG? In fact, they had corrected and added to some of the topics being taught in their classroom.

I learned first hand the rigor of the Wikipedia contribution process.

A curator aptly named the “Red Pen of Doom” had reversed most of my own additions, self-serving edits aimed at revising and correcting the history of one of my early educational video games. Why the rejection? Insufficient references.

Wikipedia’s gift to education is far more than its reference value. It is the notion of radical participation.

Kids today grow up knowing that they can be active participants in the generation and curation of knowledge. The idea that knowledge is collaborative is quite different than my experience growing up with the old Encyclopedia Brittanica. The old encyclopedias engendered the feeling that knowledge only could be generated by inaccessible experts, and never was subject to question.

The reason I’m giving $100 to Wikipedia this Thanksgiving weekend is not just because I’m thankful to Jimmy Wales in persevering with this project, which has been such a gift to the human race.

It’s also economic.

When I look at my old stale reference books that not even the public library will take off my hands, I realized that I have saved hundreds of dollars over the past few years by no longer buying quickly dated references.

Just knowing that Wikipedia exists, that everything is there, including all the references, I save money — and trees.

Wikipedia has got to be the greatest bargain of the decade. So this Thanksgiving I thought I would give a little of that back.

Keep it up, Jimmy Wales. The world needs Wikipedia to thrive!

But for the 92% of online Americans who did not use Twitter last year, including some of my friends and family who asked me “What is a hash tag?” over the holidays, the answer might not be as obvious.

Hashtags emerged because Twitter only allowed posts comprised of 140 characters of free text without any obvious way to organize and categorize the content. The beautiful simplicity of Twitter fueled rapid and viral growth, and the Twitter community looked for ways to organize the flood of information within the 140 character constraint.

Hashtags became a fundamental organizing principle because you only needed to sacrifice one character of your 140 free text field, and putting a # in front of any word gained an easy way to associate information into relevant topical streams.

Hashtag example from Catch Notes for iPhone

Catch Notes uses hashtags for the same reason: The notes field starts as a simple free text entry field, without the need to add special titles and categories. The first line of each note automatically is considered the title. Any word with a hash sign in front of it automatically becomes a category and tag for organizing and associating your notes.

I use hashtags in Catch to create interlinked streams of related information about topics ranging from restaurants to recipes, from ideas to expense reports, and just about anything else that I might want to remember and come back to later.

Whenever I find a restaurant I want to go back to, I take a quick geo-tagged photo note using Catch Notes on my iPhone, and I drop a # in front of the word restaurant. Now the note is in my #restaurant stream. Once you have created a few hashtags, the most frequently used hashtags automatically pop up in the hashtag picker when you click the # symbol on the screen.

I also might sprinkle in hashtags like #local or #roadtrip to indicate other associations with a particular restaurant note. Maybe the chef came to the table and told us how he made a dish, and I add a note with the #recipe, automatically linked to all of the other recipes that I have collected or clipped from the web. Or maybe the #restaurant note also contains a business #expense that should land in my expense report, or we talked about an #opportunity and I want to set a reminder to follow up.

Hashtag example from synced notes at Catch.com

Hashtags in Catch appear as links in the sidebar, like categories. They also appear as links within the note itself, as hashtags. We designed Catch for people who don’t necessarily have an elaborately planned filing system, and and who want to keep their organizing principles fluid. The world around us keeps changing with new information, new topics, new ideas, and it is hard to define a fixed filing system.

The idea behind hashtags in Catch is to allow users to think freely and capture their ideas in the moment without fretting about how to organize them or which folders they should go in. Organization emerges with increased use of Catch, and it is easy to change simply by adding a # in the right places.

Social Media is a Conversation: Visualization of Interactions on Facebook

In The Lord of the Rings, a palantír stone was like a crystal ball. When you looked into the stone, you could see what was happening near other palantíri around the world. If you could learn to manipulate the stones, you could gain great power by using the stones not only to see the physical world, but also to peer into the history, the thoughts, and the intentions of other people around the world.

Palantír is an apt name for a new project from Facebook that might allow us for the first time to glimpse the world from Facebook’s point of view. Palantír is a visualization of the data flowing through and collected by Facebook: Conversations, comments, photos, friend requests, pokes, status updates, and more.

Each action is represented by a light particle or an icon that floats into the atmosphere from its point of origin on the earth. Conversations and interactions between people become three dimensional splines of light or comets traversing the globe connecting the locations of the participants. These are the artifacts of the social interaction of 120 million of earth’s citizens. It is a stunning display of our interconnectedness, a heatmap of human interaction on the network.

Mark Zuckerberg first launched the website that would become Facebook on February 4th, 2004. It was only for Harvard students, to help people get to know each other. It clearly found a need: Half of the student body signed up in the first two weeks. Chris Hughes and Duston Moskovitz joined Mark to work on the site. Four months later, Facebook had added 30 more college networks. That summer, Facebook moved to Palo Alto, joined forces with the cofounder of Napster, Sean Parker, raised $500,000 from the cofounder of PayPal, Peter Thiel, and never left.

Four years later, Facebook is the 4th most trafficked website in the world, and counts 85% of American college students as members. Barely out of college himself, Facebook co-founder Chris Hughes would help change the world again as Director of Online Organizing for Barack Obama, proving the power of social media to motivate and mobilize people to action.

Facebook has become the number one place on the web to share photos, hosting more than 10 billion photos already, and adding another 30 million new photos every day. In addition to photos, users are sharing everything from what they are doing right now to the latest news, thoughts, ideas, opinions, events, music, books, movies, and more. It is hard to comprehend so much data. Facebook engineers Jack Lindamood, Kevin Der, and Dan Weatherford created Palantir to help us visualize it.

Facebook is considering releasing the Palantír to the public as a feature, so that Facebook users everwhere can look into the crystal ball of Facebook data and see how the world is buzzing with interconnectedness and conversation. This is just a tiny glimpse, however, of the stunning view of the world that must be possible from inside the walls of Facebook.

Taking notes can translate a fleeting conception in our brain into action, and that action usually involves sharing our experience or idea with someone else. As we carry more and more powerful technology around with us every moment of the day, might it be possible that we can start to translate more of our otherwise forgotten notes-to-self and fleeting experiences into shared experiences? Into the start of conversations that make our lives richer?

In 1995, Bill Gates concluded his book The Road Ahead with a prescient but disquieting idea: Carrying around mobile technology connected to networks would lead to the fully “documented life”:

“Your wallet PC will be able to keep audio, time, location, and eventually even video records of everything that happens to you. It will be able to record every word you say and every word said to you, as well as your body temperature, your blood pressure, the barometric pressure, and a variety of other data about you and your surroundings.… It will be able to track your interactions with the network—all of the commands you issue, the messages you send, the people you call or who call you.”

What made the “documented life” such a scary idea was the notion that we were not the ones doing the documenting. Someone else, enabled by pervasive computing technology, was documenting us, and using our data for some other purpose—perhaps understanding insurance risk, perhaps figuring out how to best persuade us to buy something, perhaps something more nefarious. The passive voice of the “documented life” left open the question: Documented by whom? The function of technology might be one of monitoring us rather than empowering us.

Sometimes monitoring can be a good thing—health monitoring, for example. If we have a chronic health condition like diabetes or heart disease, then we might want the reassurance that someone is monitoring us in order to identify problems early, before something gets worse. Health, after all, is rife with cognitive dissonance: We often do not do or pay attention to the things that we know consciously are healthy or are the right things to do. Remote health monitoring can help us share that responsibility with someone else, someone who can coach us and support us.

Remote health monitoring is, even if good for you, usually implemented by someone else with their own interests in mind. A telehealth monitoring device can query me at my bedside:

How are you feeling today, [insert patient name]?
Did you take your [insert prescription name] medications?
Do you have any new symptoms of your [diabetes, heart disease, asthma, COPD]?
What is your [blood glucose, blood pressure, weight, peak respiratory flow, temperature, blood oxygen saturation, pulse]?

Of course, remote monitoring systems can be more thoughtful and personalized than the example above, but the fact that remote patient monitoring encounters are pre-scripted makes them inherently impersonal, no matter how good the algorithms are at inserting my name and my needs in the right place at the right time. And the stated purpose is the automation and increased productivity of someone who otherwise might have to pay for my hospital bill.

Social media, on the other hand, is all about me sharing my experience, through my networks, with others that I choose, or with the world at large, if I so choose.

Wikipedia, the global experiment in sharing knowledge that has replaced every other encyclopedia, currently defines social media as follows:

Social media are primarily Internet- and mobile-based tools for sharing and discussing information among human beings. The term most often refers to activities that integrate technology, telecommunications and social interaction, and the construction of words, pictures, videos and audio. This interaction, and the manner in which information is presented, depends on the varied perspectives and “building” of shared meaning among communities, as people share their stories and experiences.

This is the current definition of social media, but it could change. In fact, if you are so inclined, you can register at Wikipedia right now and try your hand at improving on this or any other article. But you better be a great writer and know your references, or the TheRedPenOfDoom will undo your work. Wikipedia has become so robust and accurate because the collective intelligence of an extensive community, constantly checking and curating the content, is far greater than any individual writer or editor. In Wikipedia, together we are documenting the life of the human race.

Social media enables a documented life, but unlike The Road Ahead, the subject is clear: We are the primary actors doing the documenting. We can share our photos on Flickr, our videos on YouTube, our friends on Facebook, our musical tastes on MySpace, what we are doing right now on Twitter or FriendFeed, news and websites that interest us on Digg or StumbledUpon or Delicious, our resume on LinkedIn, our contact information on Plaxo, our ideas on WordPress or Blogger, and much more.

Despite the plethora of opportunities to share using social media, we remain challenged by the fact each service only feels appropriate for a subset of our ideas, for a subset of people with whom we might want to share. With all the fragmentation, sharing becomes work. But in the end, each service aims to do the same fundamental thing: Help us record and share our ideas. And each service is based on the same fundamental premise: That our lives are richer when we share.

When we participate in social media today, we need have a good sense of what we want to share, with whom, and with which type of media, before we even login. It doesn’t matter that every idea I might want to share starts with a private mental note to myself, that every idea that starts with a mental note might evolve to include many media types and might be something to share with many different audiences. I need to be comfortable enough with what I want to say, to whom, and how I want to express it to thrust myself into a particular public space when I login.

What I am looking for is a better way to capture and share my thoughts and experiences—first as mental notes to myself, and then later, after I refine and edit them, after I decide with whom and when and where, to share them. I am looking for something that requires no effort, no work, yet can help me record rich and detailed information. Something that can capture my shorthand chicken scratches, but knows what I mean enough to fill in the blanks and complete the sentences. I am looking for something unscripted, yet intelligent, so that the right services can do something useful with the information I collect in ways that help me.

Every WordPress blog, every Facebook update, every Digg news post, started with an experience or an idea that, for whatever reason, someone thought might be worth remembering. When we want to remember something, it is because we want to act on it; if not now, then someday. Time slips away, but some of the stream of ideas and experiences beg to be captured so that they can be shared. Each note that we want to remember has a purpose; it is an idea that seeks to achieve some result. That result is most often achieved by sharing.

Note taking is a social act. A fully documented life enabled by technology might scare us, but a life that we document more fully, so that we can share more fully, is one that just might be richer.

Google Sets Home Health Monitoring Example

For obvious examples, type “Porsche” and “Mercedes” into Google Sets, and the site returns Ferrari, BMW, Audi and Lamborghini, among others. Type in “brain fitness” and Google Sets returns brain health, mind fitness, mental health, stress, cognitive neuroscience, brain training, and more.

For something less obvious, I challenged Google with the name of my old company, Health Hero Network. Google Sets accurately found AMGA, the acronym for the American Medical Group Association, the company’s partner on an important but not very well known Medicare chronic care improvement demonstration project. Google Sets also identified two of the company’s competitors, Viterion Telehealthcare and AMD Telemedicine, but missed the larger rivals Philips Telehealth and Honeywell HomMed as well as Intel Digital Health, the latest entrant into the home health monitoring market. Google Sets also didn’t pick up the fact that Health Hero Network is now part of Robert Bosch GmbH.

Digging deeper, I tested the phrase “home health monitoring”, the field of capturing health information from patients at home and sharing it with caregivers. This niche appeared to have stumped Google Sets. The website returned a number of terms related to nursing, but veered off into some bizarrely unrelated phrases like “buy sexy Halloween shoes” and “wholesale shoes deal”. Maybe Google has discovered a connection between home health monitoring and sexy Halloween shoes that is not yet obvious to the general population. Maybe this blog post is only going to contribute to the weird association in the Google search index. If enough people press the Digg button above to send this link to the web’s largest social bookmarking site, the behavioral anthropologists studying Google search data could see a stream of even more confusing data!

Google Trends Global Warming and Beer Example

Not only is Google in a unique position to know what the world’s online population is thinking, but Google also serves as a collective memory for our thoughts and interests over time. In past blogs and presentations I have used Google Trends to illustrate how our interests change over time, illustrating the rise of sharing on social networks like Facebook, using web analytics to foretell the fates of presidential campaign contenders, and to lament the relative lack of interest in important topics like global warming compared to things like football and beer, as shown above. See my presentations on life sciences innovation, new chronic care models, and telehealth technology for more examples.

Google Labs is currently running an experiment that could extend our collective memory decades into the past with an new search filter called Timeline. For awhile now, Google has been indexing libraries and archives of books, news, and patents from the past. By looking for the date of origin in footnotes, endnotes and any other notable information in its database, Google may soon be able to give us a clearer sense for how our interests have evolved over the past century, not just the past year.

Google Medicare Experimental Search Timeline Example

Take note of Google Trends, Google Sets and Google Experimental Search using Timeline from Google Labs to get more insights on just what Google means when the company describes its mission to “organize the world’s information and make it universally accessible and useful.”

I typed the word “economy” into the search engine Google today, and saw that BarackObama.com is the third search result on the list out of 254,000,000 sites in the Google index. This is an amazing feat of Search Engine Optimization.

There are three fundamental ways to get attention on the Internet: pay for it, word of mouth, or through organic search results.

Paying for traffic by purchasing keywords from Google is expensive. It is so expensive to buy traffic that Google brought in a record $5.5 billion revenues for the three months ending September 30, 2008.

Word of mouth is the power of social networks like Facebook and Myspace. Information that we post travels through our newsfeed with lightning speed to our friends, even faster than we can email a YouTube video.

Organic search is perhaps the most important way of obtaining traffic. When we are seeking something, most of us start with a Google search. Traffic that comes in from a search result tends to be the most relevant. Rarely do we make it past the first page or two of Google results, however, before clicking on a search result or trying a new search term.

How does a website end up “above the fold” on the first Google result page out of 254,000,000 results? The exact Google algorithm is a well kept secret that Google keeps adjusting and updating, but the basic PageRank concept that Google patented 7 years ago still provides the underlying model. Google is looking for the center of attention, and their computers rank pages based on quantity and rank of the pages that are pointing to it.

Google PageRank Patent

Because lots of other sites copy content and use artificial means to get other sites to link to them, the algorithm needs to be updated constantly to do a better job of identifying original rather than copied content and to identify and discount and paid or artificial links. The way to be at the top of the list is to have quality original content useful enough to inspire lots of other quality sites to reference, plus Search Engine Optimization or SEO.

Winning SEO combines high quality, high utility content that is highly relevant to a keyword, together with coding that ensures the Google robots are able to find and prioritize the keyword you are optimizing, and intentional strategies to get high quality sites to link to your site with the intended keyword. This usually does not happen by accident. The appearance of BarackObama.com near the top the first page for a generic keyword like Economy can only be the result of very thoughtful and diligent SEO practice.

Fascinated by the ability of the Barack Obama campaign to land near the top of the list for the number one campaign issue, while JohnMcCain.com is nowhere to be found, I looked at other election issues, starting with the CNN issue list below and then adding some variations.

CNN Election Issues 2008

Here are the results. I marked each CNN issue with a plus sign in the left column. The number of searches is for the month of October 2008, as determined by the Google keyword tool made available to advertisers would like to know which keywords to purchase and how many results to expect. The Index Pages indicates just how many web pages Google has in its database competing for the given keyword.

After you do a search, Google generally shows 10 results per page, plus a sprinkling of results from other Google services like news, books, blogs and videos. I only counted actual search results, not Google promoted links. When I searched for the issue term, I looked up to four or five pages deep, and I noted the page that either BarackObama.com or JohnMcCain.com came up in the list, along with the actual placement rank.

Not just for “economy” but for many of the top issues, BarackObama.com appeared on the first or second pages, including first page results for the hottest issues as measured by actual search queries: healthcare, energy, and immigration. JohnMcCain.com was also on the first page for healthcare, and made page one for climate change and national security, but in general the John McCain website had far fewer search results for far fewer issues than Barack Obama.

In the book Click: What Millions of People Are Doing Online and Why it Matters, Bill Tancer provides an inside view into just how much data about our online behavior is routinely collected and what those clicks reveal about the thoughts and intentions of a population. As head of Global Research for Hitwise, a web analytics company now part of credit ratings giant Experian, Bill Tancer has at his fingertips a continuous datastream from 25 million Internet users, collected anonymously through Internet Service Providers (ISPs). It must be even more fascinating to see Google’s data, as the most common act we do when we are interested in something is to type it into a search engine.

After reading Bill Tancer’s book, I put some of the ideas to the test with my own research based on web analytics made publicly available for free from Google and from Compete.com. If Internet behavior is an indicator of the intentions of a population, then what can we learn about the current political campaigns of Barack Obama and John McCain? For a sense of perspective, I also included Hillary Clinton in my comparison. The Compete.com graph shows unique monthly visitors to BarackObama.com, JohnMcCain.com and HillaryClinton.com. Despite steady growth in visits to JohnMcCain.com in recent months, BarackObama.com led JohnMcCain.com in website visits for September by over 2.4 million monthly uniques visitors, with 5.5 million unique visitors to BarackObama.com compared to 3.1 million unique visitors to JohnMcCain.com.

BarackObama.com v. JohnMcCain.com v. HillaryClinton.com on Compete.com

On Google.com/Trends, Google reveals trends in the data it collects and records on what people have been typing into the search engine. I compared searches for Barack Obama, John McCain and again Hillary Clinton for all searches originating in the United States over the past 12 months. While searches for John McCain briefly overtook Barack Obama around the time of the Republican Convention, Barack Obama has had a steady lead.

Barack Obama v. John McCain v. Hillary Clinton in Google Trends

Zooming in to searches conducted in the past 30 days, we can see that interest in both candidates continues to increase as we get closer to election day on November 4. Barack Obama’s lead over John McCain in search traffic has accelerated, and by the end of October, Google searches for Barack Obama led John McCain by about 2.5 to 1.

30 Day Google Trends: Searches for Barack Obama v. John McCain v. Hillary Clinton

Google also gives us the top locations for search traffic, so we can easily see the top geographic hot spots for presidential candidate search terms, which should correlate to what people are thinking and talking about.

Top Regions for Presidential Searches on Google

Not surprisingly, there is a high overlap between the areas with the most searches for presidential candidates and the battleground states where the campaigns are the most intense. All of the key battlegrounds except Florida are in the top 10 regions in terms of presidential candidate search traffic, with the hottest races in Indiana, Ohio, North Carolina and Missouri at the top of the list.

Battleground States Map from New York Times

One of the insights Bill Tancer demonstrates in the book Click is how traditional television media can drive search traffic. A television advertisement can raise awareness of an issue, and then lead us or even explicitly give us a “call to action” to go to the Internet to dig deeper.

It is easy to jump to conclusions, because with the free web analytics used here, we don’t know what motivates the search behavior. It might easily be buzz about a parody of a candidate on Saturday Night Live, Steven Colbert, or the Daily Show with John Stewart, with a flurry of web searches to catch the clip on YouTube or Hulu.com. Some things we know for sure. First, the objective behavioral data of the Internet has proven many previously untested assumptions to be wrong. Second, despite the fact that campaigns produce more 30 second television sound bites than ever, our ability to dig deeper and learn more for ourselves means that it is a completely new world this time around.