HowDoYa

... is a search engine for finding explanations on how to do things -- like seal a driveway, grow a garden or buy a swimsuit.

[via Phil Bradley's weblog]

Google News and the dearth of local news

Google upgraded its news search this week so you can type in a city name or ZIP code to find news for where you live:

While we’re not the first news site to aggregate local news, we’re doing it a bit differently -- we're able to create a local section for any city, state or country in the world and include thousands of sources. We’re not simply looking at the byline or the source, but instead we analyze every word in every story to understand what location the news is about and where the source is located.

Google is years behind Topix, which is partly-owned by Gannett, The Courier-Journal's parent company. You've been able to search for local news by ZIP code at Topix since 2004. On its blog, Topix greeted the Google news by doing some math:

If you take the totality of news created from the mainstream press (newspapers, radio, television stations), and do the back of the napkin math for all the local stories generated on a daily basis, here's what you get

1,440 local daily papers X 6 stories a day = 8,640
2,303 news radio stations X 3 stories a day = 6,909
1,686 television stations X 4 stories a day = 6,744
-----------------------------------------------------------------------------
Total local stories per day 22,293
Number of populated US ZIP codes 32,500

Topix concluded that "Local news is not a search problem."

We started by trying to add more sources. We added government, weather and industry sources, and then we added 25,000 blogs to the mix to see what we'd get. And, pretty much, they still didn't provide the breadth of coverage around local news, especially around small towns.

...

The conclusion our users pointed us to, by sending us hundreds of stories a day through our feedback form, was that there wasn't enough coverage of most areas by the mainstream press or the blogsphere, and that the real opportunity was to become a place for people to publish commentary and stories -- because no one else was going to do it.

So, we launched the Topix forums, and two years and 25 million posts later, Topix has gone from being merely an aggregator of local news, to becoming the home of local voice on the web

I've been mostly disappointed with sites that purport to offer tailored, local news because I haven't found enough compelling content to keep me coming back. Google's latest offering needs work too. I typed in both my city name and ZIP code and got nothing, which seems more than a little ham-handed. And I have no clue why last night I typed google news in the search box and got nothing:

But it works fine today.

Regardless, if it's really true that we're all just narcissists who only care about ourselves, then maybe what we're really looking for is the future of news search as predicted by Google Blogoscoped.

Using Google's Location Syntax for Election Information

... as explained by WRAL.com:

Google News, at http://news.google.com, looks very much like Google and works very much like Google. You enter a few keywords and get your search results. But Google News is different in that it has a few special search syntax that allow you to really narrow your search. One of the syntax is called location:. In this blog post I'm going to give you some pointers on using location: to get election information.

Read/WriteWeb: "10 Semantic Apps to Watch"

The Read/WriteWeb defines a semantic application as an application that tries "to determine the meaning of text and other data, and then create connections for users." Here's its list of "10 Semantic Apps to Watch."

Deep Web Research

Guess this is Marcus P. Zillman week at Depth Reporting. Here's an article he wrote for LLRX.com on "deep Web research":

The Deep Web covers somewhere in the vicinity of 900 billion pages of information located through the world wide web in various files and formats that the current search engines on the Internet either cannot find or have difficulty accessing. Search engines currently locate approximately 20 billion pages.

In the last several years, some of the more comprehensive search engines have written algorithms to search the deeper portions of the world wide web by attempting to find files such as .pdf, .doc, .xls, ppt, .ps. and others. These files are predominately used by businesses to communicate their information within their organization or to disseminate information to the external world from their organization. Searching for this information using deeper search techniques and the latest algorithms allows researchers to obtain a vast amount of corporate information that was previously unavailable or inaccessible. Research has also shown that even deeper information can be obtained from these files by searching and accessing the "properties" information on these files.

ResearchZilla.com: How the Social Internet Simplifies Source Identification

Skip the long and mostly irrelevant preamble to get to the meat of this article, which summarizes and grades Web sites that aggregate and organize information about people.

Updated Online Research Tools by Marcus P. Zillman, M.S., A.M.H.A.

Zillman has updated his Online Research Tools (PDF), "a comprehensive listing of online research tools that offer various downloadable as well as web applications to allow you to do your research and searching on the Internet far more effective and productive."

Scitopia.org

scitopia.org is a search engine devoted to science and technology. It was created by "15 leading science and technology societies":

Searching for a better way to help researchers quickly get to the quality content they need, these society publishers developed a gateway to the research most cited in scholarly work and patents. Scitopia.org searches the entire electronic libraries of the leading voices in major science and technology disciplines and provides relevant results, without the noise of other Internet search engines. More than three million documents, including peer-reviewed journal content and technical conference papers, spanning 150 years of science and technology can be searched through the site.

But to get the full text of the articles you find you'll have to be a member of one of the societies or pay per view.

Finding Old Web Pages

...Search Engine Showdown lists the many places online old Web pages can be found.

Yahoo! Search Assist

... is an upgrade to Yahoo!'s search engine. Here's how the Yahoo! Search Blog introduces it:

Recent research conducted by Harris Interactive* indicates that consumers are suffering from "Web Search fatigue." The study revealed that while the vast majority of the population uses a search engine to find information on the Internet, only 15 percent of people find what they're looking for with their first search. Typically, people need to conduct three to four searches to find the right result.

While search engines can and will continue to improve, we think making queries smarter is also part of the solution. While there are a handful of seasoned users and professional searchers out there who employ Boolean operators, site restrict and other advanced techniques, no one (ourselves included) wants to have to think hard about which search terms to use. Our vision was to build a system that helps users confidently construct more sophisticated and targeted queries without slowing them down.

The Chicago Tribune's Internet critic, Steve Johnson, likes it a lot, saying it's "so impressive I'm going to make it my default searcher."

The Virtual Chase: Web Searching with Advanced Commands

 This tutorial explains how to search specific sites, just the URL, certain file types, use synonyms and more. It covers 5 search engines: Ask, Exalead, Google, Live and Yahoo!

Yahoo! MapMixer

 

... lets you upload the image of any map and overlay it on Yahoo! Maps:

The world is a big place. There are thousands of maps out there that provide unique details about any given destination. MapMixer is a new site that combines those maps with Yahoo! Maps to give you a better view of the world.

It's easy to mix your own map. Upload an image of your map, use our layering tool to align it with Yahoo! Maps and we'll do the rest! Your map will have all the features of Yahoo! Maps (zooming, panning). You can also syndicate it on your own site or blog.

1,070,000 search results, give or take 1,069,741

Matt Waite wrote on his blog the other day that the new Web site he created, Politifact, already had "more than 1 million results in Google" after only two weeks online. That number sounded incredible to me, so I Googled Politifact, and found that Google did indeed report, in the blue bar at the top, that the results of that search were the first 1-100 of more than 1 million:

So I scrolled through the results to see which sites these were, and, when I got to the bottom, saw links for only five actual pages of results:

And when I clicked on the link for page 5, lo and behold, suddenly there were links for only three pages of results on the bottom. And at the top, on the blue bar where just minutes before it had reported 1,070,000 results for Politifact, it reported only 259:

You can try almost any search term and get wildly inconsistent counts each time you search. SearchEngineWatch demonstrated it using the nonsensical term djkfdkjfdkjddfdfdd. And a Frenchman demonstrated it using variations of Chirac and Sarkozy.

It's been obvious for a long time that Google search results are an entirely bogus form of evidence, as this 2004 MediaBistro article pointed out:

Writers crafting trend stories—or, for that matter, profiles, or lifestyle pieces, or reviews, or even news items—are always desperate to prove the popularity of whatever hot new thing they're identifying. They could do some reporting, of course, or find some statistical research, but instead they're technologically smitten, like everyone else. What's a simpler, or faster, way of quantifying a trend than typing a key word or phrase into Google? Type in almost any person, place, or thing, and Google will bounce back to you a neat numerical value that calculates that person, place, or thing's importance to this world. The writer can sit back and let the search engine's brainy algorithms do all the work—and then even pick up some tech-savvy bonus points, too. Google, and not polls or pie charts, has emerged as a journalist's best friend—and best source.

Not that Journalists are alone in this. Judges do it too:

A New York federal judge said a Google search had helped him decide that 24 Hour Fitness should not receive an injunction against a competitor that owned 24hourfitness.com. The judge said a search for "fitness industry" on the Internet revealed more than 1.6 million hits, mainly linking to sites related to physical training and conditioning.

Google Fight even purports to pick winners based on which search terms get the most results.

Google isn't the only purveyor of flaky numbers. When Robert Scoble found similar inconsistent results from MSN and Yahoo, it prompted him to ask, "Why aren’t there any truth in advertising laws for search engines?"

The problem is that we just don't know what's going on behind the scenes, so to cite any number churned out by a search engine as proof of something without knowing how it was generated is specious. It's a particularly egregious form of confirmation bias. The only Web page count we should trust is the one where we've used our own eyeballs to scrutinize each and every page.

After I pointed out the discrepancies, Matt did the search again and updated his blog to show that the Google results "now stands at 687,000."

"Regardless," he added, "the point remains — your Google hit count is a largely meaningless milestone, except to show that the site has spread widely."

Except it doesn't even show that. When I searched for Politifact again today, it showed 446,000:

Tomorrow, who knows?

Fired U.S. Attorney document search

Alberto Gonzales has a return engagement before Congress today, so you may want to check out the DOJ Documentation Project's document search, where you can query the contents of more than 9,000 emails, memos and other documents released as part of the Congressional investigation into the firing of U.S. Attorneys:

This is a tool to search text files of the documents released by the DOJ. Each page of each released document has been scanned and turned into a text (.txt) file - there are over 9,000 such files. Some effort has also been made to clean up these files with common OCR errors, but not all of them have been found, as you will see. This is a work in progress and it is not even close to perfect. Most of the released documents were scanned files (i.e. pictures) of the documents. They have been put through an OCR process which is never perfect. Some corrections have been made and more will be made in the future and you can help.

The site doesn't explain its origins or its motives, as far as I can see, but it does link to images of the original documents on government Web sites, so at least the search results can be verified.

LOUIS: Federal government document repository and search engine

LOUIS, "a project of the Sunlight Foundation, and an effort, to paraphrase Justice Louis Brandeis, to illuminate the workings of the federal government. Our ultimate goal is to create a comprehensive, completely indexed and cross-referenced depository of federal documents from the executive and legislative branches of government." Derek Willis of Washingtonpost.com sees it as part of "The New Competition" for newspapers.

Mining social networks for sources and stories

Way back when, if you wanted to find out where someone worked and where they went to school, who their friends were, what their hobbies were and what kind of music they liked to listen to, you had to knock on a lot of doors, make numerous phone calls, dig through city directories, stitch together documents or sit them down for a long interview. Now people volunteer all of that and more on social networking sites such as MySpace, LinkedIn, Facebook, Friendster and Bebo.

A social networking site is any site that attempts to make it easier for like-minded people to find each other. You typically share personal and professional details about yourself in online profiles, and link to and chat with others who share the same interests or the same circle of friends. There are countless such sites with millions of registered users.

If you think it's just teenagers, think again. There are at least three such sites for doctors, for example, and many more for patients. There are sites for political activists, music fans and people who like to bake. When I searched MySpace for Frankfort, KY, I found multiple pages for people who said they worked for state government. Even politicians now feel obligated to join.

Journalists are mining these sites for sources and stories. Virginia Tech students, for example, wrote about the massacre on LiveJournal and other sites, provoking a virtual feeding frenzy by reporters covering the story. Social networking sites have led to stories about a convicted sex offender from New York chatting up children online, Kentucky kids charged with burglary after sharing a video of a break-in and a Houston police officer who thought it humorous to share photos of dismembered women.

If you haven't already, someday soon you will want to find someone on one of these sites. Here are some ways to do that:

• Use the site's own search engine. Some sites, like MySpace, use Google as their search provider. Others have their own, home-grown tool. Policies vary from site to site, but typically you have the option of making your profile private, so only people you invite can look at it, or public, so anyone at all is welcome. You may not be able to join some sites and view profiles at all, unless you misrepresent yourself, because they're only for certain classes of people, such as students. Read the site's terms of use and consult the appropriate ethic's policy and your own moral compass before you act.

• Use Google or Yahoo's site search. Typing site:myspace .com "Alex Davis" Courier-Journal in Google or Yahoo's search box, for example, will search MySpace for any mentions of Alex Davis and the Courier-Journal. If you try this search, by the way, you'll learn that Alex once trolled MySpace looking for people to talk about a coffeehouse.

• Use a site devoted specifically to finding people on social networking sites. Here are some and how they describe themselves:
  • Wink

    Better than white pages, Wink free people search lets you find people at social networks like MySpace, Bebo, LinkedIn & Friendster, and other online communities. Includes name search plus location, school, work, interests, and more.

  • yoName

    yoName turns your computer into a private detective. Look for anyone you want. You can even look them up by a username or an email address! If they're on any of the big-time networks like MySpace or Facebook, yoName will find them. Look up friends, family, ex-es. Look up yourself and see if someone's impersonating you. Or just have fun and look up celebrities, even if the first five entries for Paris Hilton are all "male, 39, single, in Madison, Wisconsin".

  • ProfileLinker

    ProfileLinker is an innovative web utility that allows you to link your social network profiles in one central location. You can also get message alerts from your favorite social networks, get updates on your friends, search for users across several networks, get your horoscope, weather, sports news and more.

  • UpScoop

    Import your email address book and discover which of your friends are on social networks…

  • Explode

    Explode is a social search tool that lets you find others online irrespective of which network they are on, as well as those running their own sites and blogs. It is a easy way to make connections, group these connections and interact with them either using your Explode profile or your own space somewhere else.

  • Streakr

    Discover, rate and share common interests with other communities around the world.

Pipl, a people search engine mentioned here a few days ago, also searches social networking sites.

The usual cautions apply: You can't assume anything you find is true, and you'll have to find verification elsewhere.

Also keep in mind that your snooping may not go unnoticed: If you search for an email address on yoName, for example, the site sends a message to that person telling them that they've been searched, although it doesn't say by who. There's also StalkerTrack, which helps MySpace users monitor people looking at their profiles. That doesn't mean they'll know your name or why you're looking at their page, but even when people write on Web pages accessible to anyone in the world, they persist in believing their words are somehow private.

Whether they remain so is up to you.

Pipl, a people search engine

Pipl claims to offer "The most comprehensive people search on the web." This is how the search engine answers when it asks itself the question, "What is so different about pipl?":

Pipl's query-engine helps you find deep web pages that cannot be found on regular search engines.

Unlike a typical search-engine, Pipl is designed to retrieve information in real-time from the deep web, our robots are set to interact with searchable databases and extract facts, contact details and other relevant information from personal profiles, member directories, scientific publications, court records and numerous other deep-web sources.

Pipl is not just about finding more results; we are using advanced language-analysis and ranking algorithms to bring you the most relevant bits of information about a person in a single, easy-to-read results page.

I searched my name and among the sites where Pipl found mentions of it, though not necessarily mentions of me, were Yahoo People, WhitePages.com, PeopleData, MySpace, Peoplefinders.com, TheScoop.org, the N&R News Research blog, ZoomInfo, LinkedIn, Blogger, and the SPSS mailing list. Other data sources the site suggests it searches includes Hoovers, icq, the SEC, Amazon, Infospace, Reunion.com, Friendster, Flickr and LexisNexis.

Multiple book search tool

ResearchBuzz writes about a search tool that searches Amazon, Google, and Microsoft Live's book search engines all at once. This tool demonstrates once again that it's always wise to remember that search engines don't return the same results, and therefore you should use more than one when you want to be thorough. When I used the tool to search for my name, Amazon turned up four mentions of me (a book with a chapter by my wife that mentioned me in the acknowledgments; a book about the late activist Anne Braden that cited a profile I wrote of her in the footnotes; a book on ombudsmen that looked at coverage of a police shooting I helped cover; and a history of Scott County, Ky, that mentioned my marriage to my wife, a Scott County native). Google missed two of those, but did turn up a book about Kentucky politics that cited CJ profiles of Kentucky towns. Microsoft Live's book search turned up nothing at all. This shows, if nothing else, that these book search engines can be useful tools for researching someone's background, even if that someone hasn't written a book himself.

cRANKy: Search for the 50-plus crowd

cRANKy says it is "The first age-relevant search engine." The company that created it, Eons, explains:

Are you overwhelmed by too many results when you try to search the Web to find what you’re really looking for? Eons has developed cRANKy, a specialized, age-relevant Web search engine for 50+, which starts you off with the four best Web results we can find, rated and ranked by both the cRANKy editorial team and our members. The more our members participate in rating and reviewing, the better cRANKy gets. If you’re a registered member, cRANKy will save your search history so you can easily navigate back to the Web sites you’ve visited through cRANKy. We will prompt you on your personalized homepage as well as within cRANKy each time you visit a site to remind you to let us know how you like it. Web sites reviewed by the cRANKy editorial team include rich descriptions of the site, tips, and deep links to help you navigate directly to key information. Results marked with the Eons symbol help you locate content on the Eons site related to your search.

With a name like cRANKy, clearly they don't think the more senior among us need to be flattered.

Searchbots

Searchbots lets you build your own "search robot."

A Searchbot is your own personal search robot that continuously searches the Internet trying to find all the best websites it can on your behalf. When you build a Searchbot you give it a personality and then program it's search circuits with all the things you want to find. You can search for websites based on factual information like tags and locations, or more creative ways like colour and the mood you're in. You can even ask your Searchbot a question and it will talk to other Searchbots to find you an answer...

Searchbots.net is an experimental search engine that investigates the use of mythology, personification and game theory as motivational strategies in creating a sustainable search community. Searchbots has a rich history and is unique in that it allows you to search using more "human" and entertaining types of information like colour and mood. If you picked the colour red you might get a website about tomatoes, communism or angry people.

This was too cute for me. You have to go through the tedious process of assembling your robot and giving it a personality before you can send it out in the world. Just give me a search box, please.