Datamob: "Public data put to good use"

Datamob "aims to show, in a very simple way, how public data sources are being used":

Our listings emphasize the connection between data posted by governments and public institutions and the interfaces people are building to explore that data.

It's for anyone who's ever looked at a site like MAPLight.org and wondered, "Where did they get their data?" And for anyone who ever looked at THOMAS and thought, "There's got to be a better way to organize this!"

The creators, Sean Flannagan and Lauren Sperber, say they have two broad goals:

• Encourage governments and public institutions to make more data available in developer-friendly formats like CSV, XML and RDF. Widely accessible public data enables informed civic engagement, and we believe that providing restriction-free data to developers is the best way to promote the technological innovations that will spread knowledge.
• Illuminate the process of creating interfaces, mashups and visualizations for public data, and inspire people to create new ones.

And this is how Sperber explains the name:

Well, the folks at Freebase coined the term "data mob" to describe a group of data-lovers working together to perfect a small portion of Freebase's ambitiously all-encompassing database. As for our Datamob, we hope it'll inspire more institutions with vast reserves of information to put their data out there in accessible formats—and bring together more data mobbers to bring that information to life.

[via]

How to confirm if a public figure lives at an address

Earlier this month a reporter asked NICAR-L, a computer-assisted reporting discussion list, for help with a story that hinged on whether a public figure lived at a particular address. The public figure did not own the home, and the reporter wanted to know how he could confirm the public figure lived there. The reporter had already tried phone directories and voter registration cards and he said he wasn't ready to just knock on the door and ask or set up "a surveillance operation."

Here were the suggestions from other reporters on the list:

• Hire a licensed private investigator to search auto registration records on Autotrack, an electronic public record vendor. Reporters are forbidden to access these records directly because of the Driver's Privacy Protection Act, or DPPA, but private investigators are exempt. (I do not know if the private investigator would be violating the act by obtaining the records for the reporter in this way)
• Utility records, such as water bills
• Automobile property tax records
• Pet license records
• Send the public figure a registered or certified letter, return receipt requested
• Talk to the public figure's letter carrier
• Ancestry.com's record databases
• Circulation records for the reporter's newspaper (Years ago I tried to access classified ad records at the CJ for similar reasons, and was told no)
• Ask a cop source to check auto registration records for you (this is ethically dicey for the reporter and the cop)
• Traffic citations
• Civil and criminal court records
• Marriage licenses
• A resume
• Personnel records, if a public employee
• Financial disclosure forms, if an elected or appointed government official subject to financial disclosure
• Ask his florist
• Google his name

First Lady Hillary Rodham Clinton's Daily Schedules

... are now available online. As The New York Times reports, they were made available in response to a Freedom of Information request and a lawsuit. It will be interesting to see what the blathersphere makes of these. Unfortunately, not only are they PDFs, which makes extracting useful information difficult, but 4,746 of the pages have been censored. Says the Times:

The dry records carry all the emotional punch of a factory worker’s time card, showing where she was for much of her eight years in the White House but telling nothing about what she was saying, thinking or doing.

[via]

The Public Record Research TIPS BOOK

... from Facts on Demand Press is $19.95 and is scheduled to be published at the end of this month:

Learn first hand how to use “Insider Information” for searching for public records at thousands of government public record agencies and web pages. This resource provides the tips and practical knowledge to guide you to the right source and help you become an ultra-efficient searcher.

• Field Guide to Court Record Databases
• How to Evaluate Public Record Vendor
• How to Evaluate Record Search Sites
• Why and When All Criminal Records are Not Created  Equal
• Expanded Coverage of Government Watch Lists, Sanction Lists, and Enforcement Actions
• Know the Location Anomalies that Affect Searching Liens and Assets

Text messages are increasingly being treated as public records

... reports USA Today:

Those supposedly private messages that public officials dash off on their government cellphones to friends and colleagues aren't necessarily private after all.

Courts, lawyers and states are increasingly treating these typed text messages as public documents subject to the same disclosure laws — including the federal Freedom of Information Act — that apply to e-mails and paper records.

"I don't care if it's delivered by carrier pigeon, it's a record," said Charles Davis, executive director of the National Freedom of Information Coalition at the University of Missouri. "If you're using public time or your public office, you're creating public records every time you hit send."

Vote for what you find interesting in the JFK assassination documents

You're on deadline and someone releases a highly newsworthy document with hundreds, maybe thousands of pages. Can you really know what's most important in it in time for publication? No, of course not. Maybe there's an executive summary you can read. Or maybe there are certain passages containing a significant name or event you can pick out while scanning it, so you focus on those. Maybe a source told you what to look for.

Regardless, newspapers should build into their online publishing systems more ways to solicit readers' help.

The Dallas Morning News put recently released JFK assassination documents on its site and invited the public to help it find "something interesting."

But like most newspapers, including ours, they aren't really set up to handle something like this. You'd like to be able to directly comment on the interesting portions of the documents, highlight them and share them with others -- immediately. Instead you're asked to submit a comment to a generic form that strangely asks you to "Vote":

 

It would be better to have something along the lines of what this site did with last year's immigration bill, where anyone could comment on specific passages, and the comments are next to the relevant passages, not submitted into the ether.

If there isn't such a tool freely available out there, there should be. And if you know of one, please tell me about it.

FelonSpy.com: "You need to know who your neighbors are. Especially if they're dangerous criminals"

Gary Swick sent me the link to "FelonSpy," which allows you to "Search for violent criminals in your neighborhood":

Safety starts with good information, even if it ends with you holding a loaded .44 caliber handgun. While FelonSpy.com can't help you get a gun, we can certainly help you figure out which direction to point it in.

...

Our patented Felon Search technology mines data from across the nation, from the web and otherwise, and combines it into a single, easy to use interface. Whether you're checking up on your own neighbors or trying to find out if that hotel you've been eyeing is in a safe place, we can help.

...

Simply type in the desired address, click enter and let your new knowledge be your peace of mind.

All it gives you is a name, age and conviction -- not even a conviction date. So did the 72-year-old man who lives a few blocks from commit his felony assault in his teens and live a clean life since then? Dunno, from this site.

The methodology is also dubious:

FelonSpy.com uses a broad array of databases, each completely legal when used individually. They are probably also legal when used together, but that answer hasn't yet come to us. Our servers compare city, county, state and federal criminal databases with telephone records to pinpoint the location of the person you need to know is living next door to you.

In many cases the public telephone directory has incomplete or insufficient data, so about once a month we get updated address and number records directly from a number of telephone companies and credit reporting agencies (at tremendous cost to us, I should add). Don't ask how we get most of our information, but we do it, and we get our top shelf info from sources who wish to remain anonymous, and we pay a lot of money to insure that we keep getting it.

Their about us page, above an ad for ammunition that resolves to Insulting.com, says "We are former law enforcement officers, information technologists land developers and community leaders, all of whom have given up our posts in pursuit of this noble, sometimes misunderstood quest to label the underbelly of society by their actions."

And this is who they say they track:

We track virtually everyone with a criminal record including sex offenders, ex-cons (felony and misdemeanor), and those guilty of some of the more serious traffic infractions. You have the right to know who your neighbors are. We hope to track persons accused of crimes but acquitted in the future, but at this time we do not have sufficient funding to expand our database that far.

Ask too many questions and who knows, we might be tracking you next.

Clearly, these are people we should take seriously.

UPDATE 2/20: A reader here at the CJ reports getting different results each time he searches on the same address. So do I, which I hadn't noticed before. Try it yourself and see.

He asks, "Is it legitimate?"

I had hoped the sarcasm in my original post would make my point of view clear, but I can't say definitively because I haven't taken the time -- nor do I want to take the time -- to investigate the people it says are criminals to see if they even exist. I will say definitively that you shouldn't waste your time with this site if you're seriously interested in crime in your neighborhood.

If you want a laugh, it may be worth your time. For example, the "Sponsored Links" on its home page include "Prison Dating" and "Free But Crappy Legal Assistance."

Snopes.com, meanwhile, says it is a hoax.

State Voter Registration Verification Web sites

The Department of Defense's Federal Voting Assistance Program maintains a list of State Voter Registration Web sites. Here's the direct link to Kentucky's and to Indiana's. With Kentucky's, if you have someone's first and last name and a date of birth, it will return their home address, their party affiliation, whether they're eligible to vote in the primary, their precinct and the state and federal legislative districts for their elected representatives.

Should government workers be deleting their email?

Governing.com says few governments have a system for managing their e-mail, putting their agencies at legal risk:

Millions of state and local employees in jurisdictions all over the country correspond by e-mail every day without giving much thought to what should happen to the product. They may come to regret that behavior. Not only are records, and history, being lost, but many government lawsuits now turn on what is buried in old e-mail messages. Government policy simply has not kept up with the evolving technology. "At the moment," according to Charles Davis, of the National Freedom of Information Coalition, "everyone is looking up and saying, 'Maybe we ought to be keeping this stuff.'" But few have come up with clear rules governing where and how to keep it.

Kentucky's Department for Libraries and Archives does offer guidelines on retaining state government email. These include "Guidelines for Managing E-mail in Kentucky Government" (PDF) and the "Decision Sequence for Determining E-mail Retention." (PDF) And if you wonder how Kentucky's email systems operate,  there's the Commonwealth Office of Technology's explanation of state email systems.

The Indiana Commission on Public Records offers guidelines (PDF) to Indiana agencies.

The Governing.com article describes a few of the ways government officials skirt the public record laws, including "pinning," which "allows two people to send messages back and forth directly to each other's PDAs, without going through the government computer network."

The article says the legal system is beginning to force governments to come to grips with the problem:

... if there was any doubt about the importance of public e-mail management, it should have disappeared in December 2006, with a change in the Federal Rules of Civil Procedure. Under those rules, state and local governments that become litigants in a federal case will have to produce any electronic information considered relevant to the case. If they can't easily retrieve e-mails because they haven't established an efficient way to store them, it's going to cost a lot in staff time. Employees might have to review millions of e-mails to find which ones deal with the plaintiff.

Bulk Access to Congressional Record, Federal Register and more

Tim O'Reilly reports that Carl Malamud, who is credited with shaming the SEC into making its files freely available on the Web, among other things, is now working to give bulk access to Government Printing Office data, which includes the Congressional Record, the Federal Register, presidential papers, Congressional bills, Congressional hearings and other government documents.

GovernmentDocs.org: A FOIA'd document database

As of this writing this site hasn't even officially launched, but I love the concept:

The goal of the database is to create a central repository of government documents, promoting greater transparency into the inner-workings of our government.

Traditionally, government watchdog groups have either posted FOIA documents on their websites as unsearchable PDFs, or statically highlighted several pages within a document to bolster their findings. This has historically limited the public's access to FOIA documents, and minimizes the opportunities for use by researchers, journalists and citizen reviewers for further research and disclosures. Governmentdocs.org changes that:

• Each and every document goes through an optical character recognition (OCR) process, so that the text of each document is entirely searchable.

• A powerful search engine provides full-text searches and hit highlighting.

• Citizen reviewers can add information to each document page and highlight important findings, allowing for more robust and targeted searches.

• Every page of every document has its own unique URL so that documents can be linked, shared, or posted onto websites.

• The database is a coalition effort, so all of the organizations’ documents will be housed on governmentdocs.org and searches will work across collections.

The participating organizations are Citizens for Responsibility and Ethics in Washington, the Electronic Frontier Foundation, the Project on Government Oversight, Public Citizen and the Sunlight Foundation.

Are expunged court records destroyed?

The BRB Public Records Blog says no:

With limited exceptions, the general rule is that the government does not destroy records.  In the typical scenario, even if the judge orders a set aside, the consumer’s name can still be found by searching the court indexes and the case can still be viewed as a public record. 

National Personnel Records Center Opens more than Six Million New Military Personnel Files

From a press release:

The National Personnel Records Center (NPRC) will open for the first time all of the individual Official Military Personnel Files (OMPFs) of Army, Army Air Corps, Army Air Forces, Navy, Marine Corps and Coast Guard military personnel who served and were discharged, retired or died while in the service, prior to 1946. Collectively, these files comprise more than six million records. This is the second step in the progressive opening of the entire paper and microfiche OMPF collection of over 57 million individual files. Additional military personnel records will be made available to the public each year through 2067 until the entire collection is opened.

These archived files are treasured by family members, historians, researchers, and genealogists. Contained in a typical OMPF are documents outlining all elements of military service, including assignments, evaluations, awards and decorations, education and training, demographic information, some medical information and documented disciplinary actions. Some records also contain photographs of the individual and official correspondence concerning military service.

The press release includes information on how to obtain the records.

Get My FBI File

... is a Web site that promises to help you do just that.

This web site helps you generate the letters you need to send to the FBI to get a copy of your own FBI file. We can help you get your files from other "three-letter agencies" (CIA, NSA, DIA, ...) too. It's quick, it's easy, and best of all, it's free!

There's a companion site called Get Grandpa's FBI File. Here's their frequently asked questions page.

Points to remember when making a FOIA request

... from LLRX.com's FOIA Facts, including: 

When making requests provide as much information about the topic as possible. If you know facts about the subject of your request, you should provide them to the FOIA Office. FOIA people often don’t know very much, if anything, about the subject of the requests to their agencies. If you provide as much information about the subject of your request it will expedite the processing of your request as the search for responsive documents will be that much easier. This is true for almost all agencies, even those that use computerized databases. A computerized database is only as good as the search terms provided to it.

Ways to research foreign entities

... from BRB's Public Records Blog.

Foreign lobbyist search

You can search and download data on lobbyists registered to lobby on behalf of foreign powers at the U.S. Department of Justice's Web site.

The Foreign Agents Registration Act (FARA) was enacted in 1938. FARA is a disclosure statute that requires persons acting as agents of foreign principals in a political or quasi-political capacity to make periodic public disclosure of their relationship with the foreign principal, as well as activities, receipts and disbursements in support of those activities.  Disclosure of the required information facilitates evaluation by the government and the American people of the statements and activities of such persons in light of their function as foreign agents. The FARA Registration Unit of the Counterespionage Section (CES) in the National Security Division (NSD) is responsible for the administration and enforcement of the Act.

The Hill reports that the database links "to substantial documents, such as contracts between lobbyists and foreign governments as well as advocates’ reports listing contacts between them and policymakers."

Free access to historical military records until June 6

Peter Smith of The Courier-Journal points out that Ancestry.com, a genealogy site whose records are also useful for investigative reporters, is offering free access to its vast collection of historical military records until June 6, the anniversary of D-Day:

I've been able to find such things as an image of my grandfather's World War I draft registration card (he never was drafted, but it mentions that his eyes were blue, which I never knew because I've only seen him in black-and-white photos); the history of a great-great-grandfather's Civil War regiment; the Civil War pension card of another ancestor; several references to books mentioning an ancestor who was a captain at Bunker Hill; and a World War II draft card of a great uncle who had already fought in World War I, then gone into vaudeville -- a vocation that isn't surprising considering that he wrote "yes" on the line where the card said, "Telephone."

Anyone who wants to research military records on anyone should check this out before D-Day.

Ancestry.com says its records include "every major U.S. war from the Revolutionary War through Vietnam, including draft registration cards, veterans' gravesites, soldier pension indexes, enlistment records, muster rolls and much more. "

Crowdsourcing the immigration bill

I like this example of soliciting the wisdom of crowds. The truth laid bear has taken a PDF of the federal immigration bill and converted it into a format that makes browsing online easy. The site lets readers link to and comment on specific parts of the bill. "My hope, however, is that by presenting the bill in this form, I will help make the bill more accessible to all, and provide a central spot where commentary, criticism, and suggested improvements can be assembled," the site says. "Who knows --- maybe our erstwhile leaders on Capitol Hill will take notice, and take some of our comments to heart."

The mythical national criminal record check

Reporters, who should know better, too often think that they can do a thorough criminal background check by searching a few online databases. It ain't so. The Virtual Chase reprints an article by a law librarian that explains why: "National Criminal Background Checks: Myths, Realities & Resources."

Anyone who has been asked to conduct a "national criminal background check" knows the sinking feeling that comes from facing the requestor's confident assumption that such a request is reasonable, possible, inexpensive, and fast. When a Google search brings up dozens of hits containing words like comprehensive, instant results, free, and all 50 states, it is easy to see where that confident assumption comes from. Where to start to explain all the caveats, cautions, costs, and prohibitions?