|
The biggest barriers to computer-assisted reporting are time and desire -- not money. Chase Davis, Matthew Waite and Derek Willis, who are attending the NICAR conference in Houston this weekend, list the free tools data-minded journalists can use instead of commercial alternatives, some of them costing thousands of dollars. Note that they're sharing the list on Google Docs -- also free (but not open source, as is the other software).
TableTools is a Firefox extension that lets you sort, filter or copy HTML tables on the Web. For example, if you come across a Web table that isn't sorted alphabetically and doesn't give you the option to sort, you can right click on the table, choose the sort option from the TableTools menu, and it sorts the table in place, automagically, without leaving the Web page or having to cut and paste into a spreadsheet or other software. You can also filter Web tables : TableTools adds dropdown menus to the top of a table, allowing you to filter by terms in any of the columns. You also choose to copy the filtered tables as tab-delimited text or HTML to other programs. TableTools can't handle everything, though. The download site notes it has trouble with more complex tables, such as tables that are nested.
Zillman has updated his Online Research Tools (PDF), "a comprehensive listing of online research tools that offer various downloadable as well as web applications to allow you to do your research and searching on the Internet far more effective and productive."
IBM's word processing, presentation and spreadsheet software, called Lotus Symphony, is now free for anyone:
How is Lotus Symphony different?
You're in charge! Lotus Symphony is based on the Open Document Format (ODF) standard-which means you're not locked into proprietary file formats, software licensing agreements and upgrades. Finally, free tools and freedom of choice!
Can Lotus Symphony handle my existing files?
With Lotus Symphony, you can import, edit and save a variety of file formats including Microsoft Office files. You can even export your documents to Adobe® Portable Document Format (PDF). The tools work with computers running both Microsoft Windows and Linux- environments, with support for Apple Macintosh planned for the future.
Joel Spolsky, who's always worth reading if you care about the business of software, uses the occasion to recall some software history and the original Lotus Symphony, "the software equivalent of Gigli."
The personal data mining software I wrote about recently will be back soon, under a new name: Maltego. The creator says he pulled the software offline after receiving emails accusing him of trademark infringement and misusing the terms of service of a social networking site. He's made changes to compensate and promises that version 1.0 of the new app will be available in a couple of weeks and will "totally rock."
Evolution is software that searches online sources, such as social networking sites, Wikipedia, Google Books and phone directories, for names, numbers, email addresses, phrases and Web domains, and graphically displays the links between them. I installed it after reading a Linux.com article that called it "a kick-ass application, just seething with power and potential":
Still don't grok it? Think of the NSA sifting through network traffic, looking for actionable intelligence. Or if that's too conspiracy-minded for your taste, think of trying to find something new and meaningful in the results of a Google search on Paris Hilton. Evolution is kind of like that, but more aggressive in finding results, and a lot more aggressive in trying to make sense of them.
I searched for just my name and it quickly found my telephone number, my email addresses, my wife, open record appeals I had written, this blog and my personal Web site, references to me on other sites, my latitude and longitude and home city, Google Book hits citing my name and more. You can then repeat searches (called "transforms") on entities it finds, exploring potential relationships. It also turned up a lot of information unrelated or only marginally related to me, and of course it's a lot easier to distinguish the relevant from the irrelevant if you're researching yourself. If you start cold on someone you don't know much about, you're going to have to do a lot more leg work to nail down what's meaningful and what isn't. Still, I was impressed. It runs on Windows, Macintosh and Linux. The Linux.com article quoted the creator, Roelof Temmingh, saying he's undecided about what to do with the Evolution, which is free, at least for now, and still in beta: "He said that he needs to make some money from Evolution or it will die," the article said. "He is considering everything from advertising to subscriptions, or selling the GUI and transforms, or selling only the GUI and making the transforms open source, and he is open to other suggestions."
UPDATE: You can no longer download Evolution from its Web site. Linux.com reports that the creator announced he had removed it "due to circumstances outside of my control. I am not sure how long this outage will last, but perhaps it will be permanent." The site now says that if you want to see what Evolution can do, contact evolution@paterva.com.
The blogger at Successful Software says he created a fake program that doesn't run, "Awardmestars," and it won 16 different awards from software download sites. Many Web sites selling software display icons from these download sites proclaiming that their programs have earned a "5-star rating":
The obvious explanation is that some download sites give an award to every piece of software submitted to them. In return they hope that the author will display the award with a link back to them. The back link then potentially increases traffic to their site directly (through clicks on the award link) and indirectly (through improved page rank from the incoming links). The author gets some awards to impress their potential clients and the download site gets additional traffic.
I like some of the comebacks on Slashdot:
He's obviously missing the point. Among all of the software that does nothing, his is clearly the best.
and
You think that's bad, the guy in the next office to mine has a coffee cup that reads "World's Greatest Dad!
Web Worker Daily gives a brief overview of "7 Apps for Online Note-Taking":
If you're like most of us, you deal with piles of unstructured information every day: phone numbers, ideas for later consideration, snippets of information from the web, recipes, phone messages…the list is endless. For the web worker, moving this information into an online notebook can be an attractive proposition. Rather than tie yourself to one computer, or even one operating system, you can get at your notes from anywhere that has a web browser handy.
I've tried a couple of these, such as Google Notebook and Zoho Notebook, but for various reasons found them wanting. My preferred applications for collecting Web flotsam these days are the offline OneNote and ClipMate.
Simile Timeplot is a slick, free widget for "for plotting time series and overlay time-based events over them." Examples at the site include energy prices, housing and stock market trends, immigration and George W. Bush's approval ratings. I haven't tried using it myself, but I have used Simile Timeline, a timeline maker, which uses the same data format, so I'm thinking it's got to be just as worthwhile and just as easy to use.
PC World writes about social network analysis software, called Metron EBA, that does "Enterprise Behavior Analysis":
The appliance sits on the network, takes a snapshot of its users and passively monitors traffic, tracking various modes of communication such as data, voice, e-mail or IM. Metron EBA can then display which employees or groups of employees interact the most and how it relates to business productivity. For instance, one employee may be connected to several groups and serve as an unofficial liaison, unbeknownst to upper management.
...
Metron EBA also could be used to optimize business processes, conveying which groups of people work together and what it makes sense for them to work on at any given time. In addition, the product could indicate if an employee is using different communications or speaking with people outside of the normal behavior -- which could indicate malicious behavior ...
The Green Chameleon calls it a "nasty twist" on the technique's formerly benign image, while Brad Hinton says it won't help recruit and retain staff.
Salon relays the story of how the 8-year-old son of a political scientist uncovered secret documents from the Coalition Provisional Authority, the defunct U.S.-led transitional government in Iraq:
My son made his discovery while impatiently waiting to play a computer game on my laptop. As part of a research project, I had downloaded 45 documents from a section of the CPA Web site known as Consolidated Weekly Reports. All but three of the documents were Microsoft Word. I had one of the Word documents up on my screen when my son starting toying with the computer mouse. Somehow, inadvertently, he managed to pull down the "View" menu at the top of the screen and select the "Mark up" option. If you are in a Word document where "Track changes" has been turned on, hitting "Mark up" will reveal all the deletions and insertions ever made in the document, complete with times, dates and (sometimes) the initials of the editors. When my son did it, all the deleted passages in a document with the innocuous name "Administrator's Weekly Economic Report" suddenly appeared in blue and purple. It was the electronic equivalent of seeing every draft of an author's paper manuscript and all the penciled changes made by the editors. I soon figured out that with a few keystrokes I could see the deleted passages in 20 of the 42 Word documents I'd downloaded. For an academic like myself it was a small treasure trove, and after I'd stopped hooting and hollering it took some time before I could convince my startled son that he hadn't done anything wrong.
Maybe the CPA should have read Microsoft's advice on how to prevent this sort of thing.
Zoho Creator is a free tool for creating Web applications. You use it to create Web forms that allow people to submit data, process the data, then make it searchable in an online database. You can share your applications with the world, or keep them private and make them available to only a few. You can also embed these applications in your own Web site or blog. Typically, this is the sort of thing you'd do with a programming language like PHP and a database like MySQL. Zoho Creator, however, makes it ridiculously easy to do without specialized skills. Zoho Creator does have its own simplified programming language called Deluge ("Data Enriched Language for the Universal Grid Environment") that you can use to customize your creations, but it's not required. As a demonstration, I've created a simple application that allows people to submit interesting Web sites to Depth Reporting, emails the submissions to me and makes them viewable and searchable on Zoho. If you have something interesting to share, please submit it in this form:
Given that only about 120 to 130 people a day get Depth Reporting's feed, I'm not expecting much, but we'll see.
UPDATE: If you are reading this in an RSS reader, you won't be able to see the form, so you'll have to visit the blog page itself. The code doesn't work in feed readers, which I should have checked before posting.
A mind map is one of those things you draw with a felt tip pen on a big pad of paper in a meeting where you're soliciting ideas, not all of them sensible, and trying to link them together. Mindomo is Web-based mind mapping software you can use to organize your thoughts and the thoughts of others. The site boasts: "Think clearly. Get organized. Boost your productivity. Enhance your creativity." I dunno if that's true because I haven't given Mindomo a good workout, but I have found the free Freemind, another mind mapping tool, useful on occasion. There are many similar mindmapping programs for sale, including ConceptDraw MINDMAP Personal, which normally costs $119 but which is being given away free today only. The advantage of Mindomo, presumably, is that you can easily share your mind maps and access them from anywhere. Whether the world is interested in your addled brain, however, is another matter.
Free GeoTools is a blog devoted to "Free (or cheap) GIS/GPS applications, utilities, websites, how-tos." It's been on a roll recently with links to stuff I can see myself using. Most importantly he reports that TGR2SHP, which converts U.S. Census TIGER files to shapefiles and for which I once paid $99, is now free. He wrote about it in a post on "Converting US Census TIGER Data Into Shapefiles For Free." Other posts of interest to mapping geeks include "A Free and Easy Shapefile Coordinate Converter," "Simplifying Line and Polygon Shapefiles," and a series of posts on exporting shapefiles to Google Earth.
... offered by senduit. You upload a file up to 100 megabytes, set a duration you'd like it to remain online (30 minutes to a week) and send the recipient a link.
... explained by Digital Inspiration. I can't vouch for the blog's claim that PDF password-cracking software is legal, but the post does explain some of the ways you can extract text from a PDF that is otherwise locked down.
Given the names involved, this looks like a promising way to add good search to your intranet: "IBM OmniFind Yahoo! Edition is a no-charge, entry-level enterprise search software solution that enables rapid deployment of intranet and file system search for both employees and customers." You can customize it and make it look like your site. Better yet, you don't have to pay for an expensive search appliance, as with Google. There's a Flash video explaining it.
