"Build Your Own Web Archive" at OSCON

Tomorrow, at the O'Reilly Open Source Convention in Portland, I'll be presenting a session about the Internet Archive's open source web archiving tools. Full details:

Build Your Own Web Archive: archive.org's Open Source Tools to Crawl, Access & Search Web Captures
Gordon Mohr (Internet Archive, Web Group)
11:35am Friday, 07/25/2008
Web Applications
Location: E145

The Internet Archive, with support from other libraries around the world, has helped develop a collection of open source tools in Java to support web archiving. These include the Heritrix archival web crawler, “Wayback” for replaying historic web content, and extensions to Nutch for web archive full-text search. This session will explain the design and capabilities these tools, and quickly demo their use for the creation of a small personal web archive.

Heritrix has been designed for faithful and complete content archiving but has also found use in other web search contexts. Wayback allows URL-based lookup and follow-up browsing of archived web content. Nutch, as applied to archival web crawls, allows Google-style full-text search of web content, including the same content as it changes over time. Together, they provide everything necessary to archive and access accurate historical records of web-published content.

Also: last month James Turner of O'Reilly Media spoke to me in advance of OSCON. You can read or hear the interview at: Gordon Mohr Takes Us Inside the Internet Archives.

Labels: heritrix, internet archive, nutch, nutchwax, open source, oscon, speaking, wayback

«» (1) comments

The Presidio, a land of beauty and danger, "rendered safe"

We heard a loud "boom" at the office this afternoon...

SJMN: World War I mortar shell found in San Francisco Presidio

A San Francisco police bomb squad determined that a suspicious device found in the city's Presidio this morning was a World War I mortar shell, Sgt. Wilfred Williams said.

At about 10:20 a.m., a person walking a dog reported seeing the device in the Presidio area at Inspiration Point off of Arguello Boulevard.

U.S. Park Police responded, deemed the device it to be suspicious and notified San Francisco police.

According to Williams, authorities cleared out the surrounding area as they investigated the device. The bomb squad determined the device was a World War I mortar shell and it was rendered safe at about 2:45 p.m.

Labels: bomb squad, explosions, presidio, ww1

«» (0) comments