Skip to content

Instantly share code, notes, and snippets.

@ibnesayeed
Last active August 31, 2016 00:04
Show Gist options
  • Select an option

  • Save ibnesayeed/735fdd02c28999e353a1c45de35e13f9 to your computer and use it in GitHub Desktop.

Select an option

Save ibnesayeed/735fdd02c28999e353a1c45de35e13f9 to your computer and use it in GitHub Desktop.
ArchivesUnleashed Hachathon Application
\documentclass[11pt]{article}
\usepackage{fullpage}
\pagenumbering{gobble}
\title{Archives Unleashed: Web Archive Datahon}
\author{
Sawood Alam \\
Department of Computer Science\\
Old Dominion University, Norfolk, VA - 23529 (USA)\\
\texttt{salam@cs.odu.edu}
}
\date{}
\begin{document}
\maketitle
Internet Archive has the largest number of Mementos, but it is still a small fraction of the Web. The long tail of Web archives increases the coverage, completeness, and temporal coherence. We found that for any archive other than IA, less than 4\% of the queries return any Memento. However, the archives have very little overlap so the aggregated response increases the coverage in an additive manner. An earlier study shows that for successful responses only the top three archives other than IA can generate complete TimeMaps for 52\% queries. This shows the usefulness of the Memento aggregator, but also indicates the need of query routing to only archives that are likely to return good results for the requested URI. To make routing decisions the Memento aggregator needs to know about the holdings of each archive (Archive Profile). I would like to explore the possibilities of tooling around the Memento aggregator and routing, hence access to many different datasets would be preferred.
I am a PhD student at Old Dominion University. I have great interest in Web technologies and Urdu language. My recent research and development activities include Archive Profiler\footnotemark[1] (Python scripts to generate and analyze various archive profiles), MemGator\footnotemark[2] (a cross-platform concurrent Memento aggregator CLI and Server in Go), PreserveMe! Viz\footnotemark[3]\textsuperscript{,}\footnotemark[4] (an interactive live temporal event based archival system network visualizer), a dark replica of Archive-It in ODU\footnotemark[5], Reddit Links archive\footnotemark[6], CDXJ\footnotemark[7]\textsuperscript{,}\footnotemark[8] (an Object Resource Stream serialization format), Dockerization of various archiving tools\footnotemark[9], and contribution to several opensource projects\footnotemark[10].
I attended the Archives Unleashed Hackathon, Canada, in March 2016, where I worked with my colleague Mat Kelly on Interplanetary Wayback\footnotemark[11]. We are working on a short paper about it for TPDL 2016 and a poster/demonstration for JCDL 2016.
I would like to be considered for the travel grant as a PhD student.
\footnotetext[1]{https://github.com/oduwsdl/archive\_profiler}
\footnotetext[2]{https://github.com/oduwsdl/memgator}
\footnotetext[3]{http://www.cs.odu.edu/~salam/preserveme/viz.html?log=log-star.json}
\footnotetext[4]{http://ws-dl.blogspot.com/2013/10/2013-10-23-preserve-me-if-you-can-using.html}
\footnotetext[5]{http://archiveit.prod.odu.edu:8080/ (Password protected)}
\footnotetext[6]{https://github.com/ibnesayeed/reddit-scraper (Data not published yet)}
\footnotetext[7]{http://ws-dl.blogspot.com/2015/09/2015-09-10-cdxj-object-resource-stream.html}
\footnotetext[8]{https://github.com/oduwsdl/ORS/wiki}
\footnotetext[9]{https://hub.docker.com/u/ibnesayeed/}
\footnotetext[10]{https://github.com/ibnesayeed}
\footnotetext[11]{https://github.com/oduwsdl/ipwb}
\end{document}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment