End of Term 2016 Dataset

The End of Term 2016 Dataset represents data collected by two collecting institutions. These institutions were the Internet Archive (IA) and the University of North Texas Libraries (UNT). The data is part of the initiative called the End of Term Presidential Web Archive.

Archive Location and Download

The 2016 End of Term archive is located on the eotarchive bucket at EOT-2016.

To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT, WET, and CDX files.

By adding either s3://eotarchive/ or https://eotarchive.s3.amazonaws.com/ to each line, you end up with the s3 and HTTP paths respectively.

File List #Files Total Size
Segments EOT-2016/segment.paths.gz 22  
WARC files EOT-2016/warc.paths.gz 194683 139.3 TB
WAT files EOT-2016/wat.paths.gz 194683 1.7 TB
WET files EOT-2016/wet.paths.gz 194683 330.81 GB
META files EOT-2016/meta.paths.gz 194683 167.95 GB
CDX files EOT-2016/cdx.paths.gz 194683 25.36 GB
URL Index files EOT-2016/eot-index.paths.gz 49 23.1 GB