Twitter, Inc.

COLLECTED BY

Organization: Archive Team

Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.

History is littered with hundreds of conflicts over the future of a community, group, location or business that were "resolved" when one of the parties stepped ahead and destroyed what was there. With the original point of contention destroyed, the debates would fall to the wayside. Archive Team believes that by duplicated condemned data, the conversation and debate can continue, as well as the richness and insight gained by keeping the materials. Our projects have ranged in size from a single volunteer downloading the data to a small-but-critical site, to over 100 volunteers stepping forward to acquire terabytes of user-created data to save for future generations.

The main site for Archive Team is at archiveteam.org and contains up to the date information on various projects, manifestos, plans and walkthroughs.

This collection contains the output of many Archive Team projects, both ongoing and completed. Thanks to the generous providing of disk space by the Internet Archive, multi-terabyte datasets can be made available, as well as in use by the Wayback Machine, providing a path back to lost websites and work.

Our collection has grown to the point of having sub-collections for the type of data we acquire. If you are seeking to browse the contents of these collections, the Wayback Machine is the best first stop. Otherwise, you are free to dig into the stacks to see what you may find.

The Archive Team Panic Downloads are full pulldowns of currently extant websites, meant to serve as emergency backups for needed sites that are in danger of closing, or which will be missed dearly if suddenly lost due to hard drive crashes or server failures.

Collection: ArchiveBot: The Archive Team Crowdsourced Crawler

ArchiveBot is an IRC bot designed to automate the archival of smaller websites (e.g. up to a few hundred thousand URLs). You give it a URL to start at, and it grabs all content under that URL, records it in a WARC, and then uploads that WARC to ArchiveTeam servers for eventual injection into the Internet Archive (or other archive sites).

To use ArchiveBot, drop by #archivebot on EFNet. To interact with ArchiveBot, you issue commands by typing it into the channel. Note you will need channel operator permissions in order to issue archiving jobs. The dashboard shows the sites being downloaded currently.

There is a dashboard running for the archivebot process at http://www.archivebot.com.

ArchiveBot's source code can be found at https://github.com/ArchiveTeam/ArchiveBot.

TIMESTAMPS

The Wayback Machine - https://web.archive.org/web/20200925043705/https://github.com/twitter

Repositories

finatra

Fast, testable, Scala services built on TwitterServer and Finagle

testing http microservices framework scala slf4j thrift

Scala Apache-2.0 375 2,029 6 (2 issues need help) 4 Updated Sep 25, 2020
twitter-server

Twitter-Server defines a template from which services at Twitter are built

finagle finatra

Scala Apache-2.0 190 1,132 0 0 Updated Sep 25, 2020
finagle

A fault tolerant, protocol-agnostic RPC system

mysql java redis http memcached distributed-systems scala

Scala Apache-2.0 1,362 7,724 40 (16 issues need help) 1 Updated Sep 25, 2020
scrooge

A Thrift parser/generator

android java scala lua thrift cocoa code-generation

Scala Apache-2.0 236 713 24 (6 issues need help) 0 Updated Sep 25, 2020
util

Wonderful reusable code from Twitter

utility scala finagle

Scala Apache-2.0 528 2,398 4 (1 issue needs help) 1 Updated Sep 25, 2020
dodo

The Twitter OSS Project Builder

sbt util finagle scrooge twitter-server finatra twitter-oss

Shell Apache-2.0 8 20 0 0 Updated Sep 24, 2020
rustcommon

Common Twitter Rust lib

Rust Apache-2.0 1 7 0 0 Updated Sep 24, 2020
chill

Scala extensions for the Kryo serialization library

Scala Apache-2.0 112 552 42 5 Updated Sep 24, 2020
algebird

Abstract Algebra for Scala

Scala Apache-2.0 297 2,001 70 24 Updated Sep 24, 2020
bijection

Reversible conversions between types

Scala Apache-2.0 101 606 29 5 Updated Sep 24, 2020
rezolus

Systems performance telemetry

Rust 46 861 10 1 Updated Sep 23, 2020
rpc-perf

A tool for benchmarking RPC services

Rust 48 351 6 0 Updated Sep 23, 2020
dict_minimize

Access scipy optimizers from your favorite deep learning framework.

scipy scipy-stack

Python Apache-2.0 0 38 0 0 Updated Sep 24, 2020
twitter-cldr-rb

Ruby implementation of the ICU (International Components for Unicode) that uses the Common Locale Data Repository to format dates, plurals, and more.

Ruby Apache-2.0 73 616 4 2 Updated Sep 22, 2020
twemproxy

A fast, light-weight proxy for memcached and redis

C Apache-2.0 1,808 10,141 156 58 Updated Sep 18, 2020
hadoop-lzo

Refactored version of code.google.com/hadoop-gpl-compression for hadoop 0.20

Shell GPL-3.0 300 513 22 14 Updated Sep 14, 2020
twurl

OAuth-enabled curl for the Twitter API

ruby cli twitter twitter-api curl twurl

Ruby MIT 280 1,461 7 0 Updated Sep 11, 2020
twemoji-parser

A simple library for identifying emoji entities within a string in order to render them as Twemoji.

JavaScript MIT 13 60 1 7 Updated Sep 11, 2020
scalding

A Scala API for Cascading

Scala Apache-2.0 683 3,251 281 32 Updated Sep 11, 2020
sqrl

A Safe, Stateful Rules Language for Event Streams

TypeScript 5 33 7 (2 issues need help) 5 Updated Sep 11, 2020
cli-guide.js

CLI Guide JQuery Plugin

JavaScript MIT 12 41 0 1 Updated Sep 11, 2020
hogan.js

A compiler for the Mustache templating language

JavaScript Apache-2.0 448 5,059 39 20 Updated Sep 9, 2020
labella.js

Placing labels on a timeline without overlap.

JavaScript 133 3,715 4 7 Updated Sep 7, 2020
pelikan

Pelikan is Twitter's unified cache backend

C 71 599 24 (2 issues need help) 9 Updated Sep 6, 2020
SentenTree

A novel text visualization technique

JavaScript 29 141 5 8 Updated Sep 4, 2020
pants
Forked from pantsbuild/pants
Pants Build System

Python Apache-2.0 375 74 0 4 Updated Sep 4, 2020
d3kit

D3Kit is a set tools to speed D3 related project development

JavaScript MIT 41 401 2 2 Updated Sep 4, 2020
twemoji

Emoji for everyone. https://twemoji.twitter.com/

emoji twemoji

HTML 1,339 11,706 38 19 Updated Sep 4, 2020
repo-scaffolding

Tools for creating repos based on open source standards and best practices

Python 7 18 2 1 Updated Aug 27, 2020
ccommon

Cache Commons

C 27 80 15 2 Updated Aug 22, 2020