TwiBot-22

This is the official repository of TwiBot-22 @ NeurIPS 2022, Datasets and Benchmarks Track. This dataset is collected from the Twitter website before 2022.

Introduction

TwiBot-22 is the largest and most comprehensive Twitter bot detection benchmark to date. Specifically, TwiBot-22 is designed to address the challenges of limited dataset scale, imcomplete graph structure, and low annotation quality in previous datasets. For more details, please refer to the TwiBot-22 paper and statistics.

Dataset Format

Each dataset contains node.json (or tweet.json, user.json, list.json, and hashtag.json for TwiBot-22), label.csv, split.csv and edge.csv (for datasets with graph structure). See here for a detailed description of these files.

How to download TwiBot-22 dataset

TwiBot-22 is available at Google Drive.

Please apply for access by contacting shangbin at cs.washington.edu with your institutional email address and clearly state your institution, your research advisor (if any), and your use case of TwiBot-22.

How to download other datasets

For TwiBot-20, visit the TwiBot-20 github repository.

For other datasets, please visit the Bot Repository.

After downloading these datasets, you can transform them into the 4-file format detailed in "Dataset Format". Alternatively, you can directly download our preprocessed version:

For TwiBot-20, visit the TwiBot-20 github repository, apply for TwiBot-20 access, and there will be a TwiBot-20-Format22.zip in the TwiBot-20 Google Drive link.

For other datasets, you can directly download them from Google Drive. You should adhere to the license of each dataset, the "Content redistribution" section of the Twitter Developer Agreement and Policy, the rules set by the Bot Repository, and only use these datasets for research purposes.

Requirements

• pip: pip install -r requirements.txt
• conda : conda install --yes --file requirements.txt

How to run baselines

1. clone this repo by running git clone https://github.com/LuoUndergradXJTU/TwiBot-22.git
2. make dataset directory mkdir datasets and download datasets to ./datasets
3. change directory to src/{name_of_the_baseline}
4. run experiments under the guidance of corresponding readme.md

Baseline Overview

where - represents the baseline could not scale to TwiBot-22 dataset

Precision

Recall

F1

Test1

Test2

Test3

Citation

Please cite TwiBot-22 if you use the TwiBot-22 dataset or this repository

@inproceedings{fengtwibot,
  title={TwiBot-22: Towards Graph-Based Twitter Bot Detection},
  author={Feng, Shangbin and Tan, Zhaoxuan and Wan, Herun and Wang, Ningnan and Chen, Zilong and Zhang, Binchi and Zheng, Qinghua and Zhang, Wenqian and Lei, Zhenyu and Yang, Shujie and others},
  booktitle={Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track}
}

How to contribute

1. New dataset: convert the original data to the TwiBot-22 defined schema.
2. New baseline: load well-formatted dataset from the dataset directory and define your model.

Welcome PR!

Questions?

Feel free to open issues in this repository! Instead of emails, Github issues are much better at facilitating a conversation between you and our team to address your needs. You can also contact Zhaoxuan Tan through tanzhaoxuan at stu.xjtu.edu.cn.