Twitter: How to archive your following/followers data (usernames, etc)

twitter-archive-following-followers.md

Twitter allows users to download parts of their data, see How to download your Twitter archive.

But what's not included in that data dump is the usernames/handles of the people that you follow or are following you. All you get is account IDs which is just an internal number and so a bit useless when it comes to archival.

Here's a way to get that data (you need to know how to run stuff in the terminal):

1. Go to your Twitter profile in a desktop browser (Firefox or Chrome)
2. Right click on page → Inspect → Network tab
3. Click on the Following link (e.g. https://twitter.com/{yourusername}/following)
4. Scroll through the entire list of following users (e.g. keep pressing page down, space or scroll)
5. In the filter/search bar of the Network tab: type Following? to only get the requests we're interested in
6. Right click on requests → Save All As HAR or Save all as HAR with content; Name it following.har

Then, in a terminal, with jq installed, run:

jq -r '.log.entries[] | .response.content.text | fromjson? | .data.user.result.timeline.timeline.instructions[-1].entries[] | .content.itemContent.user_results.result | values | [.rest_id, .legacy.screen_name, .legacy.name, .legacy.description] | @csv' following.har > following.csv

That will give you the ID, username, display name and description in a CSV file.

If you want to archive all the data that Twitter returns, run this to get a JSON stream:

jq -r '.log.entries[] | .response.content.text | fromjson? | .data.user.result.timeline.timeline.instructions[-1].entries[] | .content.itemContent.user_results.result | values' following.har > following.json

If you want to export your followers too, clear the Network requests and then repeat the steps but use the Followers link, filter requests by Followers? and use different filenames. The rest is the same.

(Tweet about it here: https://twitter.com/niborst/status/1589883662048579584)

Hey, any help with this?

@quiet69 Run the command in WSL, not Windows

Thank you!

Data format seems to have changed a bit and I couldn't be bothered to figure out how to fix the command, so instead I did:

jq -r '.log.entries[] | .response.content.text | fromjson?' twitter.com.har > followers.json

I then used my code editor's search features to replace all the lines that did not contain screen_name. Good enough for me.

When I type in "Following?", I got no requests.

10 nov 2024: In Chrome, when right-clicking on any of the four requests, there's no "Save All As HAR (with content)"

10 nov 2024: In Chrome, when right-clicking on any of the four requests, there's no "Save All As HAR (with content)"

I looked around for the save all, and instead I found these buttons. The rightmost are import and export.

I looked around for the save all, and instead I found these buttons. The rightmost are import and export.

Thank you

getting
jq: error: Could not open file following.har: No such file or directory
when i run the prompt, anyone know why?