Skip to content

Instantly share code, notes, and snippets.

@cmj
Last active February 4, 2025 10:59
Show Gist options
  • Save cmj/998f59680e3549e7f181057074eccaa3 to your computer and use it in GitHub Desktop.
Save cmj/998f59680e3549e7f181057074eccaa3 to your computer and use it in GitHub Desktop.
Grab oauth token for use with Nitter (requires Twitter account)
#!/bin/bash
# Grab oauth token for use with Nitter (requires Twitter account).
# results: {"oauth_token":"xxxxxxxxxx-xxxxxxxxx","oauth_token_secret":"xxxxxxxxxxxxxxxxxxxxx"}
# 2024-11-14: verified working again
# 2025-01-07: added 2FA support
username=""
password=""
# Two-Factor Authentication
# You can use any time-based one time password (TOTP) authentication app like Google Authenticator, Authy, Duo Mobile, 1Password, etc.)
# - Wait for prompt, use authentication app to get code.
# - OR enter here your one-time backup code (limit of 5 active codes, must be used in order created; out of order sequence revokes previous codes)
totp_code=""
if [[ -z "$username" || -z "$password" ]]; then
echo "needs username and password"
exit 1
fi
bearer_token='AAAAAAAAAAAAAAAAAAAAAFXzAwAAAAAAMHCxpeSDG1gLNLghVe8d74hl6k4%3DRUMF4xAQLsbeBhTSRrCiQpJtxoGWeyHrDb5te2jpGskWDFW82F'
guest_token=$(curl -s -XPOST https://api.twitter.com/1.1/guest/activate.json -H "Authorization: Bearer ${bearer_token}" -d "grant_type=client_credentials" | jq -r '.guest_token')
base_url='https://api.twitter.com/1.1/onboarding/task.json'
header=(-H "Authorization: Bearer ${bearer_token}" -H "User-Agent: TwitterAndroid/10.21.0-release.0" -H "X-Twitter-Active-User: yes" -H "Content-Type: application/json" -H "X-Guest-Token: ${guest_token}")
# start flow
flow_1=$(curl -si -XPOST "${base_url}?flow_name=login&api_version=1&known_device_token=&sim_country_code=us" "${header[@]}" \
-d '{"flow_token": null, "input_flow_data": {"country_code": null, "flow_context": {"referrer_context": {"referral_details": "utm_source=google-play&utm_medium=organic", "referrer_url": ""}, "start_location": {"location": "deeplink"}}, "requested_variant": null, "target_user_id": 0}}'
)
# get 'att', now needed in headers, and 'flow_token' from flow_1
att=$(sed -En 's/^att: (.*)\r/\1/p' <<< "${flow_1}")
flow_token=$(sed -n '$p' <<< "${flow_1}" | jq -r .flow_token)
# username
token_2=$(curl -s -XPOST "${base_url}" -H "att: ${att}" "${header[@]}" \
-d '{"flow_token": "'"${flow_token}"'", "subtask_inputs": [{"enter_text": {"suggestion_id": null, "text": "'"${username}"'", "link": "next_link"}, "subtask_id": "LoginEnterUserIdentifier"}]}' | jq -r .flow_token)
# password flow and print oauth_token and secret
flow_3=$(curl -s -XPOST "${base_url}" -H "att: ${att}" "${header[@]}" \
-d '{"flow_token": "'"${token_2}"'", "subtask_inputs": [{"enter_password": {"password": "'"${password}"'", "link": "next_link"}, "subtask_id": "LoginEnterPassword"}]}')
token_3=$(jq -r .flow_token <<< "${flow_3}")
check_2fa=$(jq -r .subtasks[0].subtask_id <<< "${flow_3}")
if [[ "${check_2fa}" != "LoginTwoFactorAuthChallenge" ]]; then
jq -c '.subtasks[0]|if(.open_account) then {oauth_token: .open_account.oauth_token, oauth_token_secret: .open_account.oauth_token_secret} else empty end' <<< "${flow_3}"
exit 0
fi
# 2FA stuff
if [[ -z "$totp_code" ]]; then
echo "@${username} - Use your code generator app to generate a code and enter it below."
read totp_code
fi
curl -s -XPOST "${base_url}" -H "att: ${att}" "${header[@]}" \
-d '{"flow_token":"'"${token_3}"'","subtask_inputs":[{"subtask_id":"LoginTwoFactorAuthChallenge","enter_text":{"text":"'"${totp_code}"'","link":"next_link"}}]}' |
jq -c 'if(.subtasks[0].open_account) then {oauth_token: .subtasks[0].open_account.oauth_token, oauth_token_secret: .subtasks[0].open_account.oauth_token_secret} else .errors[0].message end'
@AritzUMA
Copy link

I am using the https://github.com/sekai-soft/guide-nitter-self-hosting configuration that works perfect, but the inly problem that I have is that I can't add more than one account. I get rate limited easly

@cmj
Copy link
Author

cmj commented Sep 19, 2024

They are also using the same process to get old oauth tokens.

https://github.com/sekai-soft/guide-nitter-self-hosting/blob/4404ad05fb99b94c5d02bec2b53bf1076f22dda8/nitter-auth/nitter-auth.py#L149-L150 (I just tested the script; also fails)

I believe this stopped working sometime at the beginning of August, based on various github issues searches.

Here is the latest with some findings in comments:

https://gist.github.com/cmj/041bcc04f1a1ae7624a01ecd72d7a108

You can see it's a successful attempt to finish the loginFlow, but it now fails to return 'open_account' subtask. So from my understanding, you'll no longer be able to obtain oauth_token & oauth_token_secret. Existing tokens you've created will still work (I have some from years ago.)

This is why I've been toying with just using cookie authentication option. I've been running it exclusively for 2 days and seems to be working fine so far. Mostly just been tearing stuff out that's not needed so it will have to be cleaned up at some point.


So, for now, you cannot add anymore new Twitter accounts to Nitter. Unless someone finds the magical formula in getting those tokens, this seems to be the only thing useful you can extract from the loginFlow https://github.com/cmj/nitter/blob/cookie_header/twitter_auth_token.sh

@flikites
Copy link

flikites commented Sep 23, 2024

Thanks @cmj

Gonna test this out in a few.

Update: Working. Thank You!

@cmj
Copy link
Author

cmj commented Sep 25, 2024

@flikites The 'cookie' branch? Glad to hear, I've been using it exclusively for about a week and seems to be fine so far. There are a bunch of backend things that need to be polished but as a PoC it's working well.

Thanks for the feedback!

@flikites
Copy link

@flikites The 'cookie' branch? Glad to hear, I've been using it exclusively for about a week and seems to be fine so far. There are a bunch of backend things that need to be polished but as a PoC it's working well.

Thanks for the feedback!

Yes! The 'cookie_header' branch. Working good on my end as well since I've had it up.

@mrdev1337
Copy link

Thanks so much for the cookie header branch! Is it a big lift to make it work for multiple cookie headers?

@cmj
Copy link
Author

cmj commented Oct 7, 2024

@mrdev1337 Yeah that's ultimately the goal. Obviously the framework is there to handle multiple accounts, I just disabled it all to see if it worked fine with just one.

One would have to refactor in all the ratelimits too as I think they might differ than before. I don't have a comprehensive understanding of account management yet, but it's something I'm willing to look at.

Nitter is dead for new multiple-account installs otherwise.

@cmj
Copy link
Author

cmj commented Nov 14, 2024

Made some modifications and this script seems to be working again.

@muhitrhn
Copy link

muhitrhn commented Jan 6, 2025

What can I do in case the accounts has 2step verficaition enabled?

@cmj
Copy link
Author

cmj commented Jan 7, 2025

I'm not entirely sure that's possible with this onboarding flow for oauth tokens. It is, however, working for cookies using https://github.com/d60/twikit/ You can use that to get a cookie and use it with this cookie branch. Unfortunately it is just a PoC; only one account for now.

I'll poke around later this week.

https://github.com/d60/twikit/blob/main/twikit/client/client.py#L451

@muhitrhn
Copy link

muhitrhn commented Jan 7, 2025

I'm not entirely sure that's possible with this onboarding flow for oauth tokens. It is, however, working for cookies using https://github.com/d60/twikit/ You can use that to get a cookie and use it with this cookie branch. Unfortunately it is just a PoC; only one account for now.

I'll poke around later this week.

https://github.com/d60/twikit/blob/main/twikit/client/client.py#L451

Already using 4 accounts and want to use more accounts to avoid 429 error. Can't use the cookie branch really. Would be best if there's any way to find out flow to authenticate 2fa.

@cmj
Copy link
Author

cmj commented Jan 7, 2025

Already using 4 accounts and want to use more accounts to avoid 429 error. Can't use the cookie branch really. Would be best if there's any way to find out flow to authenticate 2fa.

@muhitrhn I just added 2fa options to this and it seems to work fine using phone with Authy. I haven't tested it with backup codes, but should work. You still get oauth_token and secret with response.

Apparently SMS is not available for free accounts, so using an app seems to be the preferred method...

Let me know if it works, thanks!

@muhitrhn
Copy link

muhitrhn commented Jan 8, 2025

Already using 4 accounts and want to use more accounts to avoid 429 error. Can't use the cookie branch really. Would be best if there's any way to find out flow to authenticate 2fa.

@muhitrhn I just added 2fa options to this and it seems to work fine using phone with Authy. I haven't tested it with backup codes, but should work. You still get oauth_token and secret with response.

Apparently SMS is not available for free accounts, so using an app seems to be the preferred method...

Let me know if it works, thanks!

It worked perfectly. Thank you very much!

@muhitrhn
Copy link

Hi @cmj , it is off topic but is there any graphql endpoint for X spaces from which I can find out the participants of a Space? I found one but it doesn't give all participants which is AudioSpaceById. Is there maybe any which checks if a user is in the space?

@cmj
Copy link
Author

cmj commented Jan 17, 2025

Yeah that endpoint doesn't give every single listener. It breaks them out in 3 parts {admins,speakers,listeners}. I've monitored a number of "spaces" and that seems to have a limit on listeners data, but it does change.

The short of it is, I don't think it's possible to page through all listeners. I don't think twitter had it in place to list all listeners but I'll look again.

I'll add some of my spaces/broadcasts scripts to https://github.com/cmj/twitter-tools/ soon. I'm going to guess it has to do with the thousands that may listen and that number is so insanely dynamic it's not something one can pull proper data from.

E: Sorry I didn't address your last question. No. These are the known endpoints: https://raw.githubusercontent.com/fa0311/TwitterInternalAPIDocument/refs/heads/master/docs/json/API.json

@cmj
Copy link
Author

cmj commented Jan 18, 2025

@muhitrhn I'll have to wait for a large listener group to come live to play with, but for small "spaces" I guess one could do a check.

$ cat 1BdxYEyDoYMxX.json | jq -r '.data.audioSpace.participants.listeners[] | "Listener @\(.twitter_screen_name) (\(.display_name))"' | grep @Renz1337
Listener @Renz1337 (Ellorayni Reynolds)

Which would be this if stream is live:

./AudioSpaceById.sh 1BdxYEyDoYMxX | jq -r '.data.audioSpace.participants.listeners[] | "Listener @\(.twitter_screen_name) (\(.display_name))"' | grep @user

@muhitrhn
Copy link

@cmj Right that's the same problem I am facing. I wonder how those data analytics tools does it. Is there any endpoint which checks if a user joined the space?

@cmj
Copy link
Author

cmj commented Jan 19, 2025

I played around with Spaces a bit more... The only thing that comes close is the AudioSpaceSearch, however it's seems impossible to see if a random user joins the stream to listen. It will give results if you search for the host of a stream and sometimes another admin.

I'm not finding any other endpoint that would return the Space someone is actively listening to.

You could, if you knew a general search phrase (or "space" gave me the most) and passively filter all the results for the user. That's obviously going to use a lot of resources, and you also have a listener limit.

See here: https://gist.github.com/cmj/c8b6964999029c3fdcf21da01a24ec3a?permalink_comment_id=5398547#gistcomment-5398547

@cmj
Copy link
Author

cmj commented Jan 25, 2025

@muhitrhn I stumbled across this endpoint https://x.com/i/api/fleets/v1/fleetline?only_spaces=true
You need to be following the account for you to see them actively listening to a Spaces stream.

@muhitrhn
Copy link

muhitrhn commented Jan 25, 2025

Thank you for finding it @cmj! Really appreciate it. So how can I interact with it? It says 410 Gone: The Twitter REST API v1 is no longer active. Please migrate to API v1.1. https://dev.twitter.com/docs/api/1.1/overview

Edit: Nvm now it says 403 forbidden. I think you can only access it with cookies and csrf? If that's the case can you help me maybe in converting https://github.com/PrivacyDevel/nitter to multi cookies and csrf based? Just telling me what to change will also do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment