python-gitlab login/password auth using cookies

gl-form.py

import re
import sys

import requests

import gitlab


URL = 'https://gitlab.com'
SIGN_IN_URL = 'https://gitlab.com/users/sign_in'
LOGIN_URL = 'https://gitlab.com/users/sign_in'

session = requests.Session()

sign_in_page = session.get(SIGN_IN_URL).content
for l in sign_in_page.split('\n'):
    m = re.search('name="authenticity_token" value="([^"]+)"', l)
    if m:
        break

token = None
if m:
    token = m.group(1)

if not token:
    print('Unable to find the authenticity token')
    sys.exit(1)

data = {'user[login]': 'login_or_email',
        'user[password]': 'SECRET',
        'authenticity_token': token}
r = session.post(LOGIN_URL, data=data)
if r.status_code != 200:
    print('Failed to log in')
    sys.exit(1)

gl = gitlab.Gitlab(URL, api_version=4, session=session)

Could you please add an open-source license for this gist?

Thanks for the gist!

Unfortunately non-GET requests are forbidden when using session cookie. But having the session cookie we can continue the hack to get a private access token:

page_tokens = session.get('/'.join((URL, 'profile/personal_access_tokens')))
private_token = None
if page_tokens.ok:
    root = bs4.BeautifulSoup(page_tokens.text, "html5lib")
    token = root.find_all("form", id='new_personal_access_token')[0].find_all('input', attrs={'name': 'authenticity_token'})[0]['value']

    body = {
      "personal_access_token[name]": 'mytoken',
      "personal_access_token[scopes][]": 'api',
      'authenticity_token': token
    }

    response = session.post('/'.join((URL, 'profile/personal_access_tokens')), data=body)

    if response.ok:
        private_token_page = bs4.BeautifulSoup(response.text, "html5lib")
        private_token = private_token_page.find_all('input', id='created-personal-access-token')[0]['value']

if not private_token:
    sys.exit(1)
session.headers.update({'Private-Token': private_token})

gl = gitlab.Gitlab(URL, api_version=4, session=session)

Remarks:

• I used BeautifoulSoup to help parsing HTML pages
• Duplicated names are allowed for access token name - but it will be better to avoid regenerating a new token every time you call the script.

I just came across this gist via the link in the docs: https://python-gitlab.readthedocs.io/en/stable/api-usage.html#note-on-password-authentication

I needed to authenticate via LDAP instead of the regular user/password and the example just needed a couple of tweaks to make it work:
The login url needs to use the ldap one (SIGN_IN_URL remains the same):

LOGIN_URL = 'https://gitlab.com//users/auth/ldapmain/callback'

The key names are slightly different for ldap:

data = {'username': 'login_or_email',
        'password': 'SECRET',
        'authenticity_token': token}

if you get the error "bytes expected not str", apply this fix

for l in sign_in_page.decode().split('\n'):

if you get the error "bytes expected not str", apply this fix

for l in sign_in_page.decode().split('\n'):

Or

sign_in_page = session.get(SIGN_IN_URL).text

which uses requests' encoding detection mechanism.

@MikeTuffy thank you so much matte! Exactly what I needed.

Could you please add an open-source license for this gist?

Code with no particular license is covered by Githubs Terms of Service:
https://docs.github.com/en/github/site-policy/github-terms-of-service#d-user-generated-content

So fork and relicense, or fork and continue with the ToS as license.

@MikeTuffy Thank you for your ldap Post! It works! :-)

This procedure no longer works. Instead of a sign-in page, I get a page titled "Checking your Browser - GitLab" that does not contain an authentication token.

I've been using and updating this code for some automation tasks since version Gitlab 12. Here are some notable changes I've had to make as Gitlab is updated:

Gitlab 14.5.2

'profile/personal_access_tokens'

has moved to:

'-/profile/personal_access_tokens'

Gitlab 15.1.3

The authenticity token on the personal_access_tokens page has moved into a meta tag called "csrf-token". The created-personal-access-token field is gone, but I found that you could potentially use BS to pull the token value out of a button with the "title="Copy personal access token". I opted to grab the value from the "new_token" value which was already present in the response:

page_tokens = session.get('/'.join((URL, '-/profile/personal_access_tokens')))
private_token = None
if page_tokens.ok:
  root = bs4.BeautifulSoup(page_tokens.text, "html5lib")
  token = root.find_all("meta", attrs={'name': 'csrf-token'})[0]['content']

  body = {
    "personal_access_token[name]": 'mytoken',
    "personal_access_token[scopes][]": 'api',
    'authenticity_token': token
  }

  response = session.post('/'.join((URL, '-/profile/personal_access_tokens')), data=body)

  if response.ok:
    private_token = response.json()['new_token']

  if not private_token:
    sys.exit(1)
  session.headers.update({'Private-Token': private_token})

  gl = gitlab.Gitlab(URL, api_version=4, session=session)

Hello nice work.

Today in 16.8.1-ce.0, i had to fetch again the CSRF token because the login create a new gitlab session.

So here the process that worked for me :

1. Get a first CSRF in an anon session
2. Try to login
3. Receive a 302 to /
4. Get the CSRF again from the 302 location
5. Use gitlab

In additionnal note, i changed the regex to <meta\s+name="csrf-token"\s+content="([^"]+)". It simply avoid the use of beautiful soup.