python-gitlab login/password auth using cookies

gl-form.py

import re
import sys

import requests

import gitlab


URL = 'https://gitlab.com'
SIGN_IN_URL = 'https://gitlab.com/users/sign_in'
LOGIN_URL = 'https://gitlab.com/users/sign_in'

session = requests.Session()

sign_in_page = session.get(SIGN_IN_URL).content
for l in sign_in_page.split('\n'):
    m = re.search('name="authenticity_token" value="([^"]+)"', l)
    if m:
        break

token = None
if m:
    token = m.group(1)

if not token:
    print('Unable to find the authenticity token')
    sys.exit(1)

data = {'user[login]': 'login_or_email',
        'user[password]': 'SECRET',
        'authenticity_token': token}
r = session.post(LOGIN_URL, data=data)
if r.status_code != 200:
    print('Failed to log in')
    sys.exit(1)

gl = gitlab.Gitlab(URL, api_version=4, session=session)

I've been using and updating this code for some automation tasks since version Gitlab 12. Here are some notable changes I've had to make as Gitlab is updated:

Gitlab 14.5.2

'profile/personal_access_tokens'

has moved to:

'-/profile/personal_access_tokens'

Gitlab 15.1.3

The authenticity token on the personal_access_tokens page has moved into a meta tag called "csrf-token". The created-personal-access-token field is gone, but I found that you could potentially use BS to pull the token value out of a button with the "title="Copy personal access token". I opted to grab the value from the "new_token" value which was already present in the response:

page_tokens = session.get('/'.join((URL, '-/profile/personal_access_tokens')))
private_token = None
if page_tokens.ok:
  root = bs4.BeautifulSoup(page_tokens.text, "html5lib")
  token = root.find_all("meta", attrs={'name': 'csrf-token'})[0]['content']

  body = {
    "personal_access_token[name]": 'mytoken',
    "personal_access_token[scopes][]": 'api',
    'authenticity_token': token
  }

  response = session.post('/'.join((URL, '-/profile/personal_access_tokens')), data=body)

  if response.ok:
    private_token = response.json()['new_token']

  if not private_token:
    sys.exit(1)
  session.headers.update({'Private-Token': private_token})

  gl = gitlab.Gitlab(URL, api_version=4, session=session)

Hello nice work.

Today in 16.8.1-ce.0, i had to fetch again the CSRF token because the login create a new gitlab session.

So here the process that worked for me :

1. Get a first CSRF in an anon session
2. Try to login
3. Receive a 302 to /
4. Get the CSRF again from the 302 location
5. Use gitlab

In additionnal note, i changed the regex to <meta\s+name="csrf-token"\s+content="([^"]+)". It simply avoid the use of beautiful soup.