Create Gitlab Personal Access Token using curl

gitlab_access_token.md

Create Gitlab Personal Access Token using curl

Prerequisites

• You need a Gitlab server up and running
• You need user credentials for a (admin) user on the Gitlab server
• You need curl and Perl on your server

What does it do?

1. Open the login page of Gitlab to get session cookie
2. Login to Gitlab using username and password to get authenticated session cookie
3. Open the Personal Access Tokens page
4. POSTing the Personal Access Token for to generate a personal access token
5. Scrape the personal access token from the returned HTML page

Code

gitlab_host="http://localhost:8080"
gitlab_user="root"
gitlab_password="12341234"

# 1. curl for the login page to get a session cookie and the sources with the auth tokens
body_header=$(curl -c cookies.txt -i "${gitlab_host}/users/sign_in" -s)

# grep the auth token for the user login for
#   not sure whether another token on the page will work, too - there are 3 of them
csrf_token=$(echo $body_header | perl -ne 'print "$1\n" if /new_user.*?authenticity_token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

# 2. send login credentials with curl, using cookies and token from previous request
curl -b cookies.txt -c cookies.txt -i "${gitlab_host}/users/sign_in" \
	--data "user[login]=${gitlab_user}&user[password]=${gitlab_password}" \
	--data-urlencode "authenticity_token=${csrf_token}"

# 3. send curl GET request to personal access token page to get auth token
body_header=$(curl -H 'user-agent: curl' -b cookies.txt -i "${gitlab_host}/profile/personal_access_tokens" -s)
csrf_token=$(echo $body_header | perl -ne 'print "$1\n" if /authenticity_token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

# 4. curl POST request to send the "generate personal access token form"
#      the response will be a redirect, so we have to follow using `-L`
body_header=$(curl -L -b cookies.txt "${gitlab_host}/profile/personal_access_tokens" \
	--data-urlencode "authenticity_token=${csrf_token}" \
	--data 'personal_access_token[name]=golab-generated&personal_access_token[expires_at]=&personal_access_token[scopes][]=api')

# 5. Scrape the personal access token from the response HTML
personal_access_token=$(echo $body_header | perl -ne 'print "$1\n" if /created-personal-access-token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

Usage of token in API Requests

According to the Gitlab API documentation, you can now use the personal_access_token to make API requests:

curl --header "Private-Token: ${personal_access_token}" https://gitlab.example.com/api/v4/projects

Does anyone tried the python module "python-gitlab" or cmdline tool glab, maybe it's a better choice ?

gitlab_host="https://"
gitlab_user=""
gitlab_password=""

# 1. curl for the login page to get a session cookie and the sources with the auth tokens
body_header=$(curl -k -c cookies.txt -i "${gitlab_host}/users/sign_in" -s)

# grep the auth token for the user login for
#   not sure whether another token on the page will work, too - there are 3 of them
csrf_token=$(echo $body_header | perl -ne 'print "$1\n" if /gl-show-field-errors.*?authenticity_token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

# 2. send login credentials with curl, using cookies and token from previous request
curl -k -b cookies.txt -c cookies.txt -i "${gitlab_host}/users/sign_in" \
	--data "user[login]=${gitlab_user}&user[password]=${gitlab_password}" \
	--data-urlencode "authenticity_token=${csrf_token}"

# 3. send curl GET request to personal access token page to get auth token
body_header=$(curl -k -H 'user-agent: curl' -b cookies.txt -i "${gitlab_host}-/user_settings/personal_access_tokens" -s)
csrf_token=$(echo $body_header | perl -ne 'print "$1\n" if /csrf-token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

# 4. curl POST request to send the "generate personal access token form"
#      the response will be a redirect, so we have to follow using `-L`
body_header=$(curl -k -L -b cookies.txt "${gitlab_host}-/user_settings/personal_access_tokens" \
	--data-urlencode "authenticity_token=${csrf_token}" \
	--data 'personal_access_token[name]=golab-generated&personal_access_token[expires_at]=&personal_access_token[scopes][]=api')

# 5. Scrape the personal access token from the response HTML
personal_access_token=$(echo $body_header | perl -ne 'print "$1\n" if /new_token"[[:blank:]]value="(.+?)"/' | sed -n 1p)

version GitLab Community Edition [16.11]

I wanted similar thing (i.e automation of PAT token generation) but for github enterprise server (aka on-prem github) .

Referring this script , I was able to login to our github enterprise server using username, password , setting cookies,header & this authenticity_token etc .
I was also able to generate PAT token of logged in user by firing POST curl -L(L for redirect) on /settings/tokens endpoint , however I am not able to derive its actual value (the one of format ghp_.........) , as HTML response of curl -L /settings/tokens endpoint doesn't contain newly generated PAT.
PS : New PAT is actually getting created , as I do see its name when refreshing https://<my_server>/settings/tokens in browser. Verified by seeing newly generated token's name [not the value , as value is one time watch, you can't see afterwards]. So yeah token was generated through above curl call

To debug this further (i.e why HTML response of curl -L /settings/tokens endpoint doesn't contain newly generated PAT) , Executed this flow in browser manually & observed request calls in dev tools. Noticed here that , when PAT gets generated , its actual value (i.e ghp_.....) gets displayed on page (for one time) & checking HTML response of /settings/tokens endpoint , I do see it contains generated PAT token's value. Also with these I also do see some js/css calls being fired in network request pane simultaneously before new PAT was generated.
So It seems github is doing something extra through css/js to feed this information ( in HTML response (of /settings/tokens endpoint) later & this makes automation of PAT token generation for github difficult (until we use some broswer mocking tools like selenium)

gitlab_host="https://"
gitlab_user=""
gitlab_password=""

# 1. curl for the login page to get a session cookie and the sources with the auth tokens
body_header=$(curl -k -c cookies.txt -i "${gitlab_host}/users/sign_in" -s)

# grep the auth token for the user login for
#   not sure whether another token on the page will work, too - there are 3 of them
csrf_token=$(echo "$body_header" | grep -o 'name="authenticity_token" value="[^"]*"' | head -n1 | sed 's/.*value="\([^"]*\)".*/\1/')
echo "$csrf_token"

# 2. send login credentials with curl, using cookies and token from previous request
curl -k -b cookies.txt -c cookies.txt -i "${gitlab_host}/users/sign_in" \
	--data-urlencode "user[login]=${gitlab_user}" \
	--data-urlencode "user[password]=${gitlab_password}" \
	--data-urlencode "authenticity_token=${csrf_token}"

# 3. send curl GET request to personal access token page to get auth token
body_header=$(curl -k -H 'user-agent: curl' -b cookies.txt -i "${gitlab_host}/-/user_settings/personal_access_tokens" -s)
echo "$body_header"
csrf_token=$(echo "$body_header" | grep -o 'name="csrf-token" content="[^"]*"' | head -n1 | sed 's/.*content="\([^"]*\)".*/\1/')
echo "$csrf_token"

# 4. curl POST request to send the "generate personal access token form"
#      the response will be a redirect, so we have to follow using `-L`
body_header=$(curl -k -L -b cookies.txt "${gitlab_host}/-/user_settings/personal_access_tokens" \
    --request POST \
    --header "Content-Type: application/x-www-form-urlencoded" \
    --header "X-CSRF-Token: ${csrf_token}" \
    --data-urlencode "authenticity_token=${csrf_token}" \
    --data-urlencode "personal_access_token[name]=golab-generated" \
    --data-urlencode "personal_access_token[expires_at]=" \
    --data-urlencode "personal_access_token[scopes][]=api")

# 5. Scrape the personal access token from the response HTML
personal_access_token=$(echo "$body_header" | grep -o '"new_token":"[^"]*"' | sed 's/"new_token":"\([^"]*\)"/\1/')

version GitLab Community Edition [17.5]