Skip to content

Instantly share code, notes, and snippets.

@sagikazarmark

sagikazarmark/README.md

Last active December 6, 2022 15:16
Show Gist options
  • Save sagikazarmark/f23c51295f2f615c5c64e586f2db5bb4 to your computer and use it in GitHub Desktop.
Save sagikazarmark/f23c51295f2f615c5c64e586f2db5bb4 to your computer and use it in GitHub Desktop.
Download ZIP
List all repositories on DockerHub
Raw
README.md

List all repositories on DockerHub

Go to DockerHub, specifically the repository list page.

The easiest way to grab repositories is by sending HTTP requests from the Network page of your inspector.

Look for a request URL like this: https://hub.docker.com/v2/repositories/ORGNAME/?page_size=25&page=1&ordering=last_updated

Edit and resend the request (or choose any other way you see fit) to download lists. The results are paginated, so you will have to adjust the page query variable.

Pro tip: Increase the page_size to 100 (the maximum) to download the list in less pages.

Download the list in multiple JSON files in an empty directory.

You can concatenate the lists using JQ:

jq -s '.[].results' *.json | jq -s 'add'

To export the results as CSV, you can use the following snippet:

jq -s '.[].results | del(.[].media_types, .[].description)' *.json | jq -s 'add' | jq --raw-output '(.[0] | keys), (.[] | [ keys[] as $k | .[$k] ]) | @csv' > list.csv

Note: media_types is removed, because it's an array and can't be converted to CSV. description is removed because in my example it contained quotes and it was easier to remove it than escaping quotes.

Sources

https://qmacro.org/blog/posts/2022/05/19/json-object-values-into-csv-with-jq/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment