Last active
July 18, 2023 12:23
-
-
Save jayspeidell/d10b84b8d3da52df723beacc5b15cb27 to your computer and use it in GitHub Desktop.
Sample script to download Kaggle files
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Info on how to get your api key (kaggle.json) here: https://github.com/Kaggle/kaggle-api#api-credentials | |
!pip install kaggle | |
api_token = {"username":"USERNAME","key":"API_KEY"} | |
import json | |
import zipfile | |
import os | |
with open('/content/.kaggle/kaggle.json', 'w') as file: | |
json.dump(api_token, file) | |
!chmod 600 /content/.kaggle/kaggle.json | |
!kaggle config path -p /content | |
!kaggle competitions download -c jigsaw-toxic-comment-classification-challenge | |
os.chdir('/content/competitions/jigsaw-toxic-comment-classification-challenge') | |
for file in os.listdir(): | |
zip_ref = zipfile.ZipFile(file, 'r') | |
zip_ref.extractall() | |
zip_ref.close() |
Setup and Download dataset
Imports
import json
import zipfile
import os
!pip install kaggle
api_token = {"username":"---Your Username","key":"Your API Key"}
!mkdir -p ~/.kaggle
with open('kaggle.json', 'w') as file:
json.dump(api_token, file)
!cp kaggle.json ~/.kaggle/
!ls ~/.kaggle
!chmod 600 /root/.kaggle/kaggle.json
!kaggle datasets download -d heeraldedhia/groceries-dataset
- The dataset will now be present in the /content/ folder (you can see it using
os.listdir()
)
Further, to extract the dataset,
for file in os.listdir():
if '.zip' in file:
zip_ref = zipfile.ZipFile(file, 'r')
zip_ref.extractall()
zip_ref.close()
- This will also place the files directly inside the
/content/
folder
just set the variables...
#Set the enviroment variables
import os
os.environ['KAGGLE_USERNAME'] = "xxxx"
os.environ['KAGGLE_KEY'] = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
!kaggle datasets download -d iarunava/happy-house-dataset
thanks, It works but when I participate in competition:
import os
os.environ['KAGGLE_USERNAME'] = "xxxx"
os.environ['KAGGLE_KEY'] = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
!kaggle competition download -c xxxxxxxxxx
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
É possível salvar o arquivo de download para usar em uma variável?