Skip to content

Instantly share code, notes, and snippets.

View bretth's full-sized avatar

Brett Haydon bretth

  • Sydney, Australia
View GitHub Profile
@vznncv
vznncv / paramiko_sftp_large_file_downloading_demo.py
Created January 18, 2020 09:26
Workaround to download a large file with paramiko from a sftp server
"""
The script contains example of the paramiko usage for large file downloading.
It implements :func:`download` with limited number of concurrent requests to server, whereas
paramiko implementation of the :meth:`paramiko.SFTPClient.getfo` send read requests without
limitations, that can cause problems if large file is being downloaded.
"""
import logging
import os
"""
Django ORM Optimization Tips
Caveats:
* Only use optimizations that obfuscate the code if you need to.
* Not all of these tips are hard and fast rules.
* Use your judgement to determine what improvements are appropriate for your code.
"""
# ---------------------------------------------------------------------------
@billcrook
billcrook / SFTP.py
Created April 9, 2018 15:56
An SFTP util class I created for use in my airflow pipelines. Not beautiful, but it works.
import logging
import pexpect
from airflow.hooks.base_hook import BaseHook
class SFTP(object):
"""
Requires openssh_client. Spawns process to execute sftp command.
"""