Import Python modules from GitHub

README.md

githubimport

Allows you to import Python modules from the top level of a GitHub repository. Basically, golang's import semantics but in Python fashion.

>>> import githubimport
>>> from MineRobber9000.test_modules import blah
>>> blah.foo()
"bar"

githubimport.py

from requests import get
from enum import auto, IntEnum
from importlib.machinery import ModuleSpec
from urllib.parse import urljoin
from os.path import join

class GithubImportState(IntEnum):
    USER = auto()
    REPO = auto()
    FILE = auto()

class GithubImportFinder:
    def find_spec(self, modname, path=None, mod=None):
        package, dot, module = modname.rpartition(".")
        if not dot:
            spec = ModuleSpec(
                modname,
                GithubImportLoader(),
                origin="https://github.com/" + module,
                loader_state=GithubImportState.USER,
                is_package=True)
            spec.submodule_search_locations = []
            return spec
        else:
            user, dot2, repo = package.rpartition(".")
            if not dot2:
                spec = ModuleSpec(
                    modname,
                    GithubImportLoader(),
                    origin="https://github.com/" + "/".join([package, module]),
                    loader_state=GithubImportState.REPO,
                    is_package=True)
                spec.submodule_search_locations = []
                return spec
            path = urljoin("https://github.com",
                           join(user, repo, "raw", "master", module + ".py"))
            return ModuleSpec(
                modname,
                GithubImportLoader(),
                origin=path,
                loader_state=GithubImportState.FILE)

class GithubImportLoader:
    def create_module(self, spec):
        return None  # default semantics

    def exec_module(self, module):
        path = module.__spec__.origin
        if module.__spec__.loader_state != GithubImportState.USER:
            setattr(sys.modules[module.__package__],
                    module.__name__.rpartition(".")[-1], module)
        if module.__spec__.loader_state == GithubImportState.FILE:
            # load the module
            code = get(path)
            if code.status_code != 200:
                print(path, code)
                raise ModuleNotFoundError(f"No module named {module.__name__}")
            code = code.text
            exec(code, module.__dict__)

import sys

FINDER = GithubImportFinder()
sys.meta_path.append(FINDER)

There's actually a python module that allows you to do that: https://github.com/operatorequals/httpimport - supports GitHub/BitBucket/GitLab as well.

@operatorequals check the implementation of sys.meta_path finder!

BTW: It was suggested to include that in built-in python - didn't go as planned: https://lwn.net/Articles/732194/

There's actually a python module that allows you to do that: https://github.com/operatorequals/httpimport - supports GitHub/BitBucket/GitLab as well.

Thanks for mentioning the module señor.

@operatorequals check the implementation of sys.meta_path finder!

I'm using a similar idea in:
https://github.com/operatorequals/httpimport/blob/master/httpimport.py#L345
Yet, I use insert so I can put the Finder as the first element in sys.meta_path so it gets queried first about the module and getting general priority.

There's actually a python module that allows you to do that: https://github.com/operatorequals/httpimport - supports GitHub/BitBucket/GitLab as well.

@dzervas Nice! I actually didn't know about that. I was messing around with the Meta Path Finder stuff and accidentally made this. One of my friends mentioned that it was the same import model Golang used, so I figured I'd post it.

I actually wrote one that specifically allows you to register modules as coming from anywhere, given a package name (so for instance you could define a package as coming from anywhere, even not a GitHub/BitBucket/GitLab/Gitea/whatever), but I realized I could use the package import semantics to create a simple way to import from GitHub.

I'm using a similar idea in: -snip link-
Yet, I use insert so I can put the Finder as the first element in sys.meta_path so it gets queried first about the module and getting general priority.

@operatorequals The reason I can't use sys.path.insert(0,FINDER) is that this library is basically flying blind. It doesn't understand what a GitHub user is, or what a GitHub repo is, it just understands how to get the module source, given a user, repo, and file. If I put my Meta Path Finder first in the list, it would swallow every import statement (because it wouldn't be able to differentiate between a GitHub import and a local one).