[UNTESTED] conda-tree-shaker.py

conda-tree-shaker.py

import yaml

import click

from conda.core.index import get_index
from conda.models.channel import Channel
from conda.resolve import Resolve
from conda_mirror.conda_mirror import DEFAULT_PLATFORMS


DEFAULT_CHANNEL_URL = Channel('defaults')
DEFAULT_PLATFORM = 'win-64'  # TODO: Necessary?


@click.command()
@click.argument('specs', nargs=-1, required=True)
@click.option('--upstream-channel', 'channel_url', metavar='<URL>')
@click.option('--platform', 'platform', default=DEFAULT_PLATFORM, type=click.Choice(DEFAULT_PLATFORMS), show_default=True)
@click.option('--target-directory', 'target_dir', default='mirrored', metavar='<TARGET_DIR>', show_default=True)
@click.option('--output', type=click.File(mode='w'), metavar='<YAML>')
def conda_pkg_deps_resolver(specs, channel_url=None, platform=None, target_dir=None, output=None):
    """Resolve deps of conda-pkg, and generate config.yaml.

    Highly encouraging to run this under the same environment.

    - Example:

    $ python conda-tree-shaker.py --output=tf.yaml tensorflow "tensorflow-gpu>=1.13"
    $ conda-mirror --config=tf.yaml
    $ python -m http.server

    - Use:
    $ conda config --add channels http://localhost:8000/mirrored  # TARGET_DIR.
    """
    if not channel_url:
        channel_url = DEFAULT_CHANNEL_URL

    index = get_index(
        channel_urls=(channel_url,),  # XXX: LIMITATION - conda-mirror only receive a one channel.
        platform=platform,
        prepend=False,
    )

    solver = Resolve(index, channels=(channel_url,))
    pkgs = solver.install(specs, returnall=False)
    from pprint import pprint
    pprint(pkgs)

    resolved = []
    for pkg in pkgs:
        resolved.append({
            'name': pkg.name,
            'version': pkg.version,
            'build': pkg.build_string,
        })
    if output:
        config = {
            'blacklist': [{'name': '*'}],
            'whitelist': resolved,
            'platform': platform,
            'upstream_channel': channel_url if channel_url is not DEFAULT_CHANNEL_URL else 'https://repo.anaconda.com/pkgs/main/',
            'target_directory': target_dir,
        }
        output.write(yaml.dump(config))


if __name__ == "__main__":
    conda_pkg_deps_resolver()

requirements.txt

conda-mirror
click
# conda>=4.5
# -e git+https://github.com/conda/[email protected]#egg=conda

Channel('defaults') != 'https://repo.anaconda.com/pkgs/main/'

grab missing file from conda install --download-only SPECS conda/conda#5988

Conda's default channels

Running conda-mirror multiple times covered well for fetching packages (.tar.bz2), but not for generating repodata.json which was overwritten per every requests and not covering previous fetched packages.

• Added (Respect previous downloaded packages and generated repodata.json. vericast/conda-mirror#82)

vericast/conda-mirror#65 (Specifying multiple platforms #65)

vericast/conda-mirror#65 (comment)

At <TARGET_DIR>, conda index works well. (it reads repodata.json, maybe?)

vericast/conda-mirror#47 ([BUG] Problematic implementation of the mirror #47)

conda-mirror can't fetch platform: noarch.. ??!

• Added (Fetching 'noarch' platform. vericast/conda-mirror#81)