AE5 proxy for public repos

condaproxy.md

Configure AE5 5.x to Use a Proxy for Conda Packages

Miscellaneous:

sudo yum install tmux

export HTTP_PROXY=proxy.aws.att.com:3128

export HTTPS_PROXY=$HTTP_PROXY

curl -O https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

Install Miniconda:

sh Miniconda3-latest-Linux-x86_64.sh
	[installed to ~centos/miniconda3]

source ~/.bashrc

Don't proxy to local repo:

export NO_PROXY=anaconda-test.aws.att.com

conda install -kc https://anaconda-test.aws.att.com/repository/conda/anaconda-enterprise anaconda-enterprise-cli cas-mirror git

Configure Anaconda CLI:

anaconda-enterprise-cli config set sites.master.url https://anaconda-test.aws.att.com/repository/api

anaconda-enterprise-cli config set default_site master

anaconda-enterprise-cli config set ssl_verify false

anaconda-enterprise-cli login
	[anaconda-enterprise / anaconda-enterprise]

Create an extra OPs Center user (optional):

sudo gravity enter

gravity --insecure user create --type=admin [email protected] --password=aeplatform --ops-url=https://gravity-site.kube-system.svc.cluster.local:3009

Sign into OPs Center: https://anaconda-test.aws.att.com:32009

Click "Configuration"

Edit the anaconda-enterprise-anaconda-platform.yml config map and ensure the conda section is as follows (comment existing config):

conda: # Common conda settings for editing sessions and deployments
  channels:
    - defaults
    

  default_channels: # List of channels that should be used for channel 'defaults'
    - https://repo.anaconda.com/pkgs/main
    - https://repo.anaconda.com/pkgs/free
    - https://repo.anaconda.com/pkgs/
    - https://conda.anaconda.org/t/p--fc84ea32-fc30-4c0c-a232-adf5990bea4c/p-att

Create a config file (condarc.secret.txt) for conda proxying:

##### condarc.secret.txt ################

# we'll mount this at /etc/conda/.condarc

proxy_servers:
    http: http://proxy.aws.att.com:3128
    https: https://proxy.aws.att.com:3128

#########################################

Create the kubernetes secret:

anaconda-enterprise-cli spark-config --config /etc/conda/.condarc condarc.secret.txt

Upload the secret to kubernetes:

Note: !!!This will delete any existing custom kubernetes secrets in anaconda-config-files-secret.yaml!!!

sudo kubectl replace -f anaconda-config-files-secret.yaml -n default

Restart (3) pods:

sudo gravity enter

kubectl get pods | grep 'ap-deploy\|ap-workspace\|ap-ui' | cut -d' ' -f1 | xargs kubectl delete pods

Restart project(s) in question:

Sign into UI: https://anaconda-test.aws.att.com

1. Click "Projects ->Your Project ->Session"

2. Stop the current session

3. Start a new session

4. Launch a terminal within JupyterLab

5. Show the conda config

   conda config --show

   (verify proxy config from condarc.secret.txt is set)

Prepare the project

`anaconda-project prepare`

Packages should be resolving and pulling from public Anaconda repositories.

proxy_servers:
    http: http://ams2.sme.zscaler.net:10129
    https: https://ams2.sme.zscaler.net:10129