proxy-debug-v3.sh

proxy-debug-v3.sh

# The following documents steps I took when following instructions from:
# https://askubuntu.com/a/1512388/426149

#===========================================#
# PART 1 - Test Manual SOCKS5 Configuration #
#===========================================#

# Note that all following steps, are done on the local client, the remote
# just needs to be accessible vis SSH.

# On the client define variables to talk to the remote machine, which will
# serve as the proxy. Running this SSH command will forward traffic sent to
# port 8080 on the local machine to the remote proxy.
REMOTE_USER=joncrall
REMOTE_IP=192.168.222.30
ssh -D 8080 $REMOTE_USER@$REMOTE_IP

__MANUAL_STEPS__="
Open a new terminal
"

# To verify the proxy is working, we can test the following steps:
# On the local machine, run this command to query your WAN ip address
curl ifconfig.me

# use the environment variable method to tell CURL to use a proxy
# This should resolve to the IP address of the remote proxy
http_proxy=socks5h://127.0.0.1:8080 curl ifconfig.me

# I can verify that this works in my case

# As a sanity check, kill the proxy
killall ssh

# Now test a WAN IP query:
http_proxy=socks5h://127.0.0.1:8080 curl ifconfig.me

# This should report
# Failed to connect to 127.0.0.1 port 8080 after 0 ms: Connection refused
# I verify that this is the case on my machine.

# Now restart the proxy
__MANUAL_STEPS__="
Navigate to the terminal where the original ssh session was killed
"
ssh -D 8080 $REMOTE_USER@$REMOTE_IP

# Test this also works in the browser
__MANUAL_STEPS__="

* Open Firefox

* Navigate to https://whatismyipaddress.com and note the IP address, the reported number corresponds to the WAN address of the local client machine.

* Navigate to Settings -> General -> Network Settings -> Settings

* Enable 'Manual proxy configuration'

* Set
    + SOCKS HOST: 127.0.0.1
    + PORT: 8080
    + Enable SOCKS v5

* Navigate to https://whatismyipaddress.com and note the IP address, which should correspond to the proxy server. If the proxy server has a different WAN address, (which in this case it is), then the address shown should be different. In my case this is true, verifying that the manual proxy is working.

* Change the setting back to 'No proxy', refresh https://whatismyipaddress.com and note that the IP address should return to the original one noted on the local client machine. This does work for me.

* Reenable 'Manual proxy configuration'

* Kill the ssh session that maps our local port 8080 to the proxy

* Navigate to https://whatismyipaddress.com, this should fail and say 'The proxy server is refusing connections'. I've verified this is the case.

* Restart the ssh session to reenable the proxy
"

# Everything in this part works as expected. No issues.

#======================================#
# PART 2 - Test PAC Auto Configuration #
#======================================#

# On the client machine.

# Choose a domain that the PAC file will direct to the proxy,
# everything else will use the normal connection
DOMAIN=askubuntu.com
echo "DOMAIN = $DOMAIN"
nslookup "$DOMAIN"
__MANUAL_STEPS__="
NOTE: Set this variable based on the result of nslookup
"
DOMAIN_ADDRESS=172.64.150.156

# ---
# Write the proxy file, note the bash-isms to get the domain right.
echo '
function FindProxyForURL(url, host)
{
    var socks_proxy = "SOCKS 127.0.0.1:8080";
    if (shExpMatch(host, "*.'$DOMAIN'")) {
        return socks_proxy;
    }
    return "DIRECT";
}
' > "$HOME"/proxy.pac
# Display the proxy to check that it was written correctly
cat "$HOME"/proxy.pac
# Assert that the file has the hash we expect for askubuntu.com
echo "6e532f9b0bdaf4c6074d5591e59a11a980c75ae3afd941f020d4adbe8c477fa5 $HOME/proxy.pac" | sha256sum -c
sha256sum "$HOME"/proxy.pac
echo "
The following how the PAC file should be specified as a URI

file://$HOME/proxy.pac
"
# ---

# Set in the network manager point at the PAC file
gsettings set org.gnome.system.proxy autoconfig-url "file://$HOME/proxy.pac"

# Open settings
gnome-control-center

__MANUAL_STEPS__="
Verify that Network -> Network Proxy is configured as:

* Automatic
* Has the correct URI to the PAC file
"

# Can also programatically check this
SYSTEM_PROXY_MODE=$(gsettings get org.gnome.system.proxy mode)
SYSTEM_PROXY_AUTOCONFIG_URI=$(gsettings get org.gnome.system.proxy autoconfig-url)
echo "
SYSTEM_PROXY_MODE=$SYSTEM_PROXY_MODE
SYSTEM_PROXY_AUTOCONFIG_URI=$SYSTEM_PROXY_AUTOCONFIG_URI
"
if [[ "$SYSTEM_PROXY_MODE" != "'auto'" ]]; then
    echo "System Settings Proxy Mode is INCORRECTLY SET!"
elif [[ "$SYSTEM_PROXY_AUTOCONFIG_URI" != "'file://$HOME/proxy.pac'" ]]; then
    echo "System Settings proxy URI is correctly set'"
else
    echo "System Settings Proxy mode and URI look good"
fi


__MANUAL_STEPS__="
* Open Firefox

* Navigate to Settings -> General -> Network Settings -> Settings

* Enable 'Automatic proxy configuration URL'

* Enter the PAC file URI in the text box.

* Close all firefox windows and restart it to ensure a fresh reload

* Open a new terminal on the host and run:
    sudo tcpdump -n net 172.64.150

    Note that only the first 3/4 parts of the ip address are given.
    This should be a prefix of the above value in: DOMAIN_ADDRESS
    This will show the status of connecting to the special domain on the host

* Open a new terminal on the host and run:
    REMOTE_USER=joncrall
    REMOTE_IP=192.168.222.30
    ssh -t $REMOTE_USER@$REMOTE_IP sudo tcpdump -n net 172.64.150

    This will show the status of connecting to the special domain on the proxy

* Navigate to askubuntu.com in firefox.

   This SHOULD show trafic on the proxy server, but for me it is NOT.
   I see traffic on the host.
"


#====================#
# PART 3 - Debugging #
#====================#

__doc__="
This following section doesn't have manual vs automatic steps as clearly
delineated. Read comments for context.

The manual proxy seems to work correctly, but I cannot get the PAC file to
work.

With the tcpdump commands still running on the host and the remote, check
their activity with the following commands:
"

# Connecting to askubuntu with basic curl should cause traffic on the local
# machine.  I've verified this is true.
curl https://www.askubuntu.com

# Using a manual proxy and running the command on a local machine should cause the
# trafic to appear on the remote proxy. THIS IS NOT TRUE!.
http_proxy=socks5h://127.0.0.1:8080 curl https://www.askubuntu.com

# Running the command on the proxy machine should force it to show up
# as traffic on the remote. THIS ALSO DOES NOT WORK!!!
REMOTE_USER=joncrall
REMOTE_IP=192.168.222.30
ssh $REMOTE_USER@$REMOTE_IP curl  https://www.askubuntu.com


# Nuclear option
# Physically moving to the remote machine, and running
sudo tcpdump -n net 172.64.150
curl https://www.askubuntu.com

# AND THAT STILL PRODUCED NO OUTPUT.
# I was able to get some output by opening chrome on the remote machine
# and going to 172.64.150.165 directly, but it didn't work with a domain name.

# On the proxy machine I ran:
nslookup askubuntu.com and noted that it had two addresses:

Non-authoritative answer:
Name:	askubuntu.com
Address: 104.18.37.100
Name:	askubuntu.com
Address: 172.64.150.156

# On the server running
sudo tcpdump -n net 104.18.37
# And then
curl https://www.askubuntu.com
# did correctly produce output, so perhaps I need to monitor both ip addresses

# -------
# Setup for automatic session creation
# I have now started 4 terminals:
# 2 on the local and 2 on the remote, each running:
sudo tcpdump -n net 104.18.37
sudo tcpdump -n net 172.64.150

# Read sudo password into environ
_outvar="SUDO_PASSWORD"
printf "Enter secret %s: " "$_outvar"
oldtty=$(stty -g)
stty -echo
trap 'stty echo' EXIT
read -s "$_outvar"
stty echo
trap - EXIT
echo
stty "$oldtty"

#### to start a tmux session with 4 panes
tmux new-session -d -s my_session_id1 "bash"
tmux send -t my_session_id1 "tmux split-window -h -t 0" Enter
tmux send -t my_session_id1 "tmux split-window -v -t 0" Enter
tmux send -t my_session_id1 "tmux split-window -v -t 2" Enter

# Now send a command to each pane
REMOTE_USER=joncrall
REMOTE_IP=192.168.222.30
tmux select-pane -t 0
tmux send -t my_session_id1 "sudo tcpdump -n net 104.18.37" Enter
tmux send -t my_session_id1 "$SUDO_PASSWORD" Enter
tmux select-pane -t 2
tmux send -t my_session_id1 "sudo tcpdump -n net 172.64.150" Enter
tmux send -t my_session_id1 "$SUDO_PASSWORD" Enter
tmux select-pane -t 1
tmux send -t my_session_id1 "ssh -t $REMOTE_USER@$REMOTE_IP sudo tcpdump -n net 104.18.37" Enter
tmux send -t my_session_id1 "$SUDO_PASSWORD" Enter
tmux select-pane -t 3
tmux send -t my_session_id1 "ssh -t $REMOTE_USER@$REMOTE_IP sudo tcpdump -n net 172.64.150" Enter
tmux send -t my_session_id1 "$SUDO_PASSWORD" Enter
# -------


# Verify that this produces outputs on one of the local tcpdumps
curl https://www.askubuntu.com
# looks good...
#
# Verify that this produces outputs on one of the remote tcpdumps
http_proxy=socks5h://127.0.0.1:8080 curl https://www.askubuntu.com
# BROKEN, this still touches a local tcp dump!
env http_proxy=socks5h://127.0.0.1:8080 HTTPS_PROXY=socks5h://127.0.0.1:8080 ALL_PROXY=socks5h://127.0.0.1:8080 curl https://www.askubuntu.com
# OK!!! This works! Phew, it looks like http_proxy wasnt fully respected...
# References: https://blog.emacsos.com/use-socks5-proxy-in-curl.html

# This was also enough to make it work.
ALL_PROXY=socks5h://127.0.0.1:8080 curl https://www.askubuntu.com

# This was also enough to make it work.
HTTPS_PROXY=socks5h://127.0.0.1:8080 curl https://www.askubuntu.com

# This was NOT enough to make it work.
env http_proxy=socks5h://127.0.0.1:8080 curl https://www.askubuntu.com

# However, our firefox automatic PAC config still is not using the proxy but if
# we switch back to the manual proxy, it does correctly show tcpdump activity
# on the proxy machine.


# Lets double check that our PAC file is parsing URIs correctly
pip install pacparser
python -c "if 1:
    import pacparser
    import pathlib
    proxy_fpath = pathlib.Path('~/proxy.pac').expanduser()
    pacparser.init()
    pacparser.parse_pac(proxy_fpath)
    print(pacparser.find_proxy('http://www.google.com', 'www.google.com'))
    print(pacparser.find_proxy('https://www.askubuntu.com'))
    print(pacparser.find_proxy('http://www.askubuntu.com'))
    print(pacparser.find_proxy('https://askubuntu.com'))
"
# Prints:
#DIRECT
#SOCKS 127.0.0.1:8080
#SOCKS 127.0.0.1:8080
#DIRECT

# This indicates that using the full form http://www.askubuntu.com should be
# recognized by the PAC file but pasting this into the browser still only
# generates traffic on the local host.

# Going to attempt a reboot. System and firefox config are still automatic and
# pointing at the PAC file.

# After restart, reopened the 4 terminals looking at tcpdump
# restarted the proxy port forward in separate terminmal
REMOTE_USER=joncrall
REMOTE_IP=192.168.222.30
ssh -D 8080 $REMOTE_USER@$REMOTE_IP

# Verified that the curl command still generated result on the proxy
ALL_PROXY=socks5h://127.0.0.1:8080 curl https://www.askubuntu.com

# Opened firefox, verified proxy settings were still in place,
# navigated to https://www.askubuntu.com
# BROKEN!!! Agghh! The tcp traffic is still on the local machine.


# Does there need to be special permissions on the pac file? I have:
# (pyenv3.11.2) joncrall@toothbrush:~$ ls -al proxy.pac
# -rw-rw-r-- 1 joncrall joncrall 188 May  7 16:37 proxy.pac

# Going to try editing the PAC file to ALWAYS forward to the proxy.
# Write the proxy file, note the bash-isms to get the domain right.
echo '
function FindProxyForURL(url, host)
{
    var socks_proxy = "SOCKS 127.0.0.1:8080";
    return socks_proxy;
}
' > "$HOME"/proxy.pac

# Quickly test that the syntax is correct:
python -c "if 1:
    import pacparser
    import pathlib
    proxy_fpath = pathlib.Path('~/proxy.pac').expanduser()
    pacparser.init()
    pacparser.parse_pac(proxy_fpath)
    print(pacparser.find_proxy('http://www.google.com', 'www.google.com'))
    print(pacparser.find_proxy('https://www.askubuntu.com'))
    print(pacparser.find_proxy('http://www.askubuntu.com'))
    print(pacparser.find_proxy('https://askubuntu.com'))
"
# Got:
#SOCKS 127.0.0.1:8080
#SOCKS 127.0.0.1:8080
#SOCKS 127.0.0.1:8080
#SOCKS 127.0.0.1:8080
# Looks good.


__more_notes__='
Navigate to firefox settings, and click the reload button next to the pac file.
Disabled system proxy.
Restarted firefox.

Now, navigating to askubuntu DOES cause traffic on the proxy

Re-enabled automatic system proxy.
Firefox still causes traffic on the proxy, chrome causes trafic on the localhost

Modifying the proxy.pac file to use the shExpMatch condition, then clicking reload,
causes the traffic to go back to localhost.

Modifying the condition to:

    if (shExpMatch(host, "*")) {
        return socks_proxy;
    }

DID work!

The "*askubuntu*" pattern also worked. To verify I changed it to
"*askubuntu2*", clicked reload, and it correctly went back to sending tcp
traffic to the local host. So something about the shExpMatch pattern is wrong.

Here is a list of patterns I tried:

Pattern             | Traffic
--------------------+--------
"*askubuntu*"       | proxy
"*.askubuntu*"      | local
"*askubuntu.com*"   | proxy
"*askubuntu.com"    | proxy
".askubuntu.com"    | local
"*.askubuntu.com"   | local
"askubuntu.com"     | proxy
'