A step-by-step guide how to use Python with Tor and Privoxy

TorPrivoxyPython.md

A step-by-step guide how to use Python with Tor and Privoxy

Tested on Ubuntu 18.04 Docker container. The Dockerfile is a single line FROM ubuntu:18.04. Alternatively, you can simply run docker run -it ubuntu:18.04 bash.

NOTE: stopping services didn't work for me for some reason. That's why there is kill $(pidof <service name>) after each failed service <service name> stop to kill it.

References

This guide is basically a compilation of all the resources listed below.

• How to request new TOR identity in terminal
• How to anonymize the programs from your terminal with torify
• How To: Using Tor From The Command Line
• How to change Tor identity in Python?
• Make requests using Python over Tor
• Crawling anonymously with Tor in Python
  • Alternative link (Gist) for "Crawling anonymously with Tor in Python"
• Tor IP changing and web scraping

Related

• Other awesome open-source projects
  • Tor and Privoxy (web proxy configured to route through tor) Docker container
  • Rotating TOR proxy with Docker
• Mine
  • A step-by-step guide how to use Tor without Authentication
  • TorIpChanger - Python powered way to get a unique Tor IP

Steps

1. Install and check Tor status

root@75f6721089f2:/# apt update
root@75f6721089f2:/# apt install -y tor
root@75f6721089f2:/# service tor status
 * cannot read PID file /var/run/tor/tor.pid

2. Start Tor and check it's running

root@75f6721089f2:/# service tor start
 * Starting tor daemon...          [ OK ] 
root@75f6721089f2:/# service tor status
 * tor is running

3. Try to Authenticate with nc (Netcat)

It's not possible to connect as ControlPort is not set yet.

root@75f6721089f2:/# apt install -y netcat
root@75f6721089f2:/# echo -e 'AUTHENTICATE' | nc 127.0.0.1 9051
(UNKNOWN) [127.0.0.1] 9051 (?) : Connection refused

4. Stop/kill Tor, set ControlPort and start Tor again

root@75f6721089f2:/# service tor stop
 * Stopping tor daemon...          [fail]
root@75f6721089f2:/#  kill $(pidof tor)
root@75f6721089f2:/# service tor status
 * tor is not running
root@75f6721089f2:/# echo "ControlPort 9051" >> /etc/tor/torrc
root@75f6721089f2:/# service tor start 
 * Starting tor daemon...          [ OK ] 

5. Try to Authenticate with nc again

It's possible to connect but Authentication fails.

root@75f6721089f2:/# echo -e 'AUTHENTICATE' | nc 127.0.0.1 9051
515 Authentication failed: Wrong length on authentication cookie.

6. Stop/kill Tor, set and check HashedControlPassword then start Tor again

Make sure that you have something like HashedControlPassword 16:ED2893D8EC97801C60DF4A72249CBCCD8B97B3B01A15C923DC49A0E500 (actual password hash can/will differ) in /etc/tor/torrc.

root@75f6721089f2:/# service tor stop
 * Stopping tor daemon...          [fail]
root@75f6721089f2:/# kill $(pidof tor)
root@75f6721089f2:/# echo HashedControlPassword $(tor --hash-password "my password" | tail -n 1) >> /etc/tor/torrc
root@75f6721089f2:/# tail -n 2 /etc/tor/torrc
ControlPort 9051
HashedControlPassword 16:ED2893D8EC97801C60DF4A72249CBCCD8B97B3B01A15C923DC49A0E500
root@75f6721089f2:/# service tor start 
 * Starting tor daemon...          [ OK ] 

7. Try to Authenticate with nc again

Authentication passes with a correct password.

# NOTE Use Ctrl+C to exit.
root@75f6721089f2:/# echo -e 'AUTHENTICATE' | nc 127.0.0.1 9051
515 Authentication failed: Password did not match HashedControlPassword *or* authentication cookie.
root@75f6721089f2:/# echo -e 'AUTHENTICATE "my password"' | nc 127.0.0.1 9051
250 OK

8. Check your public IP and currently used Tor ip

root@75f6721089f2:/# apt install -y curl
root@75f6721089f2:/# curl http://icanhazip.com/
89.196.159.79
root@75f6721089f2:/# torify curl http://icanhazip.com/
185.220.101.17

9. Change and check Tor IP

root@75f6721089f2:/# echo -e 'AUTHENTICATE "my password"\r\nsignal NEWNYM\r\nQUIT' | nc 127.0.0.1 9051
250 OK
250 OK
250 closing connection
root@75f6721089f2:/# torify curl http://icanhazip.com/
185.220.101.6

10. Change (with Python3) and check Tor IP

root@75f6721089f2:/# apt install -y python3 python3-pip
root@75f6721089f2:/# pip3 install stem
root@75f6721089f2:/# python3

>>> from stem import Signal
>>> from stem.control import Controller
>>> 
>>> with Controller.from_port(port=9051) as controller:
...     controller.authenticate()
...     controller.signal(Signal.NEWNYM)
... 
>>> 

root@75f6721089f2:/# torify curl http://icanhazip.com/
185.107.81.233

11. Install privoxy and check traffic is routed through Tor

Now that it's clear Tor is configured and works properly we can include privoxy to the loop.

root@75f6721089f2:/# apt install -y privoxy
root@75f6721089f2:/# service privoxy status
 * privoxy is not running
root@75f6721089f2:/# echo "forward-socks5t / 127.0.0.1:9050 ." >> /etc/privoxy/config
root@75f6721089f2:/# service privoxy start
 * Starting filtering proxy server privoxy          [ fail ]

Privoxy is unable to start, lets inspect logs.

root@ff788ea93ee0:/# cat /var/log/privoxy/logfile
2019-07-21 09:16:58.278 7f9c2c0e30c0 Fatal error: can't bind to ::1:8118: No such file or directory

The message means privoxy cannot use IPv6. Lets disable listening on an IPv6 address by commenting out the setting.

root@ff788ea93ee0:/# sed -i "s/.*\[::1\]:8118/# &/" /etc/privoxy/config

Now we should be able to start and use privoxy

root@75f6721089f2:/# service privoxy start
 * Starting filtering proxy server privoxy          [ OK ]
root@75f6721089f2:/# torify curl http://icanhazip.com/
176.10.99.200
root@75f6721089f2:/# curl -x 127.0.0.1:8118 http://icanhazip.com/
176.10.99.200

12. Change and check Tor IP solely with Python3

root@75f6721089f2:/# pip3 install requests
root@75f6721089f2:/# python3

>>> import requests
>>>
>>> from stem import Signal
>>> from stem.control import Controller
>>>
>>> response = requests.get('http://icanhazip.com/', proxies={'http': '127.0.0.1:8118'})
>>> response.text.strip()
'137.74.171.94'
>>> 
>>> with Controller.from_port(port=9051) as controller:
...     controller.authenticate(password='my password')
...     controller.signal(Signal.NEWNYM)
... 
>>> response = requests.get('http://icanhazip.com/', proxies={'http': '127.0.0.1:8118'})
>>> response.text.strip()
'87.118.92.43'
>>>
>>> response = requests.get('https://api.myip.com/', proxies={'https': '127.0.0.1:8118'})
>>> response.json()
{"ip": "87.118.92.43", "country": "Germany", "cc": "DE"}
>>> 

13. [bonus] Change and check Tor IP with TorIpChanger

root@75f6721089f2:/# pip3 install toripchanger
root@75f6721089f2:/# python3

>>> from toripchanger import TorIpChanger
>>> 
>>> tor_ip_changer = TorIpChanger(tor_password='my password', tor_port=9051, local_http_proxy='127.0.0.1:8118')
>>> tor_ip_changer.get_new_ip()
'185.24.218.182'
>>>