Use openssl to fetch an URL

0_README.md

Using openssl to fetch an URL

Many non-root containers are minimal but have /bin/sh and /usr/bin/openssl. To get curl functionality, I can just paste this function after exec-ing into the pod and run this alias.

Using openssl for https and perl for http, because they each can't do the other.

$ get_url httpbin.dev | head
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Content-Security-Policy: frame-ancestors 'self' *.httpbin.dev; font-src 'self' *.httpbin.dev; default-src 'self' *.httpbin.dev; img-src 'self' *.httpbin.dev https://cdn.scrapfly.io; media-src 'self' *.httpbin.dev; object-src 'self' *.httpbin.dev https://web-scraping.dev; script-src 'self' 'unsafe-inline' 'unsafe-eval' *.httpbin.dev; style-src 'self' 'unsafe-inline' *.httpbin.dev https://unpkg.com; frame-src 'self' *.httpbin.dev https://web-scraping.dev; worker-src 'self' *.httpbin.dev; connect-src 'self' *.httpbin.dev
Content-Type: text/html; charset=utf-8
Date: Fri, 12 Dec 2025 21:15:04 GMT
Permissions-Policy: fullscreen=(self), autoplay=*, geolocation=(), camera=(), ch-ua=*, ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-full-version-list=*, ch-ua-mobile=*, ch-ua-model=*, ch-ua-platform=*, ch-ua-platform-version=*, ch-ua-wow64=*, ch-ua-form-factor=*, ch-viewport-width=*, ch-viewport-height=*, ch-dpr=*, ch-width=*, ch-prefers-reduced-motion=*, ch-prefers-color-scheme=*, ch-downlink=*, ch-ect=*, ch-rtt=*, ch-save-data=*, ch-device-memory=*
Referrer-Policy: strict-origin-when-cross-origin
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
X-Content-Type-Options: nosniff
...

Credits

Gemini 3 Pro.

1_alias.sh

get_url() {
    url=$1
    
    # 1. Detect protocol (default to http if missing)
    proto="http"
    case "$url" in
        https://*) proto="https"; url=${url#https://} ;;
        http://*)  proto="http";  url=${url#http://} ;;
    esac

    # 2. Extract Host and Path
    host=${url%%/*}
    path=${url#"$host"}
    path=${path:-/}

    # 3. Extract Port (or set defaults)
    case "$host" in
        *:*) port=${host##*:}; host=${host%:*} ;;
        *)   if [ "$proto" = "https" ]; then port=443; else port=80; fi ;;
    esac

    # 4. Execute request based on protocol
    if [ "$proto" = "https" ] || [ "$port" = "443" ]; then
        # Use OpenSSL for HTTPS
        printf "GET %s HTTP/1.1\r\nHost: %s\r\nConnection: close\r\n\r\n" "$path" "$host" | \
        openssl s_client -quiet -connect "$host:$port" 2>/dev/null
    else
        # Use Perl for HTTP (OpenSSL cannot do plain TCP)
        # We use env vars to pass data safely into Perl
        export R_HOST="$host" R_PORT="$port" R_PATH="$path"
        perl -MIO::Socket::INET -e '
            $s = IO::Socket::INET->new(PeerAddr=>"$ENV{R_HOST}:$ENV{R_PORT}",Proto=>"tcp",Timeout=>5);
            if($s){
                print $s "GET $ENV{R_PATH} HTTP/1.1\r\nHost: $ENV{R_HOST}\r\nConnection: close\r\n\r\n";
                print while <$s>;
            } else { print "Connection failed\n"; }'
    fi
}