Skip to content

Instantly share code, notes, and snippets.

@jfmherokiller
Created May 30, 2017 04:06
Show Gist options
  • Select an option

  • Save jfmherokiller/27619f5e64ce2ba362f6aa841e11e186 to your computer and use it in GitHub Desktop.

Select an option

Save jfmherokiller/27619f5e64ce2ba362f6aa841e11e186 to your computer and use it in GitHub Desktop.
ipfs-scrape powershell version
Param(
[string]$URL
)
$olddirectory = (Get-Item -Path ".\" -Verbose).FullName
function New-TemporaryDirectory {
$parent = [System.IO.Path]::GetTempPath()
[string] $name = [System.Guid]::NewGuid()
New-Item -ItemType Directory -Path (Join-Path $parent $name)
}
$newdirectory = New-TemporaryDirectory
cd $newdirectory
& wget.exe -q --show-progress --page-requisites --html-extension --convert-links --random-wait -e robots=off -nd --span-hosts $URL 2>&1
#test -f index.html || {
#INDEX_FILE=$(ls -S | grep -i .html | head -n1)
if (-not (Test-Path 'index.html'))
{
$indexfilesubstring = "*"+$URL.Substring($URL.LastIndexOf("/") + 1)+"*"
$indexfile = (ls -Filter "*.html") | Where {$PSItem.Name -like "$indexfilesubstring"}
Rename-Item "$indexfile" ".\index.html"
}
$pagehash = (ipfs add -r --pin=false -Q ".")
Write-Host "###############"
Write-Host "## DUMP COMPLETE"
Write-Host "##"
Write-Host "## Urls:"
Write-Host "## http://localhost:8080/ipfs/$pagehash"
Write-Host "## https://ipfs.io/ipfs/$pagehash"
cd ..
Remove-Item $newdirectory.FullName -Recurse -Force
cd $olddirectory
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment