Last active
January 2, 2016 05:29
-
-
Save andreaseger/8257185 to your computer and use it in GitHub Desktop.
download episodes from elixirsips.com - subscription required
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sips/* | |
rss.xml | |
.bundle | |
auth.rb |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2.1.2 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#! /usr/bin/env ruby | |
require 'bundler/setup' | |
require 'open-uri' | |
require 'rss' | |
require 'nokogiri' | |
require 'fileutils' | |
URL = 'https://elixirsips.dpdcart.com/feed' | |
require_relative 'auth' | |
def download(files, base) | |
unless File.exists?(base) | |
FileUtils.mkdir_p(base) | |
end | |
# p files.map{|e| e[:name]} | |
files.each do |file| | |
filename = "#{base}/#{file[:name]}" | |
if File.exists?(filename) | |
puts "#{file[:name]} skipped (already loaded)" unless @silent | |
next | |
end | |
File.open(filename,'wb') do |f| | |
puts "fetching #{file[:name]}... " | |
open(file[:uri], 'rb', http_basic_authentication: [USER,PASSWORD]) do |read_file| | |
f.write(read_file.read) | |
end | |
end | |
end | |
end | |
def run start, count: 1, base: './sips' | |
open(URL, http_basic_authentication: [USER,PASSWORD]) do |rss| | |
feed = RSS::Parser.parse(rss) | |
feed.items.reverse[(start-1)..(start+count-2)].each do |item| | |
video = { uri: item.enclosure.url } | |
video[:name] = File.basename(URI.parse(video[:uri]).path) | |
doc = Nokogiri::HTML(item.description) | |
other_files = doc.xpath('.//li/a').map{|e| | |
begin | |
href = e.attributes['href'].value | |
file_id = href.match(/\?file_id=(\d+)$/)[1] | |
rescue | |
next | |
end | |
{ name: e.text, uri: "https://elixirsips.dpdcart.com/feed/download/#{file_id}/#{e.text}"} | |
}.compact.reject{|e| e[:name] == video[:name]}.reject{|e| e[:name] == video[:name].gsub('mp4','mkv')} | |
download(other_files.push(video), "#{base}/#{item.title.gsub(':','').gsub(/^Episode\s/,'')}") | |
end | |
end | |
end | |
require "optparse" | |
options = {:count => 1} | |
ARGV.options do |opts| | |
opts.banner = "Usage: #{File.basename($PROGRAM_NAME)} [OPTIONS] EPISODE" | |
opts.separator "" | |
opts.separator "Specific Options:" | |
opts.on( "-c", "--count COUNT", Integer, "Number of episodes to fetch. Default: 1" ) do |opt| | |
options[:count] = opt | |
end | |
opts.on( "-d", "--dir DIRECTORY", String, "Where to put the downloaded stuff. Default: './sips'" ) do |opt| | |
options[:base] = opt | |
end | |
@silent = false | |
opts.on( "-s", "--silent", "Be more silent regarding output to stdout") do | |
@silent = true | |
end | |
opts.separator "Common Options:" | |
opts.on( "-h", "--help", | |
"Show this message." ) do | |
puts opts | |
exit | |
end | |
begin | |
opts.parse! | |
rescue | |
puts opts | |
exit | |
end | |
end | |
run(ARGV.first.to_i, options) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# A sample Gemfile | |
source "https://rubygems.org" | |
gem 'nokogiri' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
GEM | |
remote: https://rubygems.org/ | |
specs: | |
mini_portile (0.5.2) | |
nokogiri (1.6.1) | |
mini_portile (~> 0.5.0) | |
PLATFORMS | |
ruby | |
DEPENDENCIES | |
nokogiri |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#! /usr/bin/env bash | |
# Changing directories to your project and load rvm gemset | |
export PATH="/home/sch1zo/.rbenv/bin:$PATH" | |
eval "$(rbenv init -)" | |
cd /srv/cron/elixir-sips-loader | |
# call script via bundler | |
bundle exec ruby fetch.rb -s -d /home/sch1zo/_btsync/elixir-sips |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment