iex -S mix # In a project with HTTPotion in mix.exs.
iex(1)> ProgressiveDownloader.run
Let's download 43.2 MB…
2.31% (1.0 MB)
4.63% (2.0 MB)
6.94% (3.0 MB)
9.26% (4.0 MB)
11.57% (5.0 MB)
13.89% (6.0 MB)
16.2% (7.0 MB)
18.52% (8.0 MB)
20.83% (9.0 MB)
23.15% (10.0 MB)
25.46% (11.0 MB)
27.78% (12.0 MB)
30.09% (13.0 MB)
32.4% (14.0 MB)
34.72% (15.0 MB)
37.03% (16.0 MB)
39.35% (17.0 MB)
41.66% (18.0 MB)
43.98% (19.0 MB)
46.29% (20.0 MB)
48.61% (21.0 MB)
50.92% (22.0 MB)
53.24% (23.0 MB)
55.55% (24.0 MB)
57.87% (25.0 MB)
60.18% (26.0 MB)
62.49% (27.0 MB)
64.81% (28.0 MB)
67.12% (29.0 MB)
69.44% (30.0 MB)
71.75% (31.0 MB)
74.07% (32.0 MB)
76.38% (33.0 MB)
78.7% (34.0 MB)
81.01% (35.0 MB)
83.33% (36.0 MB)
85.64% (37.0 MB)
87.96% (38.0 MB)
90.27% (39.0 MB)
92.58% (40.0 MB)
94.9% (41.0 MB)
97.21% (42.0 MB)
99.53% (43.0 MB)
100.0% (43.2 MB)
All downloaded! See: /tmp/posture.mp4
Last active
December 4, 2020 05:41
-
-
Save henrik/108e5fc23b66131fc3aa to your computer and use it in GitHub Desktop.
Example of progressive downloading with HTTPotion, showing percentage and downloaded bytes as you go.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
defmodule ProgressiveDownloader do | |
def run do | |
url = "https://ia600308.us.archive.org/2/items/HealthYo1953/HealthYo1953_512kb.mp4" | |
HTTPotion.get url, stream_to: self, timeout: :infinity | |
receive_data(total_bytes: :unknown, data: "") | |
end | |
defp receive_data(total_bytes: total_bytes, data: data) do | |
receive do | |
%HTTPotion.AsyncHeaders{headers: h} -> | |
{total_bytes, _} = h[:"Content-Length"] |> Integer.parse | |
IO.puts "Let's download #{mb total_bytes}…" | |
receive_data(total_bytes: total_bytes, data: data) | |
%HTTPotion.AsyncChunk{chunk: new_data} -> | |
accumulated_data = data <> new_data | |
accumulated_bytes = byte_size(accumulated_data) | |
percent = accumulated_bytes / total_bytes * 100 |> Float.round(2) | |
IO.puts "#{percent}% (#{mb accumulated_bytes})" | |
receive_data(total_bytes: total_bytes, data: accumulated_data) | |
%HTTPotion.AsyncEnd{} -> | |
path = "/tmp/posture.mp4" | |
File.write!(path, data) | |
IO.puts "All downloaded! See: #{path}" | |
end | |
end | |
defp mb(bytes) do | |
number = bytes / 1_048_576 |> Float.round(2) | |
"#{number} MB" | |
end | |
end |
I tried to use this snippet, but apparently there's something wrong with HTTPotion at some point:
iex(1)⋅❯ ProgressiveDownloader.run
** (Protocol.UndefinedError) protocol List.Chars not implemented for #PID<0.139.0>
(elixir) lib/list/chars.ex:1: List.Chars.impl_for!/1
(elixir) lib/list/chars.ex:12: List.Chars.to_char_list/1
lib/httpotion.ex:157: anonymous fn/1 in HTTPotion.request/5
(elixir) lib/enum.ex:1043: anonymous fn/3 in Enum.map/2
(elixir) lib/enum.ex:1385: Enum."-reduce/3-lists^foldl/2-0-"/3
(elixir) lib/enum.ex:1043: Enum.map/2
lib/httpotion.ex:157: HTTPotion.request/5
(sploit) lib/1-progressive_downloader.ex:5: ProgressiveDownloader.run/0
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Combine with https://gist.github.com/henrik/b950bfd7fa432eacd902 for a progress bar.
See it in action in https://github.com/henrik/sipper.