Download and merge HTTP Live Streaming .ts files

<?php
/*
Made by Kudusch (blog.kudusch.de, kudusch.de, @Kudusch)

---------

DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE 
                    Version 2, December 2004 

Copyright (C) 2004 Sam Hocevar <[email protected]>
Everyone is permitted to copy and distribute verbatim or modified 
copies of this license document, and changing it is allowed as long 
as the name is changed. 

DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE 
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 
   
0. You just DO WHAT THE FUCK YOU WANT TO.

---------

# How to use

- go to http://mediathek.daserste.de, open with iPad User Agent
- navigate to the desired program, get link to the master.m3u8 file from source (in the <video>-tag)
- run the script with url to the master.m3u8 file as argument (e.g. php get_tshls 'http://http://hls.daserste.de/i/videoportal/Film/c_380000/386059/ios.smil/master.m3u8'
- wait for the script to download and merge all media files; every 10 second part is about 3 MB (when it's done, it will output the runtime)
- if necessary, convert the *.ts file with a media converter eg. handbrake to a *.mp4 file
- enjoy

*/

//runtime
$startTime = microtime(get_as_float);

//get url from input
$url = $argv[1];
//get stream with highest bandwith
$streamUrl = getHighBandwidthStream($url);
//get array of all links to *.ts files
$list = getHlsFiles($streamUrl);

//make new directory
if (!is_dir('files')) {
	mkdir('files');
}

//download all files from array, name with 3 leading zeros
//if file is longer than 166.5 minutes, adjust str_pad params
$n = 1;
foreach ($list as $key) {
	$number = str_pad($n, 3, "0", STR_PAD_LEFT);
	print_r($n." ");
	file_put_contents("files/part.".$number.".ts", fopen($key, 'r'));
	$n++;
}

//merge files and delte parts
sleep(10);
mergeFiles('files');

//echo part numbers and runtime for debugging
echo("\nRun in ".(microtime(get_as_float)-$startTime)." seconds.");

//input: string, output: string
function getHighBandwidthStream($masterUrl) {
	//get content of master.m3u8
	$ch = curl_init($masterUrl);
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
	curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
	$result = curl_exec($ch);
	curl_close($ch);
	
	//return link to second last stream (https://developer.apple.com/library/ios/documentation/networkinginternet/conceptual/streamingmediaguide/FrequentlyAskedQuestions/FrequentlyAskedQuestions.html#//apple_ref/doc/uid/TP40008332-CH103-SW1)
	$result = split("#", $result);
	for ($i = 0; $i < 2; $i++) {
		array_shift($result);
	}
	$length = count($result);
	$result = split("\n", $result[$length-2]);
	return $result[1];
}

//input: string, output: array
function getHlsFiles($streamUrl) {
	//get content of *.m3u8 file
	$ch = curl_init($streamUrl);
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
	curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
	$raw = curl_exec($ch);
	curl_close($ch);
	
	//remove comments and unnecessary data
	$list_raw = split("\n", $raw);
	for ($i = 0; $i < 5; $i++) {
		array_shift($list_raw);
	}
	for ($i = 0; $i < 2; $i++) {
		array_pop($list_raw);
	}
	
	//extract file links	
	$list = array();
	$i = 1;
	foreach ($list_raw as $key) {
		if($i%2 == 0) {
			array_push($list, $key);
		}
		$i++;
	}
	
	//return array
	return $list;
}

function mergeFiles($dirName) {
	//get all *.ts files in directory
	if ($handle = opendir($dirName)) {
	while (false !== ($file = readdir($handle))) {
		if (strpos($file, ".ts") !== false) {
			$fileList = $fileList." files/".$file;
		}
	}
	closedir($handle);
	}
	
	//join and remove parts
	$shellScript = "cat ".substr($fileList, 1)." >> movie.ts";
	shell_exec($shellScript);
	shell_exec("rm -r files");
}
?>

could you explain how to run this script and download live m3u8 stream ?

when I run the script it says :
'cat is not recognized as an internal or external command, operable program or batch file.'
'rm is not recognized as an internal or external command, operable program or batch file.'

I found that using FFMpeg (for free) on the Mac Terminal command line worked perfectly to consolidate the tiny .ts files into one big mp4 video file. Example: ffmpeg -i http://embed.wistia.com/deliveries/blahBlahYadaYada.m3u8 -c copy -bsf:a aac_adtstoasc output.mp4 See: http://stackoverflow.com/questions/32528595/ffmpeg-mp4-from-http-live-streaming-m3u8-file

I am trying to download the video from:
https://thevideobee.to/rhcdmrzk4zm7.html
Which has 248 one MB segments.
The url of these 248 segments is (number near end increments from 1 to 248):
https://fsc.thevideobee.to:444/hls/exfplmti7sfr4navd5wkli7sqa4j65suzuyad4q5pgjqd6cryu2pqnyb4nlq/seg-1-v1-a1.ts
https://fsc.thevideobee.to:444/hls/exfplmti7sfr4navd5wkli7sqa4j65suzuyad4q5pgjqd6cryu2pqnyb4nlq/seg-248-v1-a1.ts
Question: how can I download all 248 segments and join into one file?

@ryancwalsh - That snippet is amazing, You have saved me countless hours of trying to figure this out.

Tried a bunch of scripts that did nothing except spew a bunch of ts files out that didnt work. Your command downloaded and joined the whole video in less than 10 seconds.

You are a true legend

Trying to get this working in Linux Mint 20.0 Cinnamon
(Linux 5.8.0-50-generic #56~20.04.1-Ubuntu x86_64 GNU/Linux)

I installed php7.4
(these packages)
php-common
php7.4-cli
php7.4-common
php7.4-json
php7.4-opcache
php7.4-readline

Ran your script as recommended.

Got these errors while running the script:

PHP Warning: Use of undefined constant get_as_float - assumed 'get_as_float' (this will throw an Error in a future version of PHP) in /home/hulk/test/get_tshls on line 34
PHP Fatal error: Uncaught Error: Call to undefined function curl_init() in /home/hulk/test/get_tshls:68
Stack trace:
#0 /home/hulk/test/get_tshls(39): getHighBandwidthStream()
#1 {main}
thrown in /home/hulk/test/get_tshls on line 68

I commented out the runtime measurement but still get stuck with the "curl_init()" function issue.

Any ideas how to fix this?