How to download .m3u8 in once time

Question

I have a .m3u8 file on remote host, with contain fixed numbers of chunk .ts file name, and not stream:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:11
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:9.736,
media_0.ts
#EXTINF:9.96,
media_1.ts
#EXTINF:10.0,
media_2.ts
#EXTINF:10.0,
media_3.ts
#EXTINF:10.0,
media_4.ts
#EXTINF:10.2,
media_5.ts
#EXTINF:10.0,

When I use this command:

# ffmpeg -i "http://example.com/chunklist.m3u8" file.mp4

frame=  582 fps=9.4 q=28.0 size=    1536kB time=00:00:23.21 bitrate= 542.1kbits/s dup=2 drop=4 speed=0.375x

It works. But It get frame by frame video and very long time needed. (It takes time almost to playback the video.)

But since the path of all the .ts files are known. (http://example.com/media_0.ts, http://example.com/media_1.ts, ...) There must be a way to get and merge them all at the same time.

But How in ffmpeg directly?!

EDIT (try a solution):

For one solution, I know how can concatenation files with ffmpeg.

ffmpeg -i "concat:0.ts|1.ts|2.ts|3.ts|4.ts|5.ts" -c copy output.mp4

This ffmpeg command was great, and works in less 1 sec time!

So try to download all .ts files with CURL with this command:

curl \
http://example.com/media_0.ts -o 0.ts \
http://example.com/media_1.ts -o 1.ts \
http://example.com/media_2.ts -o 2.ts \
http://example.com/media_3.ts -o 3.ts \
http://example.com/media_4.ts -o 4.ts \
http://example.com/media_5.ts -o 5.ts

But you can see result:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  687k  100  687k    0     0  75108      0  0:00:09  0:00:09 --:--:-- 74111
100  652k  100  652k    0     0  59404      0  0:00:11  0:00:11 --:--:-- 53400
100  673k  100  673k    0     0  48675      0  0:00:14  0:00:14 --:--:-- 55781
100  657k  100  657k    0     0  63573      0  0:00:10  0:00:10 --:--:-- 62494
100  671k  100  671k    0     0  39019      0  0:00:17  0:00:17 --:--:-- 40863
100  692k  100  692k    0     0  63480      0  0:00:11  0:00:11 --:--:-- 80049

See, total download time was 72 sec, while the total duration of all parts is 59 sec! that this time is very long!

So sorry, download all parts and then concat that, was not good solution.

EDIT 2

I try for another .m3u8 file on the another server with difference URL:

Download and concat together:

ffmpeg -i "concat:\
http://184.72.239.149/vod/smil:BigBuckBunny.smil/media_w442897525_b560000_0.ts|\
http://184.72.239.149/vod/smil:BigBuckBunny.smil/media_w442897525_b560000_1.ts|\
http://184.72.239.149/vod/smil:BigBuckBunny.smil/media_w442897525_b560000_2.ts|\
http://184.72.239.149/vod/smil:BigBuckBunny.smil/media_w442897525_b560000_3.ts|\
http://184.72.239.149/vod/smil:BigBuckBunny.smil/media_w442897525_b560000_4.ts|\
http://184.72.239.149/vod/smil:BigBuckBunny.smil/media_w442897525_b560000_5.ts\
" -c copy -y output.ts

Another command with input.txt URLs file.

ffmpeg -f "concat" -i "input.txt" -c copy -y output.ts

input.txt file:

file 'http://184.72.239.149/vod/smil:BigBuckBunny.smil/media_w442897525_b560000_0.ts'
file 'http://184.72.239.149/vod/smil:BigBuckBunny.smil/media_w442897525_b560000_1.ts'
file 'http://184.72.239.149/vod/smil:BigBuckBunny.smil/media_w442897525_b560000_2.ts'
file 'http://184.72.239.149/vod/smil:BigBuckBunny.smil/media_w442897525_b560000_3.ts'
file 'http://184.72.239.149/vod/smil:BigBuckBunny.smil/media_w442897525_b560000_4.ts'
file 'http://184.72.239.149/vod/smil:BigBuckBunny.smil/media_w442897525_b560000_5.ts'

Or this command some time if needed:

ffmpeg -f "concat" -safe "0" -protocol_whitelist "file,http,https,tcp,tls" -i "input.txt" -c copy -y output.ts

Finally, for that download speed was good, MAYBE my server target has limited bandwidth. :-(

Answer 1

This method seems to work when you receive 403 Forbidden errors while downloading the video (it emulates the same headers that your browser sends):

Using Firefox, open Developer Tools using Ctrl + Shift + I, or click on the 3-line hamburger menu in the top-right (1.) > More Tools (2.) > Web Developer Tools (3.).

Go to the Network tab (4.), reload the page, find the appropriate .m3u8 entry, typically named stream.m3u8 or something similar (5.), right-click, Copy Value (6.) > Copy as cURL (7.).

Then paste the resulting cURL command into here: https://windyakin.github.io/curl2ffmpeg. Download and install the free, open source FFMpeg tool if you don't already have it, paste the resulting ffmpeg command into your commandline (you may have to add a .exe suffix, eg: ffmpeg becomes ffmpeg.exe if you are using Windows) and enjoy the video!

Answer 2

I recently had to do this, and @sam46's answer is pretty good but life is easier if you make a list of segment URLs rather than do this count / string replace stuff. My work is based off sam46's.

This script will make a command line that can be excuted thusly, assuming you save it to a file called download-m3u8.py. It requires the requests library.

python3 download-m3u8.py https://some/url.m3u8 whatever-name.mpg

import requests
import sys
from urllib.parse import urlparse

# Some fake headers to look a little more browserly
HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0",
    "Accept": "*/*",
    "Accept-Language": "en-US,en;q=0.5",
    "Accept-Encoding": "gzip, deflate, br",
    "DNT": "1",
    "Connection": "keep-alive",
    "Pragma": "no-cache",
    "Cache-Control": "no-cache"
}

if len(sys.argv) != 3:
    print(
        "Requires parameters: m3u8_url output-file-name"
    )
    sys.exit(1)

initial_url = sys.argv[1]
output_file = sys.argv[2]
segments = []

req = requests.get(initial_url, headers=HEADERS)

# Get segments file; segments end in .ts, everything else is junk
for line in req.iter_lines():
    line = line.decode('utf8').strip()

    if line.endswith('.ts'):
        segments.append(line)

# Parse base url
parsed = urlparse(initial_url)

# open file, dump segments into it
with open(output_file, 'wb') as output:
    for line in segments:
        url = f"{parsed.scheme}://{parsed.netloc}{line}"

        print(f"Downloading {url}...")
        seg = requests.get(url, headers=HEADERS)
        output.write(seg.content)

Answer 3

You can try this command:

ffmpeg -y \
  -v warning \
  -loglevel debug \
  -i "https://m3u8.url" \
  -vcodec copy \
  -c copy -f mpegts out.ts

Followed by this command to convert ts to mp4:

ffmpeg -i out.ts -acodec copy -vcodec copy out.mp4

Answer 4

I couldn't get ffmpeg to work correctly without skipping frames (and their bug tracker is broken so I can't even report it -_-) but I did find two other methods that work:

1. Youtube-dl. The command

   youtube-dl.exe --hls-prefer-native "https://link-to-m3u8-file"
   

   just worked immediately, and ripped the raw audio from the stream in (in my case) .mp4 format

2. VLC. Go to Media --> Open Network Stream, then paste the m3u8 URL. Click the dropdown next to 'Play' and choose 'Convert', then choose your conversion settings.
   They have an option for "dump raw input", but I wasn't able to get it to work. However "mp3" worked fine.

Answer 5

The correct way to concat multiple video files from m3u8 playlist is

ffmpeg -i "http://example.com/chunklist.m3u8" -codec copy file.mp4

---

• the m3u8 playlist can be on web or locally in directory
  • it contains list of file paths relative to the playlist
• -codec copy to avoid encoding (which takes time)
• container type matters:
  • *.mp4 is fine but it seems little slow to mux when playlist is fetched from web
  • *.mkv or *.ts worked best for me

Answer 6

here's some python code that does it, you just need to provide the url of the 1st segment and the number of segments (from the .m3u8 file):

def dumpSegs(initUrl, n, path, append=False):
    """ downlaod and combine the .ts files
    given the first seg's url, the number of segments and
    the destination download path """
    with open(path, 'ab' if append else 'wb') as f:
        for i in range(1, n + 1):
            segurl = initUrl.replace('seg-1-', 'seg-{:d}-'.format(i))
            success = False
            while not success:
                try:
                    seg = requests.get(segurl, headers=HEADERS)
                    success = True
                except:
                    print('retrying...')
            f.write(seg.content)

Here's the same code with a few more bells and whistles