14 replies to this topic

[pedrotuga]

I've noticed on keepvid.com that there is now an mp4 download link for youtube videos.

I googled around and I found out that one can watch youtube videos in mpeg4 format simply by appending &tmp=18 to the url.

I turned on wireshark, typed in a youtube url using with that suffix, and there it was, the download url showed in a HTTP GET request as expected. Now, i've written youtube download scripts before, basically they all went the easy way: they extracted the identifier somewhere from the video page using regex and then the url was made using that video identifier.
That was simple, the identifier was somewhere in the middle off all that crazy javascript.

But now, when it comes to these mp4 videos I am not finding the identifier anywhere, nor I hacked the url yet. I mean, i haven't played around with url parameters.
I guess the http request this time, should be made by the flash video player this time.

Have anbody checked this out?

The goal is:
given an youtube url, getting the mp4 download link.

[sshblack]

given an youtube url, getting the mp4 download link.

can you not just go here: http://www.downloadyoutubevideos.com/

save it as an .flv file and convert it to mp4 should you need to?

[pedrotuga]

given an youtube url, getting the mp4 download link.

can you not just go here: http://www.downloadyoutubevideos.com/

save it as an .flv file and convert it to mp4 should you need to?

In that case i could go to http://keepvid.com and save it directly as an mp4.
That's not the point, I don't want to sound like i am flaming whatsoever, but... did you really read my previous post?

[mirrorshades]

That's not the point, I don't want to sound like i am flaming whatsoever, but... did you really read my previous post?

Well... I would say that one of two things happened:

1) He answered your question completely and correctly.

2) You did a poor job of explaining what you're trying to do.

Since his answer seems valid given my re-reading of your original post, I would suggest you clarify what exactly you're looking for.

[deadwax]

I think what the original poster is asking for is how to fetch the .mp4 file directly from YouTube without using any 3rd party site.

Edit: I just tried KeepVid using a YouTube video URL to see where they are getting the .mp4 file from. It appears they are actually getting the file from Google (vp.video.google.com, cache.googlevideo.com), not YouTube.

Edited by deadwax, 21 April 2008 - 03:55 PM.

[pedrotuga]

Mirrorshades, I am sorry, could i be more clearer than I have been?

The goal is:
given an youtube url, getting the mp4 download link.

I don't know how you interpret it but ssblack didn't answer my question at all.
The video itself is not the url.

I am trying to put together a small hack that grabs the download url. That's the fun of hacking.

deadwax, yes vp.video.google.com resolvs the an ip that actually delivers the video, but i don't think that's relevant, i guess there is other (sub)domains that point to the same ip, some might be something.youtube.com some might besomethingelse.google.com, it doesn't really makes so much of a difference in this case.

Edited by pedrotuga, 21 April 2008 - 04:27 PM.

[mirrorshades]

Mirrorshades, I am sorry, could i be more clearer than I have been?

I see... you're looking for the full URL of the mp4 video on youtube's site, not just the video itself in mp4 format. I think your "succinct" version makes sense now, but left a bit of room for making assumptions.

[~Total_Blackout~]

I googled around and I found out that one can watch youtube videos in mpeg4 format simply by appending &tmp=18 to the url.

its not &tmp=18 its &fmt=18

http://googlesystem....-mp4-files.html

[~Total_Blackout~]

Im not too good at programing with only one semester of java but this is what it looks like you have to do....

1) take a youtube url for example: http://www.youtube.com/watch?v=27mb_AyzGyQ (this is not me)
2)have your program search/grep/whatever the source code for the string after:

"t": "

In my example the string you would be looking for is

OEgsToPDskJd74QTpTp7STC8AsxLZNSt _linenums:0'>OEgsToPDskJd74QTpTp7STC8AsxLZNSt

3)your download link is in the from of http://www.youtube.c...mt=18&video_id=(stuff in red)&t=(stuff in blue)

So for the example above it would be http://www.youtube.c...mt=18&video_id=27mb_AyzGyQ&t=OEgsToPDskJd74QTpTp7STC8AsxLZNSt

Hope this helps

Edit: some people on youtube say that &fmt=6 is better then &fmt=18 so try both and see... I could not tell the diff

Edited by ~Total_Blackout~, 22 April 2008 - 02:29 AM.

[sshblack]

but... did you really read my previous post

Yes I did, maybe you just weren't being as concise as you thought you were. Anyway hope your issue is now solved.

Edited by sshblack, 22 April 2008 - 05:28 AM.

[pedrotuga]

Thank you guys.
It was a typo.

Total_blackout, what was confusing me was the fact that the link on keepvid.com does not look like that, so I was wondering where da heck did they got the google link from.
But with your help I actually found out. I should have analyzed the packets more carefully, it was so much noise that i didn't took the effort on setting up complicated wireshark filters.

Anyway, I found out how the whole thing works.
The request to the actual video file is still made from the javascript, the diference with the mp4 videos is that the server answers a 303 (check other) HTTP code.
The only thing we need to do then is follow the "Location:" header.

The command below and its output explain better than me :S

$  curl -A "Mozilla" -I "http://www.youtube.com/get_video?video_id=Xx8HwFpxv54&t=OEgsToPDskIk5ZmhgVGGc_RKp8AgZPkA&fmt=18"
HTTP/1.1 303 See Other
Date: Tue, 22 Apr 2008 13:20:25 GMT
Server: Apache
Set-Cookie: use_hitbox=72c46ff6cbcdb7c5585c36411b6b334edAEAAAAw; path=/; domain=.youtube.com
Set-Cookie: VISITOR_INFO1_LIVE=1CMN8SRn6fk; path=/; domain=.youtube.com; expires=Fri, 20-Apr-2018 13:20:25 GMT
Set-Cookie: GEO=19ca3cc6c79ee23505b6b6535a5c1925cxYAAABTRSxhYixzdG9ja2hvbG0sLCwsLC0x; path=/; domain=.youtube.com; expires=Thu, 24-Apr-2008 13:20:25 GMT
Set-Cookie: LOCALE_PREFERENCE=86d1d09eefe6b79b4068000ce05518a4dAUAAABlbl9VUw==; path=/; domain=.youtube.com; expires=Fri, 20-Apr-2018 13:20:25 GMT
Expires: Tue, 27 Apr 1971 19:44:06 EST
Cache-Control: no-cache
Location: http://cache.googlevideo.com/videoplayback?id=5f1f07c05a71bf9e&itag=18&begin=0&len=604800000&signature=D6F0F44C01B6DA8FEFA086A6CCB0C379E7CF7211.2D70F2C7A98644360CF69CA85D13432A2D399DD3&ip=85.224.129.21&ipbits=16&expire=1208892025&key=yt1&sver=2
Content-Type: text/html; charset=utf-8

I might write a firefox extension doing the whole thing using javascript. All the extensions out there get the script from some website using a server side script. I think they're trying to hide the trick or something.

[deadwax]

So for the example above it would be http://www.youtube.c...mt=18&video_id=27mb_AyzGyQ&t=OEgsToPDskJd74QTpTp7STC8AsxLZNSt

Did you actually test that link while posting? I get a "HTTP 401 Gone" error when I try that link. I also tried the method you described with a different YouTube video to no avail.

[~Total_Blackout~]

It loooks like the "t" code changes... you just have to find out what it is and modify.

var swfArgs = {"ad_module": 1, "video_id": "27mb_AyzGyQ", "l": 73, "sk": "o7vOOpTcoPfO_B7a9_eA-AU", "fmt_map": "6/720000/7/0/0", "t": "OEgsToPDskJ6mlaeyhTwUdIJ7kbVlNvX", "hl": "en", "plid": "AARLdxiI-uYD5-lNAAAA8QR8YAA", "vq": null, "tk": "qskJqEi3t3G8qCUmxoMZBExxDI2FMHcASXeFh-mCHXOO6ke34R4R_w=="};

[pedrotuga]

madwax, how did you tested?
At first i got a 401, then i tried changing the user-agent string and it worked

check out my snippet above, i set the user-agent to "Mozilla"

EDIT:yay, &fmt=6 shows a way better picture quality than &fmt=18

Edited by pedrotuga, 22 April 2008 - 02:41 PM.

[~Total_Blackout~]

based on the knowledge we have gathered i have made a shell script to download the HD version of youtube videos.

Put the sh file and a file called videos.txt in the same folder edit/create videos.txt and put one youtube link per line...
The program will download the &fmt=18 version because i dont think &fmt=6 is a format for all youtube videos yet but all you have to do is change it and find out if it work for you
It automatically names your files to. it comes in handy when downloading a lot of videos

#!/bin/sh# YOUTUBE DOWNLOADER HD# REMADE BY ~TOTAL_BLACKOUT~ # shouts to blsecres ON FREENODE AND binrev.com# This script is based off of youtube_downloader.sh# that can be found at [url="http://tips.webdesign10.com/files/youtube_downloader.sh.txt"]http://tips.webdesig...wnloader.sh.txt[/url]# Use this script at your own risk!# it has been edited to work with mostly pre-installed programs in linux# you may need to download sed if its not pre-installed# USAGE:# Make a file called videos.txt with the URL of 1 YouTube video on each line.# Don't leave any blank lines in the file# Put the videos.txt file in an empty folder along with this script# run this script with the following commands:# $ ./youtubehd.sh# It will take a long time to run, depending on# how many videos you have in your videos.txt file.#For even higher HD change #the line download="http://www.youtube.com/get_video?fmt=18&video_id$holy&t=$cool"#to download="http://www.youtube.com/get_video?fmt=6&video_id$holy&t=$cool"#Note: fmt=6 format is not in all youtube videos and you may get ERROR 415: Unsupported Media Type.while read inputlinedo  youtube_url="$(echo $inputline)"  wget $youtube_url -O get.code  holy="$(echo $youtube_url | grep -A 11 -o =.[^\/]*)"  cool="$(grep -o -m1 OEgs............................ get.code)"  title="$(sed -n 's%.*<title>\(.*\)</title>.*%\1%p' < get.code)"  download="http://www.youtube.com/get_video?fmt=18&video_id$holy&t=$cool"  wget -O "$title.mp4" $download  rm get.codedone < videos.txt

Edited by ~Total_Blackout~, 22 April 2008 - 08:04 PM.