Seeking some guidance with net/http

Andrew_Cowan · 4 April 2008 00:48

Heya, I am attempting to write a script that lets me login to my Youtube account to gather frequent data for the videos I have uploaded, I am stumbling over the Youtube login procedure which seems to involve both cookies and session info. I am able to get this login procedure to work with Perl and LWP::UserAgent but so far using net/http and Ruby I am failing.

Here is the code I have been playing with which involves first a POST form submission to youtube's /login script, then the code to follow the redirection back to the /index page -- but looking at the contents returned from /index it appears youtube doesn't recognize me as being logged in:

···

---

# Note: I have added a bunch of puts statements to help debug, leaving them here for this post
require 'net/http'

http = Net::HTTP.new( 'www.youtube.com', 80 )
path = '/login'
agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1'

data = 'nexturl=/&current_form=loginForm&action_login=Log+In&username=MY_USER&password=MY_PWD'

headers = {
  'Referer' => 'http://www.youtube.com/',
  'Content-Type' => 'application/x-www-form-urlencoded',
  'User-Agent' => agent
}

resp, data = http.post( path, data, headers )
cookie = resp.response['set-cookie']

if ( resp.code =~ /^3/ )
puts "Got Redirect: " + resp['location']
end

next_loc = resp['location']

puts 'Logging In to Set Cookies:'
puts 'Code=' + resp.code
puts 'Mesg=' + resp.message
puts 'Cookie=' + cookie
puts 'Next Location=' + next_loc
puts 'Data=' + data

headers = {
'Cookie' => cookie,
'User-Agent' => agent
}

resp, data = http.get( next_loc, headers )

puts '(should be) Logged In:'
puts 'Code=' + resp.code
puts 'Mesg=' + resp.message
puts 'Data=' + data

---

I am unsure what part I am missing which youtube apparently needs to complete its login procedure, I would greatly appreciate any helpful suggestions!

Thanks,
Andy

Shawn_Anderson · 4 April 2008 00:53

You may want to check out:
http://youtube-g.rubyforge.org/

at least see how they manage the authentication aspect.

/Shawn

···

On Thu, Apr 3, 2008 at 5:48 PM, Andrew Cowan <icculus@gmdstudios.com> wrote:

Heya, I am attempting to write a script that lets me login to my Youtube
account to gather frequent data for the videos I have uploaded, I am
stumbling over the Youtube login procedure which seems to involve both
cookies and session info. I am able to get this login procedure to work with
Perl and LWP::UserAgent but so far using net/http and Ruby I am failing.

Here is the code I have been playing with which involves first a POST form
submission to youtube's /login script, then the code to follow the
redirection back to the /index page -- but looking at the contents returned
from /index it appears youtube doesn't recognize me as being logged in:

---

# Note: I have added a bunch of puts statements to help debug, leaving
them here for this post
require 'net/http'

http = Net::HTTP.new( 'www.youtube.com', 80 )
path = '/login'
agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1)
Gecko/20060111 Firefox/1.5.0.1'

data =
'nexturl=/&current_form=loginForm&action_login=Log+In&username=MY_USER&password=MY_PWD'

headers = {
'Referer' => 'http://www.youtube.com/',
'Content-Type' => 'application/x-www-form-urlencoded',
'User-Agent' => agent
}

resp, data = http.post( path, data, headers )
cookie = resp.response['set-cookie']

if ( resp.code =~ /^3/ )
puts "Got Redirect: " + resp['location']
end

next_loc = resp['location']

puts 'Logging In to Set Cookies:'
puts 'Code=' + resp.code
puts 'Mesg=' + resp.message
puts 'Cookie=' + cookie
puts 'Next Location=' + next_loc
puts 'Data=' + data

headers = {
'Cookie' => cookie,
'User-Agent' => agent
}

resp, data = http.get( next_loc, headers )

puts '(should be) Logged In:'
puts 'Code=' + resp.code
puts 'Mesg=' + resp.message
puts 'Data=' + data

---

I am unsure what part I am missing which youtube apparently needs to
complete its login procedure, I would greatly appreciate any helpful
suggestions!

Thanks,
Andy

Dan6 · 4 April 2008 00:56

Try this: http://rubyforge.org/projects/youtube

Andrew_Cowan · 4 April 2008 01:05

Hiya Shawn, I did have a look at youtube-g, have the README file open as I type this in fact but I don't see anything related to authenticating an actual Youtube account, the lib is more for accessing the functions that Youtube makes available with just guest access as far as I can tell...

-Andy

Shawn Anderson wrote:

···

You may want to check out:
http://youtube-g.rubyforge.org/

at least see how they manage the authentication aspect.

/Shawn

On Thu, Apr 3, 2008 at 5:48 PM, Andrew Cowan <icculus@gmdstudios.com> wrote:

Heya, I am attempting to write a script that lets me login to my Youtube
account to gather frequent data for the videos I have uploaded, I am
stumbling over the Youtube login procedure which seems to involve both
cookies and session info. I am able to get this login procedure to work with
Perl and LWP::UserAgent but so far using net/http and Ruby I am failing.

Here is the code I have been playing with which involves first a POST form
submission to youtube's /login script, then the code to follow the
redirection back to the /index page -- but looking at the contents returned
from /index it appears youtube doesn't recognize me as being logged in:

---

# Note: I have added a bunch of puts statements to help debug, leaving
them here for this post
require 'net/http'

http = Net::HTTP.new( 'www.youtube.com', 80 )
path = '/login'
agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1)
Gecko/20060111 Firefox/1.5.0.1'

data =
'nexturl=/&current_form=loginForm&action_login=Log+In&username=MY_USER&password=MY_PWD'

headers = {
'Referer' => 'http://www.youtube.com/',
'Content-Type' => 'application/x-www-form-urlencoded',
'User-Agent' => agent
}

resp, data = http.post( path, data, headers )
cookie = resp.response['set-cookie']

if ( resp.code =~ /^3/ )
puts "Got Redirect: " + resp['location']
end

next_loc = resp['location']

puts 'Logging In to Set Cookies:'
puts 'Code=' + resp.code
puts 'Mesg=' + resp.message
puts 'Cookie=' + cookie
puts 'Next Location=' + next_loc
puts 'Data=' + data

headers = {
'Cookie' => cookie,
'User-Agent' => agent
}

resp, data = http.get( next_loc, headers )

puts '(should be) Logged In:'
puts 'Code=' + resp.code
puts 'Mesg=' + resp.message
puts 'Data=' + data

---

I am unsure what part I am missing which youtube apparently needs to
complete its login procedure, I would greatly appreciate any helpful
suggestions!

Thanks,
Andy

Andrew_Cowan · 4 April 2008 01:28

Dan Diebolt wrote:

Try this: http://rubyforge.org/projects/youtube

Dan, like the youtube-g lib this one also provides no means for authenticating a Youtube account. Though this does provide some interesting options that can help me in some respects, it doesn't provide me with the same functionality that I can have as if I were actually logged into the website, which I am trying to emulate.

Thanks
-Andy

Shawn_Anderson · 4 April 2008 01:53

This worked for me:

require 'rubygems'
require 'mechanize'

user = "user_here"
pw = "password_here"
agent = WWW::Mechanize.new
page = agent.post 'http://youtube.com/signup',
{"username"=>user,"password"=>pw,"current_form"=>"loginForm","action_login"=>"Log+In"}
page = agent.get 'http://youtube.com/'
# this returns true, showing you are logged in
p page.body.to_s.include?("/user/#{user}")

···

On Thu, Apr 3, 2008 at 6:28 PM, Andrew Cowan <icculus@gmdstudios.com> wrote:

Dan Diebolt wrote:

> Try this: http://rubyforge.org/projects/youtube
>
>
>
Dan, like the youtube-g lib this one also provides no means for
authenticating a Youtube account. Though this does provide some interesting
options that can help me in some respects, it doesn't provide me with the
same functionality that I can have as if I were actually logged into the
website, which I am trying to emulate.

Thanks
-Andy

Andrew_Cowan · 4 April 2008 02:21

Shawn, I had not heard of the mechanize lib, but sure enough it works great. Thank you for posting that, much appreciated!

Thanks
-Andy

Shawn Anderson wrote:

···

This worked for me:

require 'rubygems'
require 'mechanize'

user = "user_here"
pw = "password_here"
agent = WWW::Mechanize.new
page = agent.post 'http://youtube.com/signup',
{"username"=>user,"password"=>pw,"current_form"=>"loginForm","action_login"=>"Log+In"}
page = agent.get 'http://youtube.com/'
# this returns true, showing you are logged in
p page.body.to_s.include?("/user/#{user}")

On Thu, Apr 3, 2008 at 6:28 PM, Andrew Cowan <icculus@gmdstudios.com> wrote:

Dan Diebolt wrote:

Try this: http://rubyforge.org/projects/youtube

Dan, like the youtube-g lib this one also provides no means for
authenticating a Youtube account. Though this does provide some interesting
options that can help me in some respects, it doesn't provide me with the
same functionality that I can have as if I were actually logged into the
website, which I am trying to emulate.

Thanks
-Andy

7stud · 4 April 2008 03:59

Andrew Cowan wrote:

Shawn, I had not heard of the mechanize lib, but sure enough it works
great. Thank you for posting that, much appreciated!

Thanks
-Andy

perl has mechanize too.

···

--
Posted via http://www.ruby-forum.com/\.

Topic		Replies	Views
Net::HTTP Cookies / Headers (almost there...) ruby-talk	1	98	7 June 2007
Net/http with cookies? ruby-talk	1	104	30 August 2005
POST with Net::HTTP ruby-talk	3	110	12 June 2014
Session with http/net lib ruby-talk	0	142	26 February 2009
HTTP, client cookie and client form handling ruby-talk	0	156	15 August 2005

Seeking some guidance with net/http

Related topics