I have a problem using "scp" from a ruby scrip. It copies my sql file to
the destination but it cuts the file the middle of line 290.
When i run scp manually i the copied file is normal.
I am using `` to call the unix command
Dir.chdir(sql_folder)
`scp -r #{sql_folder} #{remotelogin}:#{remotesqlfolder}/#{document_id}`
Do you have any idea what i am doing wrong?
--
Posted via http://www.ruby-forum.com/.
What I'm using when scping is: scp -r dirToCopy destination ( -r for
recursive copy folder)
but, I believe that when dirToCopy or destination is remote, you should write:
USER@IP:/FOLDER/FILE
in your lines I can't see the @ sign
Could it be like:
`scp -r #{sql_folder} #{remotelogin}:#{remotesqlfolder}/#{document_id}`
`scp -r #{sql_folder} #{remoteUSER}@#{remotesIP}:/#{document_id}`
hope it helps,
Jean
On 9/7/06, Seid Rudy <seidom@hotmail.com> wrote:
I have a problem using "scp" from a ruby scrip. It copies my sql file to
the destination but it cuts the file the middle of line 290.
When i run scp manually i the copied file is normal.I am using `` to call the unix command
Dir.chdir(sql_folder)
`scp -r #{sql_folder} #{remotelogin}:#{remotesqlfolder}/#{document_id}`Do you have any idea what i am doing wrong?
--
Posted via http://www.ruby-forum.com/\.
Seid Rudy wrote:
I have a problem using "scp" from a ruby scrip. It copies my sql file to
the destination but it cuts the file the middle of line 290.
When i run scp manually i the copied file is normal.I am using `` to call the unix command
Dir.chdir(sql_folder)
`scp -r #{sql_folder} #{remotelogin}:#{remotesqlfolder}/#{document_id}`Do you have any idea what i am doing wrong?
I want to emphasize this is just an off-the-wall guess. The `backticked
code` is being forked, the parent process terminates prematurely, and the
forked process is killed, also prematurely.
Have you tried "system" instead of back-ticks? "system" returns a process
completion code, therefore it must wait until the process completes.
--
Paul Lutus
http://www.arachnoid.com
(The OP seems to know the proper syntax, as he is able to do it by hand.)
(I have no clue what the cause is, these are just hint what to try)
- try another file (to see whether it is dependent on some pattern in
the file, or whether it transfers the same size all the time)
- try removing few line around that line
- try using system instead of ``
- look at the logs on the remote side
On 9/7/06, Jean Verger <jean.verger@gmail.com> wrote:
What I'm using when scping is: scp -r dirToCopy destination ( -r for
recursive copy folder)but, I believe that when dirToCopy or destination is remote, you should write:
USER@IP:/FOLDER/FILE
in your lines I can't see the @ sign
Could it be like:
`scp -r #{sql_folder} #{remotelogin}:#{remotesqlfolder}/#{document_id}`
`scp -r #{sql_folder} #{remoteUSER}@#{remotesIP}:/#{document_id}`
Quoting Paul Lutus <nospam@nosite.zzz>:
Why won't you use Net::SFTP instead of calling the external scp?
Seid Rudy wrote:
I have a problem using "scp" from a ruby scrip. It copies my sql file to
the destination but it cuts the file the middle of line 290.
When i run scp manually i the copied file is normal.I am using `` to call the unix command
Dir.chdir(sql_folder)
`scp -r #{sql_folder} #{remotelogin}:#{remotesqlfolder}/#{document_id}`Do you have any idea what i am doing wrong?
I want to emphasize this is just an off-the-wall guess. The `backticked
code` is being forked, the parent process terminates prematurely, and the
forked process is killed, also prematurely.Have you tried "system" instead of back-ticks? "system" returns a process
completion code, therefore it must wait until the process completes.--
Paul Lutus
http://www.arachnoid.com
> What I'm using when scping is: scp -r dirToCopy destination ( -r for
> recursive copy folder)
>
> but, I believe that when dirToCopy or destination is remote, you should write:
>
> USER@IP:/FOLDER/FILE
>
> in your lines I can't see the @ sign
>
> Could it be like:
> `scp -r #{sql_folder} #{remotelogin}:#{remotesqlfolder}/#{document_id}`
> `scp -r #{sql_folder} #{remoteUSER}@#{remotesIP}:/#{document_id}`
scp file #{remotemachine}:
is fine, it copies file to remotemachine to the home of the user with
the same name (or as preconfigured in .ssh/config - you can create
shortcuts for remotemachine and configure what the remote username
should be). You can add /remotefolder/remotefile after the colon to
change the remote location.
>
(The OP seems to know the proper syntax, as he is able to do it by hand.)
(I have no clue what the cause is, these are just hint what to try)
- try another file (to see whether it is dependent on some pattern in
the file, or whether it transfers the same size all the time)
- try removing few line around that line
- try using system instead of ``
- look at the logs on the remote side
Try the -v switch to scp. Perhaps it will break even more but perhaps
it would do the same and show useful information.
Thanks
Michal
On 9/7/06, Jan Svitok <jan.svitok@gmail.com> wrote:
On 9/7/06, Jean Verger <jean.verger@gmail.com> wrote:
Pau Garcia i Quiles wrote:
Why won't you use Net::SFTP instead of calling the external scp?
I can guess one reason. Net::SFTP doesn't work with key-based authentication, only passwords, and there is no password given in OP's code....
I hope that I am wrong and that this has changed recently.
--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
Pau Garcia i Quiles wrote:
Why won't you use Net::SFTP instead of calling the external scp?
I have that question too. Do that do SSH format?
--
Phlip
http://www.greencheese.us/ZeekLand <-- NOT a blog!!!
Michal Suchanek wrote:
Try the -v switch to scp. Perhaps it will break even more but perhaps
it would do the same and show useful information.Thanks
Michal
hi i tried -v option and found out that scp is getting chopped @ byte
40960. Any clue how i can change the byte limit?
debug1: Sending command: scp -v -r -t
/var/local/rudyserv/wilcox/hr/_sql/
Entering directory: D0755 0 069999
Sending file modes: C0644 40960 069999.sql
Sending file modes: C0644 0 069999.err
Sending file modes: C0644 0 069999.tst
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: channel 0: free: client-session, nchannels 1
debug1: fd 0 clearing O_NONBLOCK
debug1: fd 1 clearing O_NONBLOCK
debug1: Transferred: stdin 0, stdout 0, stderr 0 bytes in 0.1 seconds
debug1: Bytes per second: stdin 0.0, stdout 0.0, stderr 0.0
debug1: Exit status 0
rudy@localhost $ ls -al ../sql/069999/069999.sql
-rw-r--r-- 1 seid users 41455 Sep 9 03:18 ../sql/069999/069999.sql
--
Posted via http://www.ruby-forum.com/\.
Actually, it works with key-based authentication:
require 'net/ssh'
require 'net/sftp'
Net::SSH.start( '192.168.1.37',
:username=>'pgquiles'
) do |session|
session.sftp.connect do |sftp|
sftp.put_file("backup.tar.bz2", "/media/hda3/backup.tar.bz2")
end
end
On Thursday 07 September 2006 22:29, Joel VanderWerf wrote:
Pau Garcia i Quiles wrote:
> Why won't you use Net::SFTP instead of calling the external scp?I can guess one reason. Net::SFTP doesn't work with key-based
authentication, only passwords, and there is no password given in OP's
code....I hope that I am wrong and that this has changed recently.
--
Pau Garcia i Quiles
http://www.elpauer.org
(Due to the amount of work, I usually need 10 days to answer)
i spawn scp all the time from ruby, with gigantic files of tens of gigs and
have never seen this. i think this maybe either an os issue or your disk is
failing (though didn't you mention it worked by hand?).
one other possiblity is some pipe filling issue. by using backticks you are
relying on ruby to read stdout as fast as scp produces it. i can't imagine
why there would be an issue there, but i'd try your command using something
like
system "scp #{ src } #{ dst } >#{ log } 2>&1"
are you on windows by any chance?
-a
On Sat, 9 Sep 2006, Seid Rudy wrote:
Michal Suchanek wrote:
Try the -v switch to scp. Perhaps it will break even more but perhaps
it would do the same and show useful information.Thanks
Michal
hi i tried -v option and found out that scp is getting chopped @ byte
40960. Any clue how i can change the byte limit?debug1: Sending command: scp -v -r -t
/var/local/rudyserv/wilcox/hr/_sql/
Entering directory: D0755 0 069999
Sending file modes: C0644 40960 069999.sql
Sending file modes: C0644 0 069999.err
Sending file modes: C0644 0 069999.tst
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: channel 0: free: client-session, nchannels 1
debug1: fd 0 clearing O_NONBLOCK
debug1: fd 1 clearing O_NONBLOCK
debug1: Transferred: stdin 0, stdout 0, stderr 0 bytes in 0.1 seconds
debug1: Bytes per second: stdin 0.0, stdout 0.0, stderr 0.0
debug1: Exit status 0rudy@localhost $ ls -al ../sql/069999/069999.sql
-rw-r--r-- 1 seid users 41455 Sep 9 03:18 ../sql/069999/069999.sql
--
what science finds to be nonexistent, we must accept as nonexistent; but what
science merely does not find is a completely different matter... it is quite
clear that there are many, many mysterious things.
- h.h. the 14th dalai lama
Pau Garcia i Quiles wrote:
On Thursday 07 September 2006 22:29, Joel VanderWerf wrote:
Pau Garcia i Quiles wrote:
Why won't you use Net::SFTP instead of calling the external scp?
I can guess one reason. Net::SFTP doesn't work with key-based
authentication, only passwords, and there is no password given in OP's
code....I hope that I am wrong and that this has changed recently.
Actually, it works with key-based authentication:
Hey, it does work! Thanks.
--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
Pau Garcia i Quiles wrote:
Pau Garcia i Quiles wrote:
> Why won't you use Net::SFTP instead of calling the external scp?I can guess one reason. Net::SFTP doesn't work with key-based
authentication, only passwords, and there is no password given in OP's
code....I hope that I am wrong and that this has changed recently.
Actually, it works with key-based authentication:
require 'net/ssh'
require 'net/sftp'Net::SSH.start( '192.168.1.37',
:username=>'pgquiles'
) do |session|session.sftp.connect do |sftp|
sftp.put_file("backup.tar.bz2",
"/media/hda3/backup.tar.bz2")
end
end
I get this error, not sure why?
---- code -------------
require 'rubygems'
require_gem 'net-ssh'
require_gem 'net-sftp'
Net::SSH.start( 'host','user','password') do |session|
session.sftp.connect do |sftp|
sftp.put_file("#{log}", "#{remotesqlfolder}/#{date_folder}.log")
end
end
---- end of code ------
--- error -------------
/usr/lib/ruby/gems/1.8/gems/net-ssh-1.0.10/lib/net/ssh/userauth/services.rb:129:
warning: `*' interpreted as argument prefix
/usr/lib/ruby/gems/1.8/gems/net-ssh-1.0.10/lib/net/ssh/connection/driver.rb:74:
warning: `&' interpreted as argument prefix
/usr/lib/ruby/gems/1.8/gems/net-ssh-1.0.10/lib/net/ssh/service/process/open.rb:130:
warning: `&' interpreted as argument prefix
/usr/lib/ruby/gems/1.8/gems/net-ssh-1.0.10/lib/net/ssh/service/process/open.rb:131:
warning: `&' interpreted as argument prefix
/usr/lib/ruby/gems/1.8/gems/net-ssh-1.0.10/lib/net/ssh/service/process/open.rb:139:
warning: `&' interpreted as argument prefix
/usr/lib/ruby/gems/1.8/gems/net-ssh-1.0.10/lib/net/ssh/service/process/open.rb:140:
warning: `&' interpreted as argument prefix
/usr/lib/ruby/gems/1.8/gems/net-ssh-1.0.10/lib/net/ssh/service/process/open.rb:141:
warning: `&' interpreted as argument prefix
/usr/lib/ruby/gems/1.8/gems/net-ssh-1.0.10/lib/net/ssh/service/process/open.rb:142:
warning: `&' interpreted as argument prefix
/usr/lib/ruby/gems/1.8/gems/net-ssh-1.0.10/lib/net/ssh/service/process/popen3.rb:143:
warning: `&' interpreted as argument prefix
/usr/lib/ruby/gems/1.8/gems/net-ssh-1.0.10/lib/net/ssh/service/process/popen3.rb:162:
warning: `&' interpreted as argument prefix
/usr/lib/ruby/gems/1.8/gems/net-sftp-1.1.0/lib/net/sftp/protocol/driver.rb:105:
warning: `&' interpreted as argument prefix
/usr/lib/ruby/gems/1.8/gems/net-sftp-1.1.0/lib/net/sftp/protocol/driver.rb:106:
warning: `&' interpreted as argument prefix
/usr/lib/ruby/gems/1.8/gems/net-ssh-1.0.10/lib/net/ssh/connection/driver.rb:378:
warning: instance variable @processing_data_requests not initialized
/usr/lib/ruby/gems/1.8/gems/net-ssh-1.0.10/lib/net/ssh/connection/channel.rb:495:
warning: instance variable @on_window_adjust not initialized
/usr/lib/ruby/gems/1.8/gems/net-sftp-1.1.0/lib/net/sftp/operations/abstract.rb:64:
warning: `*' interpreted as argument prefix
--- end of error ------
On Thursday 07 September 2006 22:29, Joel VanderWerf wrote:
--
Posted via http://www.ruby-forum.com/\.
unknown wrote:
system "scp #{ src } #{ dst } >#{ log } 2>&1"
are you on windows by any chance?
no, unix
I did that and i got an empty log.
I also rerun the script without deleting the sql file so it appended to
the file like it should. When i looked at the destination file i noticed
the file stopped at the same spot. But when i look up further to the top
of the destinatio file i noticed it had the a complete content of the
sql file followed by an incomplete content of the sql file. Ofcourse the
source has content of two complete sql files appended in one.
Seid Rudy wrote:
Pau Garcia i Quiles wrote:
Pau Garcia i Quiles wrote:
> Why won't you use Net::SFTP instead of calling the external scp?I can guess one reason. Net::SFTP doesn't work with key-based
authentication, only passwords, and there is no password given in OP's
code....I hope that I am wrong and that this has changed recently.
Actually, it works with key-based authentication:
---- code -------------
require 'rubygems'require_gem 'net-ssh'
require_gem 'net-sftp'Net::SSH.start( 'host','user','password') do |session|
session.sftp.connect do |sftp|
sftp.put_file("#{log}", "#{remotesqlfolder}/#{date_folder}.log")
end
end
---- end of code ------
--- error -------------
On Thursday 07 September 2006 22:29, Joel VanderWerf wrote:
-----------------------
Even though it gave me bunch of warnings it copied the file. BUT i still
have the same problem and that is it chops from the bottom of the
destination file (N) bytes.
i have tried
`scp -r #{sql_folder} #{remotelogin}:#{remotesqlfolder}/`
system("scp -r #{sql_folder} #{remotelogin}:#{remotesqlfolder}/")
system "scp -r #{sql_folder} #{remotelogin}:#{remotesqlfolder}"
and the aboth
All have the same problem @ the same spot of the file.
Does anybody got any idea i can try?
--
Posted via http://www.ruby-forum.com/\.
Seid Rudy wrote:
unknown wrote:
system "scp #{ src } #{ dst } >#{ log } 2>&1"
are you on windows by any chance?
no, unix
I did that and i got an empty log.
I also rerun the script without deleting the sql file so it appended to
the file like it should. When i looked at the destination file i noticed
the file stopped at the same spot. But when i look up further to the top
of the destinatio file i noticed it had the a complete content of the
sql file followed by an incomplete content of the sql file. Ofcourse the
source has content of two complete sql files appended in one.
I DON'T THINK IT'S scp ISSUE.
i get the same problem when i do this
system("cat #{sqloutput}/#{date_folder}.sql
#{sqloutput}/#{date_folder}.jnk >#{sqloutput}/#{date_folder}.out")
hmmm. are you sure you don't have disk issues?
-a
On Sat, 9 Sep 2006, Seid Rudy wrote:
Seid Rudy wrote:
unknown wrote:
On Sat, 9 Sep 2006, Seid Rudy wrote:
system "scp #{ src } #{ dst } >#{ log } 2>&1"
are you on windows by any chance?
no, unix
I did that and i got an empty log.
I also rerun the script without deleting the sql file so it appended to
the file like it should. When i looked at the destination file i noticed
the file stopped at the same spot. But when i look up further to the top
of the destinatio file i noticed it had the a complete content of the
sql file followed by an incomplete content of the sql file. Ofcourse the
source has content of two complete sql files appended in one.I DON'T THINK IT'S scp ISSUE.
i get the same problem when i do thissystem("cat #{sqloutput}/#{date_folder}.sql
#{sqloutput}/#{date_folder}.jnk >#{sqloutput}/#{date_folder}.out")
--
what science finds to be nonexistent, we must accept as nonexistent; but what
science merely does not find is a completely different matter... it is quite
clear that there are many, many mysterious things.
- h.h. the 14th dalai lama