Hi depesz,
I've looked for things like that, but it doesn't seem like they've been
exposed in the Ruby API. If I try to make a DBI prepared statement, it
will just slurp-read the first time I call execute on it. The native
PostgreSQL-extension doesn't seem to have any classes for cursors or
prepared statements, and neither does the pure Ruby version.
Does anyone know if ActiveRecord might be a solution? Does it have it's
own implementation of a PostgreSQL connector?
Helge Elvik
···
-----Original Message-----
From: hubert depesz lubaczewski [mailto:depesz@depesz.com]
Sent: 4. august 2006 11:31
To: ruby-talk ML
Subject: Re: [postgres] Is there a way to avoid having the library
slurp-read the whole result-set?
On 2006-08-04, Helge Elvik <helge.elvik@gispartner.no> wrote:
Am I missing something, or is the really now way of receiving the
results serially without the libraries slurping everything up in RAM
first? The table I'm having trouble with currently has about 2 million
use cursor's. (in posrtgresql docs look for:
declare
fetch
move
close
depesz
--
http://www.depesz.com/index.php/2006/07/28/sprzedaje-samochod/
I've looked for things like that, but it doesn't seem like they've been
exposed in the Ruby API. If I try to make a DBI prepared statement, it
will just slurp-read the first time I call execute on it. The native
PostgreSQL-extension doesn't seem to have any classes for cursors or
prepared statements, and neither does the pure Ruby version.
Hm, I wonder why would you need special API for that?
PGconn.exec should be sufficient:
require 'postgres'
c = PGconn.new
c.exec("BEGIN")
c.exec("DECLARE my_curs CURSOR FOR SELECT * FROM generate_series(1,20000)")
while (rs = c.exec("FETCH FORWARD 1000 FROM my_curs")) && rs.num_tuples > 0
rs.each { |r| print r[0], "," }
rs.close
end
c.exec("CLOSE my_curs")
c.exec("COMMIT")
Does anyone know if ActiveRecord might be a solution? Does it have it's
own implementation of a PostgreSQL connector?
ActiveRecord just uses ruby-postgres or postgres-pr libs
···
2006/8/4, Helge Elvik <helge.elvik@gispartner.no>:
Does anyone know if ActiveRecord might be a solution? Does it have it's
own implementation of a PostgreSQL connector?
I'm on thin ice here, but a quick browsing of the postgresql_adabter.rb in
AR reads:
if @async
@connection.async_exec(sql)
else
@connection.exce(sql)
end
(lines 162..166)
So, it seems to me that AR would not solve your problem. Maybe one could
patch it with the sub-slurp posted in this thread.
···
--
Jon Egil Strand
Phone: +47 98232340
jes@luretanker.no
I've found that AR exacts such a high price in performance that I
write applications using both AR and a native driver. I use AR for all
the lightweight stuff (for the productivity benefits), and code
against the native adapter for everything heavy, like the query that
is the subject of this thread. Even using AR's capabilities for
directly executing raw SQL code is measurably slower than using the
underlying adapters. You do lose portability across database engines
with this approach, but high performance in any database generally
requires a lot of engine-specific tunings anyway.
···
On 8/5/06, Jon Egil Strand <jes@luretanker.no> wrote:
>
So, it seems to me that AR would not solve your problem. Maybe one could
patch it with the sub-slurp posted in this thread.
indeed - i sent a patch in some time ago to address this - but no joy.
-a
···
On Sat, 5 Aug 2006, Jon Egil Strand wrote:
Does anyone know if ActiveRecord might be a solution? Does it have it's
own implementation of a PostgreSQL connector?
I'm on thin ice here, but a quick browsing of the postgresql_adabter.rb in
AR reads:
if @async
@connection.async_exec(sql)
else
@connection.exce(sql)
end
(lines 162..166)
So, it seems to me that AR would not solve your problem. Maybe one could
patch it with the sub-slurp posted in this thread.
--
happiness is not something ready-made. it comes from your own actions.
- h.h. the 14th dali lama