DBI/OCI8 & binary data

Env.: ruby 1.6.8, ruby-dbi 0.18, ruby-oci8 0.1.2.
Oracle 9.2.0.1.0, Red-hat 7.2.

I’m trying to insert binary data into a BLOB field and it always with
either

/usr/lib/ruby/1.6/oci8.rb:300: `ORA-00933: SQL command not properly ended’ (OCIError)

or with

/usr/lib/ruby/1.6/oci8.rb:239: `Not supported type (DBI::Binary)’ (ArgumentError)

The former is when I try to insert direct binary data stored in a string
and the latter when I use DBI::Binary…

Any idea?

Code generating the second error:

···

-=-=-
matricule = argv[0].to_s
path = LOGO_PATH + argv[1].to_s
begin
image_raw = File.open(path).read
image = DBI::Binary.new(image_raw)
rescue => msg
$stderr.puts(“Error: #{msg}”)
exit 1
end

req = <<-"EOR"
insert into prs_photo (mime_type, object) values (?,?)
where c_matricule=?
EOR

begin
$dbh.transaction do
sth = $dbh.prepare(req)
sth.bind_param(1, “image/jpeg”)
sth.bind_param(2, image, type => OCI8::RAW)
sth.bind_param(3, matricule)
sth.execute
end
rescue DBI::DatabaseError => err
$stderr.puts "Error: #{err.err} #{err.errstr}"
exit 1
end
-=-=-

Ollivier ROBERT -=- Eurocontrol EEC/AMI -=- roberto@eurocontrol.fr
Usenet Canal Historique FreeBSD: The Power to Serve!

Hi,

Ollivier Robert roberto@REMOVETHIS.eu.org writes:

Env.: ruby 1.6.8, ruby-dbi 0.18, ruby-oci8 0.1.2.
Oracle 9.2.0.1.0, Red-hat 7.2.

I’m trying to insert binary data into a BLOB field and it always with
either

/usr/lib/ruby/1.6/oci8.rb:300: `ORA-00933: SQL command not properly ended’ (OCIError)

(snip)

image_raw = File.open(path).read
image = DBI::Binary.new(image_raw)

I re-read my codes. I forgot many of them. :slight_smile:
DBI::Binary is not support by DBD::OCI8.
Please use binary data stored in a string.

req = <<-“EOR”
insert into prs_photo (mime_type, object) values (?,?)
where c_matricule=?
EOR

Do you want to do ‘update’ or ‘insert’?

If you want to do ‘update’:
update prs_photo set mime_type = ?, object = ?
where c_matricule = ?

If you want to do ‘insert’:
insert into prs_photo (mime_type, object, c_matricule) values (?,?,?)

‘insert’ with ‘where’ clause is not allowed.

 sth.bind_param(2, image, type => OCI8::RAW)

Please change to
sth.bind_param(2, image, type => DBI::SQL_BINARY)

But I have not tested your code and my modified code yet. :-<
I’ve just read and thought. I’ll test them on next weekend.

Cheers.

···


KUBO Takehiro
kubo@jiubao.org

In article 198Luz-0DA-00@mail.jiubao.org,

I re-read my codes. I forgot many of them. :slight_smile:
DBI::Binary is not support by DBD::OCI8.
Please use binary data stored in a string.

Right.

‘insert’ with ‘where’ clause is not allowed.

You’re right, brainfart on my side.

 sth.bind_param(2, image, type => OCI8::RAW)

Please change to
sth.bind_param(2, image, type => DBI::SQL_BINARY)

Hmmm, now it is failing with that:

/usr/lib/ruby/1.6/oci8.rb:300: ORA-01461: can bind a LONG value only for insert into a LONG column' (OCIError) from /usr/lib/ruby/1.6/oci8.rb:300:in exec’
from /usr/lib/ruby/1.6/DBD/OCI8/OCI8.rb:121:in execute' from /usr/lib/ruby/site_ruby/1.6/dbi/dbi.rb:743:in execute’
from insert-photo.rb:67:in main' from insert-photo.rb:62:in transaction’
from insert-photo.rb:62:in `main’
from insert-photo.rb:77

Putting DBI::SQL_BINARY, DBI::SQL_BLOB or nothing doesn’t change the error
code.

But I have not tested your code and my modified code yet. :-<
I’ve just read and thought. I’ll test them on next weekend.

Arigato Kubo-san.

···

KUBO Takehiro kubo@jiubao.org wrote:

-=-=-
#! /usr/bin/env ruby

$Id: //depot/caerdonn/roberto/src/ruby/eproject/test-ba.rb#3 $

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Misc. VCS stuff

RCS_ID = %q$Id$
RCS_REV = (RCS_ID.split[1]).to_s.split(/#/)[1]
MYNAME = File.basename($0)

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Non standard packages

require “oci8”
require “dbi”

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

XXX CONFIG XXX

DBD = “OCI8”
DB = “AMITEST”
USER = “oradmin”
PASS = “*********”

LOGO_PATH = “…/”

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Global variables

$dbh = nil

def connect_acb
begin
dbh = DBI.connect(“dbi:#{DBD}:#{DB}”, USER, PASS)
$stderr.puts(“Connect to #{DB}”)
rescue DBI::DatabaseError => err
$stderr.puts(“Error: #{err.errstr}”)
exit 1
end
return dbh
end

def main(argv)
dbh = connect_acb
mime_type = “image/jpeg”

matricule = argv[0].to_s
path = LOGO_PATH + argv[1].to_s
begin
image_raw = File.open(path).read
rescue => msg
$stderr.puts(“Error: #{msg}”)
exit 1
end

req = <<-“EOR”
insert into prs_photo (c_matricule, mime_type, object) values (?,?,?)
EOR

begin
dbh.transaction do
sth = dbh.prepare(req)
sth.bind_param(1, matricule)
sth.bind_param(2, “image/jpeg”)
sth.bind_param(3, image_raw) # or :type => DBI::SQL_BINARY
sth.execute
end
rescue DBI::DatabaseError => err
$stderr.puts “Error: #{err.err} #{err.errstr}”
exit 1
end
return 0
end

if $0 == FILE then
exit(main(ARGV) || 1)
end
-=-=-

Ollivier ROBERT -=- Eurocontrol EEC/ITM -=- roberto@eurocontrol.fr
Usenet Canal Historique FreeBSD: The Power to Serve!

Ollivier Robert roberto@REMOVETHIS.eu.org writes:

Please change to
sth.bind_param(2, image, type => DBI::SQL_BINARY)

Hmmm, now it is failing with that:

/usr/lib/ruby/1.6/oci8.rb:300: ORA-01461: can bind a LONG value only for insert into a LONG column' (OCIError) from /usr/lib/ruby/1.6/oci8.rb:300:in exec’
from /usr/lib/ruby/1.6/DBD/OCI8/OCI8.rb:121:in execute' from /usr/lib/ruby/site_ruby/1.6/dbi/dbi.rb:743:in execute’
from insert-photo.rb:67:in main' from insert-photo.rb:62:in transaction’
from insert-photo.rb:62:in `main’
from insert-photo.rb:77

Putting DBI::SQL_BINARY, DBI::SQL_BLOB or nothing doesn’t change the error
code.

I ran your code. But there was no error in my environment. It is
almost same with your environment. I couldn’t guess the reason.
But even though no error, the stored data was truncated (about 7800
bytes).

I’ve added BLOB locator support to Ruby/OCI8 and DBD::OCI8.
Please get from:
http://www.jiubao.org/ruby-oci8/ruby-oci8-0.1.3-pre1.tar.gz

I’ll put ruby-oci8-0.1.3.tar.gz in a few days.

req = <<-“EOR”
insert into prs_photo (c_matricule, mime_type, object) values (?,?,?)
EOR

begin
dbh.transaction do
sth = dbh.prepare(req)
sth.bind_param(1, matricule)
sth.bind_param(2, “image/jpeg”)
sth.bind_param(3, image_raw) # or :type => DBI::SQL_BINARY
sth.execute
end
rescue DBI::DatabaseError => err
$stderr.puts “Error: #{err.err} #{err.errstr}”
exit 1
end
return 0
end

To use BLOB support of Ruby/OCI8, modify above code as following.

···

req = <<-“EOR”
insert into prs_photo (c_matricule, mime_type, object) values (?,?, EMPTY_BLOB())
EOR

blob_sql = <<-“EOR”
select object from prs_photo where rowid = ?
EOR

begin
dbh.transaction do
# insert one row.
sth = dbh.execute(req, matricule, “image/jpeg”)
# get the rowid of inserted row.
rowid = sth.func(:rowid) # call driver specific code.
# get a BLOB locator
loc = dbh.select_one(blob_sql, rowid)[0] # 1st row, 1st column
# write data to the locator.
loc.write(image_raw)
end
rescue DBI::DatabaseError => err
$stderr.puts “Error: #{err.err} #{err.errstr}”
exit 1
end

To insert BLOB data:
1st: insert ‘EMPTY_BLOB()’ to the BLOB column.
2nd: get rowid of the inserted row.
3rd: select the inserted BLOB column as a BLOB locator.
4th: write data to the locator.

To update BLOB data:
1st: select a BLOB column which you want to update as a BLOB locator.
2rd: write data to the locator by OCI8::BLOB#write(data).
3nd: fix the length of its content by OCI8::BLOB#truncate(data.size).
If you forget to truncate the content and old data is longer
than new one, garbage remains at the end.

To select BLOB data:
1st: select a BLOB column as a BLOB locator.
2nd: read its content by OCI8::BLOB#read

To delete BLOB data:
just delete the row. :-p

Cheers

KUBO Takehiro

KUBO Takehiro kubo@jiubao.org writes:

I’ve added BLOB locator support to Ruby/OCI8 and DBD::OCI8.
Please get from:
http://www.jiubao.org/ruby-oci8/ruby-oci8-0.1.3-pre1.tar.gz

I’ll put ruby-oci8-0.1.3.tar.gz in a few days.

I put ruby-oci8-0.1.3.tar.gz just now.

http://www.jiubao.org/ruby-oci8/ruby-oci8-0.1.3.tar.gz

Some new APIs are added to OCI8::BLOB and some return values are
changed from 0.1.3-pre1

API reference of OCI8::BLOB is available at

http://www.jiubao.org/ruby-oci8/api.en.html

Even though you use Ruby/DBI, BLOB column is selected as a instance of
OCI8::BLOB.

Cheers

···


KUBO Takehiro

In article 199h8R-1dU-00@mail.jiubao.org,

···

KUBO Takehiro kubo@jiubao.org wrote:

I put ruby-oci8-0.1.3.tar.gz just now.

http://www.jiubao.org/ruby-oci8/ruby-oci8-0.1.3.tar.gz

Some new APIs are added to OCI8::BLOB and some return values are
changed from 0.1.3-pre1

Thanks, I’ll test that tomorrow at work.

Domo arigato KUBO-san.

Ollivier ROBERT -=- Eurocontrol EEC/ITM -=- roberto@eurocontrol.fr
Usenet Canal Historique FreeBSD: The Power to Serve!

In article 199h8R-1dU-00@mail.jiubao.org,

···

KUBO Takehiro kubo@jiubao.org wrote:

http://www.jiubao.org/ruby-oci8/ruby-oci8-0.1.3.tar.gz

Apart from changing two «"» into «'» at line 106 of oraconf.rb, compiles
and installs fine.

Now, when I try to read a BLOB – inserted in the DB by another program –
I get « uninitialized BLOB » (see below).

Any idea?

-=-=-
# -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
# Return pathname of the logo
#
def get_logo
bytes = 0

  req = <<-"EOR"

select l_name_logo, mime_type, object_size, object from acb_logo
where id_activity=:1
EOR

  loc = nil
  begin
    sth = $dbh.prepare(req)
    sth.execute(@id_activity)
    row = sth.fetch
    loc = row[:OBJECT]
    raw_logo = loc.read             # XXX this give uninitialized BLOB
  rescue DBI::DatabaseError => err
    $stderr.puts "Error: #{err.errstr}"
    $dbh.disconnect
    exit 2
  rescue RuntimeError => msg
    $stderr.puts "Error: #{msg}"
    $dbh.disconnect
    exit 3
  end

  # Generate an external image file <id_activity>.<ext>
  # where <ext> depends on the stored Content-Type.
  #
  fname = @id_activity.to_s
  case row[:MIME_TYPE].to_s
  when /jpeg/
    fname += ".jpeg"
  when /gif/
    fname += ".gif"
  when /png/
    fname += ".png"
  else
    fname += ".gif"
  end
  full = row[:OBJECT_SIZE].to_i
  begin
    if File.exist?(LOGO_PATH) then
      if not File.directory?(LOGO_PATH)
        raise TypeError
      end
    else
      File.mkdir(LOGO_PATH)
    end
    File.open(LOGO_PATH + "/" + fname, "w+") do |ofile|
      bytes = ofile.write(raw_logo)
    end
    raise IOError if bytes != full
  rescue => msg
    $stderr.puts("Error I/O: #{msg}: #{bytes} written out of #{full}")
    $dbh.disconnect
    exit 3
  end
  return fname
end # get_logo

-=-=-

Ollivier ROBERT -=- Eurocontrol EEC/AMI -=- roberto@eurocontrol.fr
Usenet Canal Historique FreeBSD: The Power to Serve!

Ollivier Robert roberto@REMOVETHIS.eu.org writes:

In article 199h8R-1dU-00@mail.jiubao.org,

http://www.jiubao.org/ruby-oci8/ruby-oci8-0.1.3.tar.gz

Apart from changing two «"» into «'» at line 106 of oraconf.rb, compiles
and installs fine.

I’ll fix it next release.

Now, when I try to read a BLOB – inserted in the DB by another program –
I get « uninitialized BLOB » (see below).

Could your another program read a BLOB?
ruby-oci8 raises ‘uninitialized BLOB’ when OCI8::BLOB#available? is
false. OCI8::BLOB#available? is OCILobLocatorIsInit() in C. I don’t
know the portable ways to check whether the BLOB column is initialized
or not. I would read the document of LOB at the next weekend.

···

KUBO Takehiro kubo@jiubao.org wrote:


KUBO Takehiro

In article 19BFXp-2c5-00@mail.jiubao.org,

Could your another program read a BLOB?

Well, I can see data in the table with a SQL browser (TORA but Toad under
Windows see it too).

ruby-oci8 raises ‘uninitialized BLOB’ when OCI8::BLOB#available? is
false. OCI8::BLOB#available? is OCILobLocatorIsInit() in C. I don’t
know the portable ways to check whether the BLOB column is initialized
or not. I would read the document of LOB at the next weekend.

Thanks.

···

KUBO Takehiro kubo@jiubao.org wrote:

Ollivier ROBERT -=- Eurocontrol EEC/AMI -=- roberto@eurocontrol.fr
Usenet Canal Historique FreeBSD: The Power to Serve!

Ollivier Robert roberto@REMOVETHIS.eu.org writes:

In article 19BFXp-2c5-00@mail.jiubao.org,

Could your another program read a BLOB?

Well, I can see data in the table with a SQL browser (TORA but Toad under
Windows see it too).

ruby-oci8 raises ‘uninitialized BLOB’ when OCI8::BLOB#available? is
false. OCI8::BLOB#available? is OCILobLocatorIsInit() in C. I don’t
know the portable ways to check whether the BLOB column is initialized
or not. I would read the document of LOB at the next weekend.

Sorry I forgot to return nil when fetched data is null.
I think error data which raises ‘uninitialized BLOB’ is null.
Please try following tar-ball:
http://www.jiubao.org/ruby-oci8/ruby-oci8-0.1.3-test1.tar.gz

If my assumption is correct, it will raise “undefined method `read’
for nil (NameError)” instead of “uninitialized BLOB”. If so, please
check whether that data is null or not by following SQL.

SELECT id_activity FROM acb_logo WHERE object is null;

If not so, please send dump file to me.

$ exp userid=/ tables=acb_logo file=acb_logo.dmp
$ bzip2 -9 acb_logo.dmp

Please compress as small as possible. The band-wide between the
Internet and my home is extremely narrow.

···

KUBO Takehiro kubo@jiubao.org wrote:


KUBO Takehiro

In article 19C0p7-0Po-00@mail.jiubao.org,

If my assumption is correct, it will raise “undefined method `read’
for nil (NameError)” instead of “uninitialized BLOB”. If so, please
check whether that data is null or not by following SQL.

(rdb:1) p row
[“s2-logo.gif”, “image/gif”, 8443, nil]
test-ba.rb:682: raw_logo = loc.read
(rdb:1)
n
test-ba.rb:682: undefined method read’ for nil’ (NameError)
test-ba.rb:682:in get_logo': undefined method read’ for nil (NameError)
from test-ba.rb:660:in get_attrs' from test-ba.rb:758:in to_html’
from test-ba.rb:882:in `main’
from test-ba.rb:890

SELECT id_activity FROM acb_logo WHERE object is null;

If not so, please send dump file to me.

The query returns two rows which are supposed to have files.

So I’d say the images were not uploaded properly in the table. I’m looking
into it with the guy who did the import.

Thanks a lot for your help.

···

KUBO Takehiro kubo@jiubao.org wrote:

Ollivier ROBERT -=- Eurocontrol EEC/ITM -=- roberto@eurocontrol.fr
Usenet Canal Historique FreeBSD: The Power to Serve!