# Image decompression, eruby **URL:** \ **Category:** ruby-talk\ **Created:** [5 January 2005 23:01 UTC](https://rubytalk.org/t/image-decompression-eruby/15620 "2005-01-05T23:01:06Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![Belorion](https://avatars.discourse-cdn.com/v4/letter/b/ea666f/32.png) [@Belorion](https://rubytalk.org/u/Belorion)\ **Post date:** [5 January 2005 23:01 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/1 "2005-01-05T23:01:06Z")
I am working on a website with a MySQL backend. The site allows users to upload files, such as jpg, gif, etc. I'm having issues loading images from the database and getting them to display in the webbrowser. Since the raw image encoding contains characters MySQL doesn't like when doing an INSERT, I encoded the image using Base64.encode64(). So, when I pulled the data back out, I do a Base64.decode64(). I also do a: print cgi.header( "type=\>image/jpg" ) so I have the correct header, and then I dump the data by simply using \<%= data %\> as the only information that is sent to the browser other than print cgi.header. However, when I load the image, I get browser errors. In Firefox, it knows it is image/jpg, but I get "The image (image path) cannot be displayed, because it contains errors." And IE6 completely barfs, asking to save the file because it doesn't know what type it is (and saving the file as a jpg and then opening that up does not work) What am I doing wrong? Does encoding/decoding in Base64 totally screw this up? Any other recommended approaches? As I mentioned, Firefox is aware of the fact it is image/jpg, but IE does not. In either case, the raw data is not a properly encoded jpeg. ---
**Author:** ![Carlos](https://avatars.discourse-cdn.com/v4/letter/c/ecae2f/32.png) [@Carlos](https://rubytalk.org/u/Carlos)\ **Post date:** [5 January 2005 23:24 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/2 "2005-01-05T23:24:48Z")
[Belorion \, 2005-01-06 00.01 CET] > I am working on a website with a MySQL backend. The site allows > users to upload files, such as jpg, gif, etc. I'm having issues > loading images from the database and getting them to display in the > webbrowser. > > Since the raw image encoding contains characters MySQL doesn't like > when doing an INSERT, I encoded the image using Base64.encode64(). > > So, when I pulled the data back out, I do a Base64.decode64(). > > I also do a: > > print cgi.header( "type=\>image/jpg" ) so I have the correct header, Shouldn't that be cgi.header("type"=\>"image/jpg") ? > and then I dump the data by simply using \<%= data %\> as the only > information that is sent to the browser other than print cgi.header. Just a guess: check for blanks/new lines around the \<%= data %\>. ... > What am I doing wrong? Does encoding/decoding in Base64 totally screw > this up? Any other recommended approaches? As I mentioned, Firefox is > aware of the fact it is image/jpg, but IE does not. In either case, > the raw data is not a properly encoded jpeg. The other possibility is to add a header "Content-Transfer-Encoding: base64" and send the base64 encoded photo directly. Good luck. ---
**Author:** ![Kaspar\_Schiess](https://avatars.discourse-cdn.com/v4/letter/k/d78d45/32.png) [@Kaspar\_Schiess](https://rubytalk.org/u/Kaspar_Schiess)\ **Post date:** [6 January 2005 12:31 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/3 "2005-01-06T12:31:53Z")
(In response to news:a48d774d050105150167c9244@mail.gmail.com by Belorion) > Since the raw image encoding contains characters MySQL doesn't like > when doing an INSERT, I encoded the image using Base64.encode64(). I recommend storing the binary data in BLOB's or whatever that data type is called with mysql. Then you can insert data by doing       insert into mytable (blobcol) values (?) The question mark is a placeholder for the data. You can bind your binary data to it by calling DBI#execute as follows:       dbh.execute( stmt, data ) Note that there are some RDBMS' out there that disallow BLOB inserts. If this is the case, you need to first insert all non-blob columns and then go back and UPDATE your table's blob columns (selecting the id's you have just inserted). This data then comes back from a SELECT as normal binary data which is exactly (and without detour) your image. It can be sent to the browser by the following code (untested):       # image contained in +data+       require 'cgi'       c = CGI.new       c.print c.header( 'type' =\> 'image/gif', 'length' =\> data.length )       STDOUT.binmode if PLATFORM =~ /32/       c.print data The binmode part is UGLY, but there seems to be no other way to do this. happy hacking ! kaspar nb: This is a part of the DBI manual:       execute( stmt, \*bindvars ) {|statement\_handle| aBlock}       Immediately executes the SQL statement stmt after binding the values in bindvars to the placeholders in the statement. kaspar hand manufactured code - www.tua.ch/ruby ---
**Author:** ![Belorion](https://avatars.discourse-cdn.com/v4/letter/b/ea666f/32.png) [@Belorion](https://rubytalk.org/u/Belorion)\ **Post date:** [6 January 2005 15:37 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/4 "2005-01-06T15:37:23Z")
> Shouldn't that be cgi.header("type"=\>"image/jpg") ? Yes, it should, and it is in my code ๐Ÿ˜‰ > Just a guess: check for blanks/new lines around the \<%= data %\>. Nope, can't find any blank lines.... > The other possibility is to add a header "Content-Transfer-Encoding: base64" > and send the base64 encoded photo directly. Good idea. Gave it a shot, but couldn't get it to work. Same errors. > I recommend storing the binary data in BLOB's or whatever that data type is > called with mysql. Then you can insert data by doing >         insert into mytable (blobcol) values (?) > > The question mark is a placeholder for the data. You can bind your binary > data to it by calling DBI#execute as follows: >         dbh.execute( stmt, data ) I hadn't seen that before. I gave it a shot. Seems to work fine for getting the browser to display an image. I take that back, dbh.do( stmt, data ) works fine, I get an error with dbh.execute. However, the image is coming back with glaring errors. After further investigation it appears that the raw data in the MySQL table has issues, though I can't tell if it's from the way the data is inserted, or if is corrupted from the actual upload. So my first question regarding displaying the image via eruby in my browser has been answered (thanks!), but maybe someone has an idea as to why my data is being corrupted (the image comes back with a ton of artifacts, especially with strange bars of color). The way I am uploading my file is derived from a script I found at [http://www.zytrax.com/tech/lang/ruby/#upload\](http://www.zytrax.com/tech/lang/ruby/#upload%5C). Specifically, my HTML for uploading looks like: \
\\> \ \ And my upload specific ruby in attach.rhtml looks like: fromfile = cgi.params['myfile'].first fromfile.binmode # Tried both with and without this line dbh.do( "INSERT INTO attachments VALUES( ? )", [ index, fromfile.original\_filename, fromfile.read] ) So, somewhere in that process the data gets corrupted, and the image comes out half garbled. Any ideas? Plain text files, which are handled using the exact same script, come out just fine. ---
**Author:** ![Carlos](https://avatars.discourse-cdn.com/v4/letter/c/ecae2f/32.png) [@Carlos](https://rubytalk.org/u/Carlos)\ **Post date:** [6 January 2005 16:06 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/5 "2005-01-06T16:06:47Z")
[Belorion \, 2005-01-06 16.37 CET] [...] > Specifically, my HTML for uploading looks like: > > \ action='attach.rhtml' method='post'\> > \\> > \ > \ > > And my upload specific ruby in attach.rhtml looks like: > > fromfile = cgi.params['myfile'].first > fromfile.binmode # Tried both with and without this line > dbh.do( "INSERT INTO attachments VALUES( ? )", [ index, > fromfile.original\_filename, fromfile.read] ) > > So, somewhere in that process the data gets corrupted, and the image > comes out half garbled. Any ideas? Plain text files, which are > handled using the exact same script, come out just fine. I don't have any idea about dbi or mysql... Attacking the problem by the web side :), maybe the upload got interrupted and you are getting a truncated file. If you add a parameter like   \ at the end of your form, you can check it to know if all the previous fields arrived whole. At least, if that's not the problem, you'll have a variable less to ponder (and it is a good general tip, too :). ---
**Author:** ![Osuka\_Adartse](https://avatars.discourse-cdn.com/v4/letter/o/e9c0ed/32.png) [@Osuka\_Adartse](https://rubytalk.org/u/Osuka_Adartse)\ **Post date:** [6 January 2005 17:30 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/6 "2005-01-06T17:30:51Z")
Belorion wrote: > > Shouldn't that be cgi.header("type"=\>"image/jpg") ? > > Yes, it should, and it is in my code ๐Ÿ˜‰ > > > Just a guess: check for blanks/new lines around the \<%= data %\>. > > Nope, can't find any blank lines.... > > > The other possibility is to add a header "Content-Transfer-Encoding: base64" > > and send the base64 encoded photo directly. > > Good idea. Gave it a shot, but couldn't get it to work. Same errors. > > > I recommend storing the binary data in BLOB's or whatever that data type is > > called with mysql. Then you can insert data by doing > >        insert into mytable (blobcol) values (?) > > > > The question mark is a placeholder for the data. You can bind your binary > > data to it by calling DBI#execute as follows: > >        dbh.execute( stmt, data ) > > I hadn't seen that before. I gave it a shot. Seems to work fine for > getting the browser to display an image. I take that back, dbh.do( > stmt, data ) works fine, I get an error with dbh.execute. However, > the image is coming back with glaring errors. After further > investigation it appears that the raw data in the MySQL table has > issues, though I can't tell if it's from the way the data is inserted, > or if is corrupted from the actual upload. > > So my first question regarding displaying the image via eruby in my > browser has been answered (thanks!), but maybe someone has an idea as > to why my data is being corrupted (the image comes back with a ton of > artifacts, especially with strange bars of color). > > The way I am uploading my file is derived from a script I found at > [Tech Stuff - Ruby](http://www.zytrax.com/tech/lang/ruby/#upload%5C). > > Specifically, my HTML for uploading looks like: > > \ action='attach.rhtml' method='post'\> > \\> > \ > \ > > And my upload specific ruby in attach.rhtml looks like: > > fromfile = cgi.params['myfile'].first > fromfile.binmode # Tried both with and without this line > dbh.do( "INSERT INTO attachments VALUES( ? )", [ index, > fromfile.original\_filename, fromfile.read] ) > > So, somewhere in that process the data gets corrupted, and the image > comes out half garbled. Any ideas? Plain text files, which are > handled using the exact same script, come out just fine. let me guess! The image starts OK but then horizontal lines start to appear with degenerating colors? if so the data in DB is probably OK as long as you use MySql::quote(img\_data) at insert time. Ok try this, select the image data and write it to disk,just an idea since I had once a similar problem. res=@my.query("select pik from test where idpiks=1;") res.each{|r| img=r[0]} #well r[0] it's the actual image. File.open("e:/temp/test.jpg","wb").write(img) now compare it, if it's OK then no upload problems or in db but at "browser display". If not make sure the #quote method is being used. how I displayed the image imager.rb ... res=@my.query("select pik from test where idpiks=#{cgi.params['CID']};") res.each{|r| img=r[0]}    puts "Content-Type: image/jpeg"    puts "Content-Length: #{img.size}"    puts "Content-Title:MyTest.jpg"    puts    $stdout.binmode \<\< img #this is the culprit in windows, notice 'wb' mode in File#Write so #binmode, it drove me nuts back then, the answer was obvious... } hope it helps Adartse ---
**Author:** ![Kaspar\_Schiess](https://avatars.discourse-cdn.com/v4/letter/k/d78d45/32.png) [@Kaspar\_Schiess](https://rubytalk.org/u/Kaspar_Schiess)\ **Post date:** [7 January 2005 11:38 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/7 "2005-01-07T11:38:06Z")
(In response to news:a48d774d0501060737414a8a92@mail.gmail.com by Belorion) > So my first question regarding displaying the image via eruby in my > browser has been answered (thanks!), but maybe someone has an idea as > to why my data is being corrupted (the image comes back with a ton of > artifacts, especially with strange bars of color). I remember having the same effect and would like to follow up on your 'solution'-post by saying that it is definitely possible to store images 1:1 in a suited mysql datatype. I remember having to tweak different things:       - Upload, for it was only sending filename first. This is common, and related to the form not being 'multipart' or so.       - Storage, related to not using ? placeholders at first (like you) and then related to mysql dropping the connection after a number of bytes transferred which need to be set to a more reasonable (=REAL BIG) value. This is probably happening to you. This issue surfaces both ways, sending data to mysql (solution here is placeholders) and receiving, so either the INSERT inserts not all data, or the SELECT does not fetch it entirely. Note that I don't really recommend the use of #quote on such data, since it has been known to fail and violate the DBI abstraction. Placeholders are really the way to go.       - Output, the usual binary/non-binary output CRUFT that windows forces you to occupy your mind with. I think the solution to that problem has been posted several times. The b) part of the Storage Problem can be solved by fiddling with LongReadLen, LongTruncOk options on the DBI handle. More details on this in the Perl (yes I said the word) manual page on DBI. (Perl documentation rocks, perhaps because the language is not as easy to read back.) But of course once you got it working, I don't expect you to change your solution again. This is more for posteriority. my best regards, kaspar hand manufactured code - www.tua.ch/ruby ---
**Author:** ![Belorion](https://avatars.discourse-cdn.com/v4/letter/b/ea666f/32.png) [@Belorion](https://rubytalk.org/u/Belorion)\ **Post date:** [6 January 2005 16:11 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/8 "2005-01-06T16:11:09Z")
> I don't have any idea about dbi or mysql... > > Attacking the problem by the web side :), maybe the upload got interrupted > and you are getting a truncated file. > > If you add a parameter like > >         \ > > at the end of your form, you can check it to know if all the previous fields > arrived whole. At least, if that's not the problem, you'll have a variable > less to ponder (and it is a good general tip, too :). I just tried dumping fromfile.read to a file and then viewing the file. It turns out that the data at that point in time is \*fine\*. So I know the data is arriving whole. So, it has something to do specifically with inserting the binary data into MySQL, which I suppose is beyond the normal scope of this list (though I won't discourage any more suggestions!) I am going to do some digging on binary data in MySQL, but if anyone has any more suggestions I'd love to hear them. ---
**Author:** ![Belorion](https://avatars.discourse-cdn.com/v4/letter/b/ea666f/32.png) [@Belorion](https://rubytalk.org/u/Belorion)\ **Post date:** [6 January 2005 17:38 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/9 "2005-01-06T17:38:17Z")
> let me guess! The image starts OK but then horizontal lines start to > appear with degenerating colors? if so the data in DB is probably OK as > long as you use MySql::quote(img\_data) at insert time. That was precisely the problem! > Ok try this, select the image data and write it to disk,just an idea > since I had once a similar problem. > > res=@my.query("select pik from test where idpiks=1;") > res.each{|r| img=r[0]} #well r[0] it's the actual image. > File.open("e:/temp/test.jpg","wb").write(img) > > now compare it, if it's OK then no upload problems or in db but at > "browser display". If not make sure the #quote method is being used. > > how I displayed the image > imager.rb > ... > res=@my.query("select pik from test where idpiks=#{cgi.params['CID']};") > res.each{|r| img=r[0]} >    puts "Content-Type: image/jpeg" >    puts "Content-Length: #{img.size}" >    puts "Content-Title:MyTest.jpg" >    puts >    $stdout.binmode \<\< img #this is the culprit in windows, notice 'wb' > mode in File#Write so #binmode, it drove me nuts back then, the answer > was obvious... > } > > hope it helps > Adartse Thanks for that solution as well. I currently have it working using Base64 encoding, so I may not switch it over, but thanks anyway. And thanks to all for their help. ---
**Author:** ![Belorion](https://avatars.discourse-cdn.com/v4/letter/b/ea666f/32.png) [@Belorion](https://rubytalk.org/u/Belorion)\ **Post date:** [12 January 2005 21:55 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/10 "2005-01-12T21:55:06Z")
>         - Output, the usual binary/non-binary output CRUFT that windows > forces you to occupy your mind with. I think the solution to that problem > has been posted several times. > # image contained in +data+ >       require 'cgi' >     c = CGI.new >      c.print c.header( 'type' =\> 'image/gif', 'length' =\> data.length ) >       STDOUT.binmode if PLATFORM =~ /32/ >       c.print data\> Okay ... so everything was working fine and dandy in my Firefox browser on windows. However, I still cannot convince IE6 that the file is of type jpeg. When I click on the link for the image, IE6 doesn't know the file type (it asks to save the file of unknown type). I've tried this with and without STDOUT.binmode. Firefox works in either case, properly identifying it as content type: image/jpg... but windows does not. The really funny part is if I screw up and put a typo in my cgi script, I get a server error (as expected), but when I hit refresh after fixing the typo the image pops up just fine. But, that is the \*only\* time I can get IE6 to display the image. Any suggestions? ---
**Author:** ![Roland\_Schmit](https://avatars.discourse-cdn.com/v4/letter/r/ecb155/32.png) [@Roland\_Schmit](https://rubytalk.org/u/Roland_Schmit)\ **Post date:** [6 January 2005 16:24 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/11 "2005-01-06T16:24:36Z")
Hi, Belorion schrieb: > > I don't have any idea about dbi or mysql... > > > > Attacking the problem by the web side :), maybe the upload got interrupted > > and you are getting a truncated file. > > > > If you add a parameter like > > > >        \ > > > > at the end of your form, you can check it to know if all the previous fields > > arrived whole. At least, if that's not the problem, you'll have a variable > > less to ponder (and it is a good general tip, too :). > > I just tried dumping fromfile.read to a file and then viewing the > file. It turns out that the data at that point in time is \*fine\*. So I know the data is arriving whole. So, it has something to do > specifically with inserting the binary data into MySQL, which I > suppose is beyond the normal scope of this list (though I won't > discourage any more suggestions!) I am going to do some digging on > binary data in MySQL, but if anyone has any more suggestions I'd love > to hear them. you can try to convert the binary data to base64 and then store the base64-data in the database: require "base64" ... b64\_data = Base64.encode64(binary\_data) # store to db ... # read from db binary\_data = Base64.decode64(b64\_data) ... Obviously you must convert back after reading the data from the database ๐Ÿ˜‰ For me base64-formatted data is easier to handle when looking for differences between two files. Regards, Roland ---
**Author:** ![Kaspar\_Schiess](https://avatars.discourse-cdn.com/v4/letter/k/d78d45/32.png) [@Kaspar\_Schiess](https://rubytalk.org/u/Kaspar_Schiess)\ **Post date:** [12 January 2005 23:24 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/12 "2005-01-12T23:24:40Z")
(In response to news:a48d774d05011213552552e222@mail.gmail.com by Belorion) > The really funny part is if I screw up and put a typo in my cgi > script, I get a server error (as expected), but when I hit refresh > after fixing the typo the image pops up just fine. But, that is the > \*only\* time I can get IE6 to display the image. Any suggestions? Yes, although this is not a 'solution'-type suggestion but rather a 'how to progress from here'-type suggestion: Get yourself a decent debugging proxy (LiveHTTPHeaders for Firefox, httpdebug.pl in Perl, any other) that allows you to inspect headers sent from and to IE when accessing REAL images. Then compare with what you send and make up for the lack of headers. Sometimes it has helped to create cgi's that are called cgi.jpg, cgi.png, etc.. to fool browsers with a simpler logic into believing the image. This can be hard to get to work with your webserver though. Tell me if this works out, I may have the time to look at this in more detail. kaspar hand manufactured code - www.tua.ch/ruby ---
**Author:** ![Belorion](https://avatars.discourse-cdn.com/v4/letter/b/ea666f/32.png) [@Belorion](https://rubytalk.org/u/Belorion)\ **Post date:** [6 January 2005 16:36 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/13 "2005-01-06T16:36:44Z")
I've tried that once with no success, but maybe I'll give it a shot again. Although, it occurrs to me... my "test" which told me my MySQL data was bad may have been incorrect. What, precisely, is the best/correct way to write out binary data? Basically, what I was doing was: sth = dbh.prepare( "SELECT content FROM attachment WHERE index=41" ) sth.execute data = sth.fetch[0] f = File.new( "test.jpg", "w+" ) f \<\< data f.close Is that correct way to write out binary data? > **ยทยทยท** > > On Fri, 7 Jan 2005 01:24:36 +0900, Roland Schmitt \ wrote: > > > Hi, > > you can try to convert the binary data to base64 and then store the > > base64-data in the database: > > > > require "base64" > > ... > > b64\_data = Base64.encode64(binary\_data) > > # store to db > > ... > > # read from db > > binary\_data = Base64.decode64(b64\_data) > > ... > > > > Obviously you must convert back after reading the data from the database > >   ;-) > > > > For me base64-formatted data is easier to handle when looking for > > differences between two files. > > > > Regards, > > Roland ---
**Author:** ![Belorion](https://avatars.discourse-cdn.com/v4/letter/b/ea666f/32.png) [@Belorion](https://rubytalk.org/u/Belorion)\ **Post date:** [12 January 2005 23:30 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/14 "2005-01-12T23:30:33Z")
Thanks for the suggestion. My problem, it turns out, was much simpler. I was using a content type of 'image/jpg' when really IE6 wants to see 'image/jpeg'. Which I suppose makes sense, since jpeg is the correct mime notation... thanks for the help though! ---
**Author:** ![Belorion](https://avatars.discourse-cdn.com/v4/letter/b/ea666f/32.png) [@Belorion](https://rubytalk.org/u/Belorion)\ **Post date:** [6 January 2005 16:47 UTC](https://rubytalk.org/t/image-decompression-eruby/15620/15 "2005-01-06T16:47:05Z")
I feel dumb. the Base64 was one of the first approaches I used. I just tried it again and it worked. I'm not sure what I am doing right this time that I was doing wrong the last... but it works now! Thanks everyone for the help! And, for anyone searching Ruby talk for a binary file upload solution using MySQL, here ya go: HTML: \ \ \ In attach.rhtml, the relavent code for insertion into MySQL is: require 'cgi' require 'base64' require 'dbi' require 'stringio' # not sure if you need this myfile = cgi.params['myfile'].first dbh = DBI.connect( "DBI:Mysql:userdb", "user", "psswd" ) [dbh.do](http://dbh.do)( "INSERT INTO table VALUES( ? )", Base64.encode64(myfile.read) ] ) dbh.disconnect And then, to view the file using eruby: \<% require 'cgi'     require 'dbi'     require 'base64'          cgi = CGI.new          dbh = DBI.connect( "DBI:Mysql:userdb", "user", "psswd" )     sth = dbh.prepare( " (statement to select out your data) "     sth.execute     data = sth.fetch[0] # assuming your binary data is the only thing returned          print cgi.header("type"=\>"image/jpg", "length"=\>data.length)     cgi.print Base64.decode64(data) %\>