Sysread and buffered I/O

Gavin Sinclair wrote:

[...]

>

What about this?

  1) read(n):
       current method; no problem if less than n bytes read,
       and no blocking

  2) read(n, :noeof):
       as (1), but raises EOFError on end of file

:raise_on_eof reads better for me. but may be too long.

Regards,

   Michael

In article <opsbhnakfau5o8pp@233.dallas-20rh16rt.tx.dial-access.att.net>,
  "John Feezell" <JohnFeezell@3wplace.com> writes:

Here are some other possible names that seems to fit with readchar,
readline,
and readlines and yet catch the idea of "readpartial."

readportion
readparcel
readbundle
readatmost

Personally, "readatmost" would fit closes in my mind to the above
description.

"at most" doesn't represent the difference between IO#readpartial and
IO#read. IO#read also reads at most <i>integer</i> bytes from the I/O
stream.

The difference is the blocking behaviour.

% (sleep 1; echo abc; sleep 1; echo def) | ./ruby -e 't1 = Time.now; p STDIN.readpartial(4096); t2 = Time.now; p t2-t1'
"abc\n"
1.001541
% (sleep 1; echo abc; sleep 1; echo def) | ./ruby -e 't1 = Time.now; p STDIN.read(4096); t2 = Time.now; p t2-t1'
"abc\ndef\n"
2.011527

Both IO#readpartial and IO#read blocks until some data are available.
But after some data are avalable, IO#readpartial just return the data
and doesn't block anymore. IO#read blocks until EOF or specified
length, though.

···

--
Tanaka Akira

In article <46-1183202235.20040722005816@soyabean.com.au>,
  Gavin Sinclair <gsinclair@soyabean.com.au> writes:

What about this?

  1) read(n):
       current method; no problem if less than n bytes read,
       and no blocking

  2) read(n, :noeof):
       as (1), but raises EOFError on end of file

  3) read(n, :noeof, :pblock):
       as (2), but *partially* blocks; i.e. blocks only if no data
       is immediately available

  4) read(n, :exact):
       as (1), raise some error if less than n bytes read
       (implies :noeof)

  5) read(n, :exact, :block):
       as (4), but block until n bytes are available

  6) read(n, :exact, :pblock):
       as (4), but *partially* block, as per (3)

All methods can accept a String parameter which acts as the receiving
buffer. The 'n' parameter must come first; the order of the rest
doesn't matter.

The naming of "pblock" could definitely be better...

I don't see a difference between 4th and 5th.
I can't understand 6th.

Do you have usecase for them?

I think such symbols based behavior specification tends to allow
specifications no one need. So it increase the possibility which an
user select an unsuitable combination by mistake. Also the
implementation should define meanings of all combination of symbols
and implement them, even if a combination is never used. So it waists
developper resources.

···

--
Tanaka Akira

In article <Pine.LNX.4.60.0407210648370.13231@harp.ngdc.noaa.gov>,
  "Ara.T.Howard" <ahoward@noaa.gov> writes:

why not

   IO#receive

It doesn't represent the difference from IO#read.

···

--
Tanaka Akira

In article <874qnyevv4.fsf@dessyku.is-a-geek.org>,
  "Yohanes Santoso" <ysantoso-rubytalk@dessyku.is-a-geek.org> writes:

The behaviour described above is similar to the behaviour of posix's
read function. posix's read function does a short read[1] if there is
not enough data available to satisfy the request, but perform a full
read if there is enough data. it will also block until there is some
data to be read.

It is not wonder. readpartial is a stdio friendly sysread. sysread
calls posix's read function.

I'm not sure that the name represent "stdio friendly" property well.

···

--
Tanaka Akira

Mark Firestone wrote:

That is entirely possible (;

I'm not sure if that is the way to do it or not. My SQL is not the
greatest.

Can you explain again, what exactly is your problem.
Are there performance problems with getting the whole query result set at once?

Postgres is a good choice :wink:

Regards,

   Michael

Thanks. That makes the difference clearer.
Well then, how about,

readforedata #suggest that only the "first" or "some" data is read
readandhalt #suggest reading is only partial

···

On Thu, 22 Jul 2004 16:01:06 +0900, Tanaka Akira <akr@m17n.org> wrote:

In article <opsbhnakfau5o8pp@233.dallas-20rh16rt.tx.dial-access.att.net>,
  "John Feezell" <JohnFeezell@3wplace.com> writes:

Here are some other possible names that seems to fit with readchar,
readline,
and readlines and yet catch the idea of "readpartial."

readportion
readparcel
readbundle
readatmost

Personally, "readatmost" would fit closes in my mind to the above
description.

"at most" doesn't represent the difference between IO#readpartial and
IO#read. IO#read also reads at most <i>integer</i> bytes from the I/O
stream.

The difference is the blocking behaviour.

% (sleep 1; echo abc; sleep 1; echo def) | ./ruby -e 't1 = Time.now; p STDIN.readpartial(4096); t2 = Time.now; p t2-t1'
"abc\n"
1.001541
% (sleep 1; echo abc; sleep 1; echo def) | ./ruby -e 't1 = Time.now; p STDIN.read(4096); t2 = Time.now; p t2-t1'
"abc\ndef\n"
2.011527

Both IO#readpartial and IO#read blocks until some data are available.
But after some data are avalable, IO#readpartial just return the data
and doesn't block anymore. IO#read blocks until EOF or specified
length, though.

consider

   IO.read

     actively takes characters from IO until EOF is found

   IO.receive

     passively receives any waiting characters from IO

??

-a

···

On Thu, 22 Jul 2004, Tanaka Akira wrote:

In article <Pine.LNX.4.60.0407210648370.13231@harp.ngdc.noaa.gov>,
"Ara.T.Howard" <ahoward@noaa.gov> writes:

why not

   IO#receive

It doesn't represent the difference from IO#read.
--
Tanaka Akira

--

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it;
and a weed grows, even though we do not love it. --Dogen

===============================================================================

Hi All,

JRuby list is quiet, so I was wondering if anyone here might be able to help with the following question (below).

Thanks,
Nick

···

----------------------------------------------------------------------------------------------
Hi,

I was asking about the syntax of java package includes a while back and gave this example:

require "java"
module Java
    include_package "java.util"
end
v=Java::Vector.new

In wanting to not have to type Java:: for each class, I tried a Ruby idiom I recently learned (I'm fairly new to it):

require "java"
module Java
    include_package "java.util"
end
include Java
v=Vector.new

Which doesn't work. Is this a current limitation in Jruby? Does it have something to do with the lazy-loading I heard mentioned before? Or am I missing something. I figure if this worked, it would be 10 to 20 lines of code or so to implement my own "import" method that works just like Java or Jython syntax, and that would be cool.

On a related topic, I tried to use Class.forname("...") Java idiom to load a JDBC driver (with the appropriate Module include_package and prefixes), and it failed. Is there a known limitation here?

Final note- any advocacy tips/issues for recommending JRuby over Jython?

Thanks,
Nick

Well, I've written re-written an old BBS program. Right now it holds it's
messages in individual text files (really marshaled object files).

I can't begin to tell you how much this sucks.

So, I'm moving it all to sql. I have a table which is a message board. the
messages are records. each message has a unique number used as a new
message index. each user has a pointer to that record.

I need to be able to go up and down the sql table, reading individual
messages in each direction, and be able to jump to a message by number, etc.
Right now I am making a array of the message numbers, and looking up the
messages that way.

So if I want message 4 in the list, which is really message 3219, because
the ones before it have been deleted, then I ask the array...

index_tbl[message] = whatever the number I should pull out of the table is.

I got the index_tbl by doing a select on the board table, ordered by the
message number.

Anyway, before each operation, I have to rebuild the table, because the
whole thing is multi-user... and someone else might have done something
like add or delete a message. it works fine when there are a few messages,
but I'm not sure how it will perform when it hits 1000's of messages.

My friend pointed me at postresql... it seems lean and fast.

Thanks for your help!

mark

"But Schindler is bueno! Senior Burns is El Diablo!"

···

--------------------------------------------------------------
Website - http://www.retrobbs.org
Tradewars - telnet tradewars.retrobbs.org
BBS - http://bbs.retrobbs.org:8000
IRC - irc.retrobbs.org #main
WIKI - http://www.tpoh.org/cgi-bin/tpoh-wiki

----- Original Message -----
From: "Michael Neumann" <mneumann@ntecs.de>
To: "ruby-talk ML" <ruby-talk@ruby-lang.org>
Sent: Wednesday, July 21, 2004 2:31 PM
Subject: Re: ruby postgresql question

Mark Firestone wrote:
> That is entirely possible (;
>
> I'm not sure if that is the way to do it or not. My SQL is not the
> greatest.

Can you explain again, what exactly is your problem.
Are there performance problems with getting the whole query result set
at once?

Postgres is a good choice :wink:

Regards,

   Michael

In article <opsbjixabhu5o8pp@233.dallas-20rh16rt.tx.dial-access.att.net>,
  "John Feezell" <JohnFeezell@3wplace.com> writes:

readportion
readparcel
readbundle

readforedata #suggest that only the "first" or "some" data is read
readandhalt #suggest reading is only partial

Matz, is there a name good enough?

···

--
Tanaka Akira

In article <Pine.LNX.4.60.0407220742560.5080@harp.ngdc.noaa.gov>,
  "Ara.T.Howard" <ahoward@noaa.gov> writes:

   IO.read

     actively takes characters from IO until EOF is found

   IO.receive

     passively receives any waiting characters from IO

I see.

However, another problem is it is confusing with BasicSocket#recv.

···

--
Tanaka Akira

Mark Firestone wrote:

Well, I've written re-written an old BBS program. Right now it holds it's
messages in individual text files (really marshaled object files).

I can't begin to tell you how much this sucks.

So, I'm moving it all to sql. I have a table which is a message board. the
messages are records. each message has a unique number used as a new
message index. each user has a pointer to that record.

can you describe the table layout in sql?

I need to be able to go up and down the sql table, reading individual
messages in each direction, and be able to jump to a message by number, etc.

what does "reading individual messages in _each direction_" mean? Has this something to do with the hierarchy of the messages? (In-response-To etc.)?

Right now I am making a array of the message numbers, and looking up the
messages that way.

So if I want message 4 in the list, which is really message 3219, because
the ones before it have been deleted, then I ask the array...

I don't really understand :slight_smile:

index_tbl[message] = whatever the number I should pull out of the table is.

I got the index_tbl by doing a select on the board table, ordered by the
message number.

Anyway, before each operation, I have to rebuild the table, because the
whole thing is multi-user... and someone else might have done something
like add or delete a message. it works fine when there are a few messages,
but I'm not sure how it will perform when it hits 1000's of messages.

hm, maybe it's better to use transactions and don't cache the messages on the client side.

My friend pointed me at postresql... it seems lean and fast.

yes, but maybe SQlite is even better suited for your purposes (easier to setup, single database file, no server, faster).

I probably don't understand what a BBS is, but isn't that something like a "forum" where you can post messages and respond to others' messages?

Here's how I would design the messages table:

   create table messages (
     id integer primary key,
     original_id integer, /* to be backwards compatible with old messages */
     parent integer null references messages (id),
     .... /* other attributes */
     body text
   );

Regards,

   Michael

i wouldn't chose the word 'confusing' - reading the man page of recv makes it
sound VERY close to what you want. however, using exactly 'recv' WOULD be
confusing... checking on dictionary.com for receive :

   Entry: receive
   Function: verb
   Definition: accept
   Synonyms: accept, acquire, admit, apprehend, appropriate, arrogate, assume, be given, be informed, be told, catch, collect, come by, come into, cop, corral, derive, draw, earn, gain, gather, get, get from, grab, hear, hold, inherit, make, obtain, perceive, pick up, pocket, procure, pull, pull down, reap, redeem, secure, seize, snag, take, take in, take possession, win
   Concept: taking
   Source: Roget's New Millennium™ Thesaurus, First Edition (v 1.0.5)
   Copyright © 2004 by Lexico Publishing Group, LLC. All rights reserved.

   Entry: receive

so i'd suggest

   receive
   accept
   reap

i like the sound of

   IO#reap

alot, eg. :

···

On Fri, 23 Jul 2004, Tanaka Akira wrote:

In article <Pine.LNX.4.60.0407220742560.5080@harp.ngdc.noaa.gov>,
"Ara.T.Howard" <ahoward@noaa.gov> writes:

   IO.read

     actively takes characters from IO until EOF is found

   IO.receive

     passively receives any waiting characters from IO

I see.

However, another problem is it is confusing with BasicSocket#recv.
--
Tanaka Akira

   #
   # read all chars available on pipe
   #
     buf = pipe.reap

i think this method is much needed and no name exists for something like it.
if we pick one that is memorable people will use it and remember it.

cheers.

-a
--

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it;
and a weed grows, even though we do not love it. --Dogen

===============================================================================

can you describe the table layout in sql?

Sure. Here is the sql statement for a message board table

@db.exec("CREATE TABLE #{table} (delete boolean DEFAULT false, \
           locked boolean DEFAULT false, number int PRIMARY KEY, \
            m_to varchar(40), \
           m_from varchar(40), msg_date timestamp, subject varchar(40),\
           msg_text text, exported boolean DEFAULT false)")

what does "reading individual messages in _each direction_" mean? Has
this something to do with the hierarchy of the messages? (In-response-To
etc.)?

Ok. It's an old text (telnet) based Bulletin Board System. You go to look
at messages
post since you last read them. The system find this point for you. Then
you press enter at the prompt to get the new messages, one at a time. Or
you can jump to a particular message by typing it's number, or you can go
backwards, if you like.

I don't really understand :slight_smile: [about making the table...]

Well, messages get deleted, but the message numbers in the table don't
change. The message numbers (when you are reading them) *do* change because
they are always 1..<highest message> so the system needs to know which
actual message to pull up when you type it's number in. They won't be the
same, after one message is deleted. So I am doing a ...

for row in @db.query("SELECT number FROM #{table} ORDER BY number")
  hash << row[0].to_i
end

and making an array, full of message numbers. A lookup table. So I can
look up which message to pull up.

hm, maybe it's better to use transactions and don't cache the messages
on the client side.

I'm not caching the messages, just making the lookup table. I'm concerned
about future performace problems.

yes, but maybe SQlite is even better suited for your purposes (easier to
setup, single database file, no server, faster).

I'll look into that. I've already got this thing working though... Plus, I
want to write a web front end for this one day...

I probably don't understand what a BBS is, but isn't that something like
a "forum" where you can post messages and respond to others' messages?

Yep. But text based. Have a look. Telnet to: bbs.retrobbs.org 2323 and
have a look, if you like.

Here's how I would design the messages table:

   create table messages (
     id integer primary key,
     original_id integer, /* to be backwards compatible with old
messages */
     parent integer null references messages (id),
     .... /* other attributes */
     body text
   );

that's pretty similar to what I've done. Thanks again!

Mark

"But Schindler is bueno! Senior Burns is El Diablo!"

···

----- Original Message -----
From: "Michael Neumann" <mneumann@ntecs.de>
To: "ruby-talk ML" <ruby-talk@ruby-lang.org>
Sent: Wednesday, July 21, 2004 3:42 PM
Subject: Re: ruby postgresql question

--------------------------------------------------------------
Website - http://www.retrobbs.org
Tradewars - telnet tradewars.retrobbs.org
BBS - http://bbs.retrobbs.org:8000
IRC - irc.retrobbs.org #main
WIKI - http://www.tpoh.org/cgi-bin/tpoh-wiki

In article <Pine.LNX.4.60.0407222054440.8991@harp.ngdc.noaa.gov>,
  "Ara.T.Howard" <ahoward@noaa.gov> writes:

i wouldn't chose the word 'confusing' - reading the man page of recv makes it
sound VERY close to what you want.

Yes. It is not surprised that BasicSocket#recv and IO#sysread is
similar because recv is a system call which is similar to read system call.

Also BasicSocket#recv is stdio unfriendly similar to IO#sysread:

% ruby -rsocket -e 'TCPSocket.open("www.ruby-lang.org", 80) {|s|
s.print "GET / HTTP/1.0\r\n\r\n"
p s.gets
p s.recv(100)
}'
"HTTP/1.1 302 Found\r\n"
-e:4:in `recv': recv for buffered IO (IOError)
        from -e:4
        from -e:1:in `open'
        from -e:1

I think it is confusing that assign different meaning for abbreviation
and non-abbreviation of single word: "recv" and "receive". Their
meaning doesn't contain stdio friendly/unfriendly property.

so i'd suggest

   receive
   accept
   reap

Since there is a system call "accept" and TCPServer#accept, IO#accept is
not an option.

i like the sound of

   IO#reap

alot, eg. :

   #
   # read all chars available on pipe
   #
     buf = pipe.reap

i think this method is much needed and no name exists for something like it.
if we pick one that is memorable people will use it and remember it.

I like a method name longer than 4bytes ("read".length). Because I
encourage IO#read over IO#readpartial in usual case.

···

--
Tanaka Akira

Mark Firestone wrote:

From: "Michael Neumann" <mneumann@ntecs.de>
To: "ruby-talk ML" <ruby-talk@ruby-lang.org>
Sent: Wednesday, July 21, 2004 3:42 PM
Subject: Re: ruby postgresql question

can you describe the table layout in sql?

Sure. Here is the sql statement for a message board table

@db.exec("CREATE TABLE #{table} (delete boolean DEFAULT false, \
           locked boolean DEFAULT false, number int PRIMARY KEY, \
            m_to varchar(40), \
           m_from varchar(40), msg_date timestamp, subject varchar(40),\
           msg_text text, exported boolean DEFAULT false)")

what does "reading individual messages in _each direction_" mean? Has
this something to do with the hierarchy of the messages? (In-response-To
etc.)?

Ok. It's an old text (telnet) based Bulletin Board System. You go to look
at messages
post since you last read them. The system find this point for you. Then
you press enter at the prompt to get the new messages, one at a time. Or
you can jump to a particular message by typing it's number, or you can go
backwards, if you like.

aha, now I understand :slight_smile:

I don't really understand :slight_smile: [about making the table...]

Well, messages get deleted, but the message numbers in the table don't
change. The message numbers (when you are reading them) *do* change because
they are always 1..<highest message> so the system needs to know which
actual message to pull up when you type it's number in. They won't be the
same, after one message is deleted. So I am doing a ...

for row in @db.query("SELECT number FROM #{table} ORDER BY number")
  hash << row[0].to_i
end

Okay, you can use Cursors if you want:

   BEGIN;
     DECLARE CURSOR c SCROLL CURSOR FOR
       SELECT number FROM table ORDER BY number;

     FETCH c;

     FETCH BACKWARD 1 IN c;

     CLOSE c;
   END;

Look for FETCH and DECLARE CURSOR in the Postgres docs.

Regards,

   Michael

···

----- Original Message -----

[Mark Firestone <hash_bang@retrobbs.org>, 2004-07-21 17.45 CEST]

Well, messages get deleted, but the message numbers in the table don't
change. The message numbers (when you are reading them) *do* change because
they are always 1..<highest message> so the system needs to know which
actual message to pull up when you type it's number in. They won't be the
same, after one message is deleted. So I am doing a ...

for row in @db.query("SELECT number FROM #{table} ORDER BY number")
  hash << row[0].to_i
end

and making an array, full of message numbers. A lookup table. So I can
look up which message to pull up.

In postgresql you can use the OFFSET and LIMIT modifiers. For example
  SELECT * FROM table ORDER BY number OFFSET #{nth_row} LIMIT 1
retrieves the nth row.

···

--

then perhaps using a short name, but it should be an extension to force
require statement :

   require 'io/reap'

   buf = pipe.reap

this would have the additional advantage that IO#reap is not normally defined.

-a

···

On Fri, 23 Jul 2004, Tanaka Akira wrote:

i like the sound of

   IO#reap

alot, eg. :

   #
   # read all chars available on pipe
   #
     buf = pipe.reap

i think this method is much needed and no name exists for something like it.
if we pick one that is memorable people will use it and remember it.

I like a method name longer than 4bytes ("read".length). Because I
encourage IO#read over IO#readpartial in usual case.

--

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
A flower falls, even though we love it;
and a weed grows, even though we do not love it. --Dogen

===============================================================================

In article <Pine.LNX.4.60.0407230652540.13379@harp.ngdc.noaa.gov>,
  "Ara.T.Howard" <ahoward@noaa.gov> writes:

then perhaps using a short name, but it should be an extension to force
require statement :

   require 'io/reap'

   buf = pipe.reap

this would have the additional advantage that IO#reap is not normally defined.

I don't like such small extension library.

···

--
Tanaka Akira