Read individual characters off socket?

Hi! I'm trying to read a character-at-a-time off a network socket; while (say) getch kinda-sorta does that, it only does it once the remote end has submitted the entire line via <CR>. Clearly, I'm missing something where character-at-a-time interaction could occur.

Any pointers?

Thanks!

-Ken

E.g., "character-at-a-time," but only when <CR> is submitted:

require 'socket'
server_socket = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM, 0)
pry(main)> sockaddr = Socket.pack_sockaddr_in( 2200, 'localhost' )
server_socket.bind( sockaddr )
server_socket.listen(5)
client, client_sockaddr = server_socket.accept
while (foo = client.getc)
puts foo
end

Hi! I'm trying to read a character-at-a-time off a network socket; while
(say) getch kinda-sorta does that, it only does it once the remote end has
submitted the entire line via <CR>. Clearly, I'm missing something where
character-at-a-time interaction could occur.

Any pointers?

E.g., "character-at-a-time," but only when <CR> is submitted:

What characters are you sending with the client?
If it's a multi-byte character, then maybe the "character" was
not finished?

require 'socket'
server_socket = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM, 0)
pry(main)> sockaddr = Socket.pack_sockaddr_in( 2200, 'localhost' )
server_socket.bind( sockaddr )
server_socket.listen(5)
client, client_sockaddr = server_socket.accept
while (foo = client.getc)
puts foo
end

Which client are you using?
If it's the telnet command, telnet itself might not send write a
packet until the user hits "Enter" for <CR>.

Instead, try TCPSocket from another terminal:

  conn = TCPSocket.new('localhost', 2200)
  conn.write("a") # one TCP packet sent
  # client.getc should return 1 byte

Keep in mind doing stuff one-byte-at-a-time is extremely
inefficient because of TCP/IP header overhead and your
network administrators will not be happy with you.

Normally Ruby servers and network clients use IO#readpartial or
IO#read_nonblock to read a bunch at once.

···

Ken D'Ambrosio <ken@jots.org> wrote:

Fuck I’m too high for that

El El sáb, 30 jun. 2018 a las 01:24, Eric Wong <e@80x24.org> escribió:

···

Ken D'Ambrosio <ken@jots.org> wrote:
> Hi! I'm trying to read a character-at-a-time off a network socket; while
> (say) getch kinda-sorta does that, it only does it once the remote end
has
> submitted the entire line via <CR>. Clearly, I'm missing something where
> character-at-a-time interaction could occur.
>
> Any pointers?
>
> E.g., "character-at-a-time," but only when <CR> is submitted:

What characters are you sending with the client?
If it's a multi-byte character, then maybe the "character" was
not finished?

> require 'socket'
> server_socket = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM, 0)
> pry(main)> sockaddr = Socket.pack_sockaddr_in( 2200, 'localhost' )
> server_socket.bind( sockaddr )
> server_socket.listen(5)
> client, client_sockaddr = server_socket.accept
> while (foo = client.getc)
> puts foo
> end

Which client are you using?
If it's the telnet command, telnet itself might not send write a
packet until the user hits "Enter" for <CR>.

Instead, try TCPSocket from another terminal:

        conn = TCPSocket.new('localhost', 2200)
        conn.write("a") # one TCP packet sent
        # client.getc should return 1 byte

Keep in mind doing stuff one-byte-at-a-time is extremely
inefficient because of TCP/IP header overhead and your
network administrators will not be happy with you.

Normally Ruby servers and network clients use IO#readpartial or
IO#read_nonblock to read a bunch at once.

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

--
Enviado con Gmail Mobile

can you send me information in spanish please thank you

···

2018-06-30 1:05 GMT-04:00 Diego Fernandez <diego79@gmail.com>:

Fuck I’m too high for that

El El sáb, 30 jun. 2018 a las 01:24, Eric Wong <e@80x24.org> escribió:

Ken D'Ambrosio <ken@jots.org> wrote:
> Hi! I'm trying to read a character-at-a-time off a network socket;
while
> (say) getch kinda-sorta does that, it only does it once the remote end
has
> submitted the entire line via <CR>. Clearly, I'm missing something
where
> character-at-a-time interaction could occur.
>
> Any pointers?
>
> E.g., "character-at-a-time," but only when <CR> is submitted:

What characters are you sending with the client?
If it's a multi-byte character, then maybe the "character" was
not finished?

> require 'socket'
> server_socket = Socket.new(Socket::AF_INET, Socket::SOCK_STREAM, 0)
> pry(main)> sockaddr = Socket.pack_sockaddr_in( 2200, 'localhost' )
> server_socket.bind( sockaddr )
> server_socket.listen(5)
> client, client_sockaddr = server_socket.accept
> while (foo = client.getc)
> puts foo
> end

Which client are you using?
If it's the telnet command, telnet itself might not send write a
packet until the user hits "Enter" for <CR>.

Instead, try TCPSocket from another terminal:

        conn = TCPSocket.new('localhost', 2200)
        conn.write("a") # one TCP packet sent
        # client.getc should return 1 byte

Keep in mind doing stuff one-byte-at-a-time is extremely
inefficient because of TCP/IP header overhead and your
network administrators will not be happy with you.

Normally Ruby servers and network clients use IO#readpartial or
IO#read_nonblock to read a bunch at once.

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

--
Enviado con Gmail Mobile

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

E.g., "character-at-a-time," but only when <CR> is submitted:

What characters are you sending with the client?
If it's a multi-byte character, then maybe the "character" was
not finished?

Just straight-up eight-bit ASCII. (Long story short, this is meant to interact with an old-school bulletin board system. No fancy Unicode or anything going on -- or, indeed, even supported.)

Which client are you using?
If it's the telnet command, telnet itself might not send write a
packet until the user hits "Enter" for <CR>.

Nah -- telnet is "realtime." I.e., when I use telnet to go to said bulletin board, individual characters are immediately acted on.

Keep in mind doing stuff one-byte-at-a-time is extremely
inefficient because of TCP/IP header overhead and your
network administrators will not be happy with you.

Fortunately, I'm the network admin. :wink: And, yeah, crazy inefficient to use a 1500-byte packet to send a one-byte payload, but such are the vagaries of unbuffered interaction.

Normally Ruby servers and network clients use IO#readpartial or
IO#read_nonblock to read a bunch at once.

Hrmm... THIS, I will have to do some reading up on.

Thank you!!

-Ken

···

On 2018-06-30 00:24, Eric Wong wrote:

GAH! I have learned something. Telnet apparently operates differently depending on whether it's connecting to port 23 (the one the BBS is on), or other ports (e.g., port 2200, which is what I was using, "because privileged ports.") Once I connected, then did a "^]", typed "mode character" -- setting character-at-a-time, and not LINEMODE -- the code I posted originally worked fine.

That'll learn me to not give due thought to client-side considerations.

-Ken

···

On 2018-07-01 10:53, Ken D'Ambrosio wrote:

On 2018-06-30 00:24, Eric Wong wrote:

Which client are you using?
If it's the telnet command, telnet itself might not send write a
packet until the user hits "Enter" for <CR>.

Nah -- telnet is "realtime." I.e., when I use telnet to go to said
bulletin board, individual characters are immediately acted on.

在 2018年7月1日,下午10:53,Ken D'Ambrosio <ken@jots.org> 写道:

E.g., "character-at-a-time," but only when <CR> is submitted:

What characters are you sending with the client?
If it's a multi-byte character, then maybe the "character" was
not finished?

Just straight-up eight-bit ASCII. (Long story short, this is meant to interact with an old-school bulletin board system. No fancy Unicode or anything going on -- or, indeed, even supported.)

Which client are you using?
If it's the telnet command, telnet itself might not send write a
packet until the user hits "Enter" for <CR>.

Nah -- telnet is "realtime." I.e., when I use telnet to go to said bulletin board, individual characters are immediately acted on.

Keep in mind doing stuff one-byte-at-a-time is extremely
inefficient because of TCP/IP header overhead and your
network administrators will not be happy with you.

Fortunately, I'm the network admin. :wink: And, yeah, crazy inefficient to use a 1500-byte packet to send a one-byte payload, but such are the vagaries of unbuffered interaction.

Normally Ruby servers and network clients use IO#readpartial or
IO#read_nonblock to read a bunch at once.

Hrmm... THIS, I will have to do some reading up on.

Thank you!!

-Ken

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

···

On 2018-06-30 00:24, Eric Wong wrote:

Interesting! It seems telnet is too smart for this kind of testing. I think
netcat might be a better tool. You can type in a character and hit ^D to
send it (otherwise it's line buffered).

Best regards
Greg

Nah; telnet's good. It's *me* that's broke. I've learned a lot more
about the telnet protocol, itself. E.g., the telnet server, itself, is
what requests character mode (or linemode). So, for example, the server
could do something like this:
socket.print(255.chr, 254.chr, 34.chr) # IAC [Interpret As Command],
DONT, Linemode

telnetd -- by default, at least -- issues just that as part of the
initial handshake. Opening a port and listening, though, not so much.

Now I know! And, by issuing just that command, was able to force the
telnet client into character mode. Yay!

-Ken

···

On 2018-07-09 14:41, Greg Navis wrote:

Interesting! It seems telnet is too smart for this kind of testing. I think netcat might be a better tool. You can type in a character and hit ^D to send it (otherwise it's line buffered).

Best regards
Greg

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

Gosh darn it. I meant to send along a link for reference; here's a
decent one:

http://pcmicro.com/netfoss/telnet.html

···

On 2018-07-09 20:01, Ken D'Ambrosio wrote:

Nah; telnet's good. It's *me* that's broke. I've learned a lot more about the telnet protocol, itself. E.g., the telnet server, itself, is what requests character mode (or linemode). So, for example, the server could do something like this:
socket.print(255.chr, 254.chr, 34.chr) # IAC [Interpret As Command], DONT, Linemode

telnetd -- by default, at least -- issues just that as part of the initial handshake. Opening a port and listening, though, not so much.

Now I know! And, by issuing just that command, was able to force the telnet client into character mode. Yay!

-Ken

On 2018-07-09 14:41, Greg Navis wrote:

Interesting! It seems telnet is too smart for this kind of testing. I think netcat might be a better tool. You can type in a character and hit ^D to send it (otherwise it's line buffered).

Best regards
Greg

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

If you really want to dive into it, there's these:

* RFC 854 - Telnet Protocol Specification
* Telnet Options
* RFC 1184 - Telnet Linemode Option

Telnet was great fun, back in the day. I can't honestly say I miss it,
though.

But now I have to ask: are you implementing a telnet server, or just
reading bytes from a TCP stream? Because netcat might actually be a better
answer *in this instance *.

Matthew Kerwin

···

On Tue., 10 Jul. 2018, 11:57 Ken D'Ambrosio, <ken@jots.org> wrote:

Gosh darn it. I meant to send along a link for reference; here's a decent
one:

The Telnet Protocol

On 2018-07-09 20:01, Ken D'Ambrosio wrote:

Nah; telnet's good. It's *me* that's broke. I've learned a lot more
about the telnet protocol, itself. E.g., the telnet server, itself, is
what requests character mode (or linemode). So, for example, the server
could do something like this:
socket.print(255.chr, 254.chr, 34.chr) # IAC [Interpret As Command], DONT,
Linemode

telnetd -- by default, at least -- issues just that as part of the initial
handshake. Opening a port and listening, though, not so much.

Now I know! And, by issuing just that command, was able to force the
telnet client into character mode. Yay!

-Ken

On 2018-07-09 14:41, Greg Navis wrote:

Interesting! It seems telnet is too smart for this kind of testing. I
think netcat might be a better tool. You can type in a character and hit ^D
to send it (otherwise it's line buffered).

Best regards
Greg

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

But now I have to ask: are you implementing a telnet server, or just reading bytes from a TCP stream? Because netcat might actually be a better answer *in this instance *.

Telnet was great fun, back in the day. I can't honestly say I miss it, though.

Well... it's like this. Our BBS -- running code from '88 and earlier --
is strictly telnet. (Well... probably some dialup code in there, too,
but that's so crufty it makes the '88 telnet stuff look new.) However,
that's not entirely true; there are two components to the BBS: the core
BBS code itself (which -- ah, the joys of legacy code -- actually runs
one instance/user), and a "connector" front end that accepts telnet
connections and then passes them through to the BBS code. All of which
worked for ~23 years or so terrifically.

Then came the Russians. More specifically, the Russian Dyn/DDoS attack.
It took us *down*. Now, you may say to yourself, "Who in their right
mind would DDoS a BBS used by a few-hundred old fogies?" And the answer
is "Nobody -- intentionally." But the way that the damn worm
*propagates* is through embedded devices with telnet as the admin
mechanism, and user "admin" as the user. So we've been *slammed* for
some two years straight with One Bajillion bogus login attempts over
telnet. And the connector happily accepts them -- they can't log in
(there's no user "admin", nor credentials for same), but they take up
lots and lots of sockets, causing the connector to die fairly often, and
exposing a few edge condition bugs in the core BBS code.

I'd like to set up something to replace the connector: a telnet proxy of
sorts. First and foremost would be to discard attempted "admin" logins.
After that, to just sit there and pass stuff through to the BBS code.

Regret you asked yet? :wink:

-Ken

Matthew Kerwin

···

On 2018-07-10 17:20, Matthew Kerwin wrote:

On Tue., 10 Jul. 2018, 11:57 Ken D'Ambrosio, <ken@jots.org> wrote:

Gosh darn it. I meant to send along a link for reference; here's a decent one:

The Telnet Protocol

On 2018-07-09 20:01, Ken D'Ambrosio wrote:

Nah; telnet's good. It's *me* that's broke. I've learned a lot more about the telnet protocol, itself. E.g., the telnet server, itself, is what requests character mode (or linemode). So, for example, the server could do something like this:
socket.print(255.chr, 254.chr, 34.chr) # IAC [Interpret As Command], DONT, Linemode

telnetd -- by default, at least -- issues just that as part of the initial handshake. Opening a port and listening, though, not so much.

Now I know! And, by issuing just that command, was able to force the telnet client into character mode. Yay!

-Ken

On 2018-07-09 14:41, Greg Navis wrote:

Interesting! It seems telnet is too smart for this kind of testing. I think netcat might be a better tool. You can type in a character and hit ^D to send it (otherwise it's line buffered).
Best regards
Greg

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

Unsubscribe:
<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

Unsubscribe:
<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;

So, if you write a new gateway which can eagerly drop attempted
"admin" logins, you can save the BBS back-end from being flooded out.
But won't the gateway itself still run out of sockets? You can tweak
TCP parameters and stuff (e.g. reducing TIME_WAIT) but those have
their own risks.

If it's really Mirai (and variants) you're up against, why not just
have the telnet side of the gateway listen on a completely different
port? AFAIK Mirai attacks TCP/23 and TCP/2323. So set it to listen
on TCP/7777 or something.

Cheers

···

On 11 July 2018 at 12:07, Ken D'Ambrosio <ken@jots.org> wrote:

I'd like to set up something to replace the connector: a telnet proxy of
sorts. First and foremost would be to discard attempted "admin" logins.
After that, to just sit there and pass stuff through to the BBS code.

--
  Matthew Kerwin
  https://matthew.kerwin.net.au/

So, if you write a new gateway which can eagerly drop attempted
"admin" logins, you can save the BBS back-end from being flooded out.
But won't the gateway itself still run out of sockets? You can tweak
TCP parameters and stuff (e.g. reducing TIME_WAIT) but those have
their own risks.

From the OS side, I don't think we're being hammered so hard that it affects the availability of sockets. It just makes the connector app get upset fairly frequently (e.g., it's down as I type this -- even though it's still accepting connection attempts). I, myself, set up an open telnet port on one of my VMs, and it was truly impressive how many connection attempts got going, but it was more of a "boy, that's really annoying" than "holy crow, port 23 is officially down."

If it's really Mirai (and variants) you're up against, why not just
have the telnet side of the gateway listen on a completely different
port? AFAIK Mirai attacks TCP/23 and TCP/2323. So set it to listen
on TCP/7777 or something.

We thought about that -- but a number of people use clients, and not all of them are terribly technically savvy in figuring out how to change default port settings, etc. It's a shame -- there aren't many BBSes left, and the botnet propagation mechanism took down a handful of them permanently. (At least one threw in the towel and migrated to a Slack channel.) But I'm taking this as a bit of a fun challenge, too: I'll learn some, and help out the BBS. Win-win. I hope. :wink:

-Ken

···

On 2018-07-10 23:02, Matthew Kerwin wrote:

On 11 July 2018 at 12:07, Ken D'Ambrosio <ken@jots.org> wrote:

Have you tried to use something like iptables and fail2ban?
If you have a log about failed logins you should be able to ban bots by IP

Mikol

PS this is my first reply to this mailing list :slight_smile:

···

On 12 Jul 2018, at 03:47, Ken D'Ambrosio <ken@jots.org> wrote:

On 2018-07-10 23:02, Matthew Kerwin wrote:

On 11 July 2018 at 12:07, Ken D'Ambrosio <ken@jots.org> wrote:

So, if you write a new gateway which can eagerly drop attempted
"admin" logins, you can save the BBS back-end from being flooded out.
But won't the gateway itself still run out of sockets? You can tweak
TCP parameters and stuff (e.g. reducing TIME_WAIT) but those have
their own risks.

From the OS side, I don't think we're being hammered so hard that it affects the availability of sockets. It just makes the connector app get upset fairly frequently (e.g., it's down as I type this -- even though it's still accepting connection attempts). I, myself, set up an open telnet port on one of my VMs, and it was truly impressive how many connection attempts got going, but it was more of a "boy, that's really annoying" than "holy crow, port 23 is officially down."

If it's really Mirai (and variants) you're up against, why not just
have the telnet side of the gateway listen on a completely different
port? AFAIK Mirai attacks TCP/23 and TCP/2323. So set it to listen
on TCP/7777 or something.

We thought about that -- but a number of people use clients, and not all of them are terribly technically savvy in figuring out how to change default port settings, etc. It's a shame -- there aren't many BBSes left, and the botnet propagation mechanism took down a handful of them permanently. (At least one threw in the towel and migrated to a Slack channel.) But I'm taking this as a bit of a fun challenge, too: I'll learn some, and help out the BBS. Win-win. I hope. :wink:

-Ken

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk&gt;