Curses - how to use unicode

ruby-talk
1

Yesterday I got xterm working with UTF-8. I had made an oneliner that
outputted a small box.

puts [0x250c, 0x2510, 10, 0x2514, 0x2518].pack("U*")

When I try the same through curses.. then no unicode.
I don't see the small box.

require 'curses'
Curses.init_screen
Curses.cbreak
Curses.raw
Curses.noecho
Curses.nonl
window = Curses.stdscr
window.keypad(true)
window.setpos(0, 0)
window.addstr([0x250c, 0x2510].pack("U*"))
window.setpos(1, 0)
window.addstr([0x2514, 0x2518].pack("U*"))
Curses.getch
Curses.close_screen

What am I doing wrong?

ruby 1.8.1 (2004-04-24) [i386-linux-gnu]
/* $Id: curses.h.in,v 1.148 2004/01/14 23:50:12 tom Exp $ */

btw: I did the following to enable UTF-8 in an xterm:

···

  xset +fp ~/local/lib/X11/fonts
  setenv LANG dk.UTF-8
  xterm +u8 -fa '-misc-*-*-*-*-*-20-*-*-*-*-*-iso10646-1' &

--
Simon Strandgaard

2

very likely ruby's package for curses doesn't link with the wide-character
variation (libncursesw for ncurses versus libncurses). That's needed to
build up the contents of each cell on the screen (internally Unicode).

···

Simon Strandgaard <neoneye@gmail.com> wrote:

Yesterday I got xterm working with UTF-8. I had made an oneliner that
outputted a small box.

puts [0x250c, 0x2510, 10, 0x2514, 0x2518].pack("U*")

When I try the same through curses.. then no unicode.
I don't see the small box.

--
Thomas E. Dickey

ftp://invisible-island.net

3

Ok.. I have now tried to use the lib directly from ruby.
It still outputs garbage. I am unsure if im using the right calls
for outputting ?

require 'dl/import'
module Curses
extend DL::Importable
dlload "libcurses.so"
typealias "WINDOW", "void*"
typealias "attr_t", "unsigned long"
extern "WINDOW initscr()"
extern "int endwin()"
extern "int getch()"
extern "int cbreak()"
extern "int noecho()"
extern "int refresh()"
extern "int addstr(const char *)"
end
Curses.initscr
Curses.cbreak
Curses.noecho
s = [0x250c, 0x2510, 10, 0x2514, 0x2518].pack("U*")+0.chr
Curses.addstr(s)
Curses.refresh
Curses.getch
Curses.endwin

Im now trying to install ncurses-5.4 to see if it makes a difference.
btw: Thomas E. Dickey you amaze me everytime you find xterm/ncurses
related stuff in different newsgroups.. how do you do?

···

On Tue, 21 Dec 2004 20:52:02 +0900, Thomas Dickey <dickey@saltmine.radix.net> wrote:

Simon Strandgaard <neoneye@gmail.com> wrote:
> Yesterday I got xterm working with UTF-8. I had made an oneliner that
> outputted a small box.

> puts [0x250c, 0x2510, 10, 0x2514, 0x2518].pack("U*")

> When I try the same through curses.. then no unicode.
> I don't see the small box.

very likely ruby's package for curses doesn't link with the wide-character
variation (libncursesw for ncurses versus libncurses). That's needed to
build up the contents of each cell on the screen (internally Unicode).

--
Simon Strandgaard

4

Ah.. I overlooked the 'w'.. I added --enable-widec when duing ./configure,
and now its much better. Thanks
Is widec still experimental ?

···

On Tue, 21 Dec 2004 21:39:31 +0100, Simon Strandgaard <neoneye@gmail.com> wrote:

On Tue, 21 Dec 2004 20:52:02 +0900, Thomas Dickey > <dickey@saltmine.radix.net> wrote:
> Simon Strandgaard <neoneye@gmail.com> wrote:
> > Yesterday I got xterm working with UTF-8. I had made an oneliner that
> > outputted a small box.
>
> > puts [0x250c, 0x2510, 10, 0x2514, 0x2518].pack("U*")
>
> > When I try the same through curses.. then no unicode.
> > I don't see the small box.
>
> very likely ruby's package for curses doesn't link with the wide-character
> variation (libncursesw for ncurses versus libncurses). That's needed to
> build up the contents of each cell on the screen (internally Unicode).

--
Simon Strandgaard

5

google of course (even with the new format). That's how I noticed your
response today.

Of course google has noticable bias - if I change the keywords slightly, it
frowns and says "Do you mean" and lists the keys I use more often.

···

Simon Strandgaard <neoneye@gmail.com> wrote:

Im now trying to install ncurses-5.4 to see if it makes a difference.
btw: Thomas E. Dickey you amaze me everytime you find xterm/ncurses
related stuff in different newsgroups.. how do you do?

--
Thomas E. Dickey

ftp://invisible-island.net

6

no - the "experimental" in the configure --help was something I overlooked
removing for 5.4. Not that there aren't remaining issues, but they take
longer to find and longer to fix.

···

Simon Strandgaard <neoneye@gmail.com> wrote:

> very likely ruby's package for curses doesn't link with the wide-character
> variation (libncursesw for ncurses versus libncurses). That's needed to
> build up the contents of each cell on the screen (internally Unicode).

Ah.. I overlooked the 'w'.. I added --enable-widec when duing ./configure,
and now its much better. Thanks
Is widec still experimental ?

--
Thomas E. Dickey

ftp://invisible-island.net

7

[snip]

I have compiled ncurses-5.4 with --enable-widec --with-shared.
I now have a "libncursesw.so" file..

I am still unsure what function I should use in order to output UTF-8
encoded strings.. or if I should output with another unicode encoding?

require 'dl/import'
module Curses
  extend DL::Importable
  dlload "libncursesw.so"
  typealias "WINDOW", "void*"
  typealias "wchar_t", "unsigned long"
  extern "int add_wch(const char *)"
  extern "int addstr(const char *)"
  extern "int cbreak()"
  extern "int curs_set(int)"
  extern "int endwin()"
  extern "int getch()"
  extern "WINDOW initscr()"
  extern "int noecho()"
  extern "int refresh()"
  
  # is const char * correct here ?
  extern "int add_wchstr(const char *)"
  extern "int add_wchnstr(const char *, int)"
end

str = [0x250c, 0x2510, 10, 0x2514, 0x2518].pack("U*") + "\000"
p str # -> "\342\224\214\342\224\220\n\342\224\224\342\224\230\000"
puts str # this renders a nice box
gets

Curses.initscr
Curses.cbreak
Curses.noecho
Curses.add_wchstr(str) # no box..
Curses.getch
Curses.endwin

···

--
Simon Strandgaard

8

[Simon Strandgaard <neoneye@gmail.com>, 2004-12-21 23.41 CET]

Hi, Simon:

  # is const char * correct here ?
  extern "int add_wchstr(const char *)"
  extern "int add_wchnstr(const char *, int)"

Probably you want "addwstr" and "addnwstr". The "wch" of "add_wchn?str"
seems to be a struct that encodes both character and attribute.

str = [0x250c, 0x2510, 10, 0x2514, 0x2518].pack("U*") + "\000"

On the other hand, the parameters to "addn?wstr" are wchar_t *, which I
think are defined as 32 bit integers. So I guess you should either use
"pack('L*')", or change the function declaration.

(Nothing tested :).

Good luck.

9

Curses.addstr() should give a box.

···

Simon Strandgaard <neoneye@gmail.com> wrote:

Curses.add_wchstr(str) # no box..

--
Thomas E. Dickey

ftp://invisible-island.net

10

[snip]

On the other hand, the parameters to "addn?wstr" are wchar_t *, which I
think are defined as 32 bit integers. So I guess you should either use
"pack('L*')", or change the function declaration.

Using addwstr with pack("L*") doesn't seem to work for me either.
How would a good representation of wchar_t string be like?

···

On Wed, 22 Dec 2004 11:58:29 +0900, Carlos <angus@quovadis.com.ar> wrote:

#
# Using ncurses-5.4 with --enable-widec --with-shared
#
require 'dl/import'
module Curses
  extend DL::Importable
  dlload "libncursesw.so"
  typealias "WINDOW", "void*"
  extern "int addstr(const char *)"
  extern "int cbreak()"
  extern "int endwin()"
  extern "int getch()"
  extern "WINDOW initscr()"
  extern "int noecho()"
  extern "int refresh()"
  extern "int addwstr(const unsigned long *)"
  extern "int addnwstr(const unsigned long *, int)"
end

str = [0x250c, 0x2510, 10, 0x2514, 0x2518].pack("L*")
Curses.initscr
Curses.cbreak
Curses.noecho
Curses.addnwstr(str, str.size) # no box..
Curses.getch
Curses.endwin

--
Simon Strandgaard

11

[Simon Strandgaard <neoneye@gmail.com>, 2004-12-22 08.29 CET]

> On the other hand, the parameters to "addn?wstr" are wchar_t *, which I
> think are defined as 32 bit integers. So I guess you should either use
> "pack('L*')", or change the function declaration.

Using addwstr with pack("L*") doesn't seem to work for me either.
How would a good representation of wchar_t string be like?

On the other other hand, I've found that if you use the functions without
"w", you can output UTF-8 strings directly :). It's more easy so...

And I've also found the secret to make ncursesw print non-ascii characters:
set the locale LC_ALL to "".

This works:

···

#
# Using ncurses-5.4 with --enable-widec --with-shared
#
require 'dl/import'
module Curses
  extend DL::Importable
  dlload "libncursesw.so"
  typealias "WINDOW", "void*"
  extern "int addstr(const char *)"
  extern "int cbreak()"
  extern "int endwin()"
  extern "int getch()"
  extern "WINDOW initscr()"
  extern "int noecho()"
  extern "int refresh()"
  extern "int addstr(const char *)"
  extern "int addnstr(const char *, int)"
end

module LibC
  extend DL::Importable
  dlload "libc.so.6"
  extern "void setlocale(int, const char *)"
end
  
LibC.setlocale(6, "") # LC_ALL == 6
str = [0x250c, 0x2510, 10, 0x2514, 0x2518, 0].pack("U*")
Curses.initscr
Curses.cbreak
Curses.noecho
Curses.addstr(str) # a box..
Curses.addstr("simple UTF-8 string ¿←↓↑→? ¡↑!\000")
Curses.getch
Curses.endwin
__END__

Good luck.

12

yes (but it's not portable: won't work with Solaris curses). There are
some post-5.4 fixes to make this work, btw.

···

Carlos <angus@quovadis.com.ar> wrote:

On the other other hand, I've found that if you use the functions without
"w", you can output UTF-8 strings directly :). It's more easy so...

And I've also found the secret to make ncursesw print non-ascii characters:
set the locale LC_ALL to "".

--
Thomas E. Dickey

ftp://invisible-island.net

13

Ah.. I had set the LANG wrong.. now it works.
I see the box.. I see the unicode letters.. I can now sleep well :slight_smile:

setenv LANG da_DK.UTF-8

I tried and tried.. and now it works..

what a pleasant christmas present.. Thanks Thomas and Carlos.

Now we all just wait for ruby-1.8.2 :slight_smile:

Happy Xmas.

···

On Wed, 22 Dec 2004 22:32:01 +0900, Thomas Dickey <dickey@saltmine.radix.net> wrote:

Carlos <angus@quovadis.com.ar> wrote:

> On the other other hand, I've found that if you use the functions without
> "w", you can output UTF-8 strings directly :). It's more easy so...

> And I've also found the secret to make ncursesw print non-ascii characters:
> set the locale LC_ALL to "".

yes (but it's not portable: won't work with Solaris curses). There are
some post-5.4 fixes to make this work, btw.

--
Simon Strandgaard

14

[snip]

I tried and tried.. and now it works..

I can now do some box-drawing (beware this is not yet the editor in action)

http://aeditor.rubyforge.org/aeditor_shots/043.png

Code is here:
http://rubyforge.org/cgi-bin/viewcvs.cgi/projects/experimental/unicode/curses_wide2.rb?rev=1.4&cvsroot=aeditor&content-type=text/vnd.viewcvs-markup

I have wrapped Ncurses fairly big Window structure via Ruby/DL, so it now can be
used from within ruby.

···

On Wed, 22 Dec 2004 23:51:36 +0100, Simon Strandgaard <neoneye@gmail.com> wrote:

--
Simon Strandgaard