Ruby-Tk and Tk 8.6

I have seen multiple bug reports of Ruby-Tk scripts crashing if they attempt to load Tk 8.6, the current supported version of Tcl/Tk.

Are there any special technical issues preventing Ruby from working with Tk 8.6, or is it just a matter of submitting a patch to update the build bits? Tk 8.6 works just fine with other scripting languages with Tk bindings, cf. Python and Perl, so it is not clear to me what the issue is. I have not been able to find details in the various bug reports I've seen.

Please advise.

···

--
Kevin Walzer
Code by Kevin/Mobile Code by Kevin
http://www.codebykevin.com
http://www.wtmobilesoftware.com

Hi,

···

From: Kevin Walzer <kw@codebykevin.com>
Subject: Ruby-Tk and Tk 8.6
Date: Wed, 08 Oct 2014 10:41:35 -0400
Message-ID: <54354D1F.2080207@codebykevin.com>

I have seen multiple bug reports of Ruby-Tk scripts crashing if they
attempt to load Tk 8.6, the current supported version of Tcl/Tk.

I'm very sorry, but current Ruby/Tk doesn't supprot Tcl/Tk 8.6.
Currently, I have no idea to fix tailcall (one of the new feature of Tcl 8.6)
problem. If without that, I cannot claim that Ruby/Tk works with Tcl/Tk 8.6.

The problem depends on Ruby/Tk's callback strategy to support threads.
All Tk commands must run on a specific thread.
However, Ruby's any threads may call Tcl/Tk commands.
Tcl's tailcall command replaces the top of function call stack.
When the callstack is tcl->ruby->tcl, Ruby's callback function may be
skipped or fail to get a proper return value of the tcl function.
If all Ruby/Tk users accept that Ruby/Tk works on the main thread only,
supporting Tcl8.6 may not be difficult.
But it is extremely serious incompatibility.

I know that some kind rubyists provide patches for Tcl8.6.
But partial support may cause unexpected trouble.
I think that "never work" is better than "partial work".
--
Hidetoshi NAGAI (nagai@ai.kyutech.ac.jp)
Department of Artificial Intelligence, Kyushu Institute of Technology

This is considered to be the best practice in Python and, I believe, Perl: call Tk on the main thread only. Tkinter developers, if they must use multiple threads, use worker threads to do background tasks and data processing and then notify Tk on the main thread. I'm a newcomer to Ruby, but I am quite surprised to hear that Ruby developers can call Tk from any thread.

My strong suggestion is that it would be best to simply make Ruby's mechanism to support Tk on the main thread only. If narrowing the scope of Ruby's support for Tk to the main thread will make it relatively easy to support 8.6, then that seems to be an obvious and necessary step. This is especially true if the issue is primarily one of developer practice, i.e. developers are used to calling Tk from any thread, rather than an inherent technical limitation that requires deep changes in Ruby's internals.

Perl and Python's support for Tk does not depend on Tk's version. Both support 8.5 and 8.6 seamlessly. I believe it's a reasonable expectation that Ruby do the same.

--Kevin

···

On 10/8/14, 6:29 PM, Hidetoshi NAGAI wrote:

If all Ruby/Tk users accept that Ruby/Tk works on the main thread only,
supporting Tcl8.6 may not be difficult.
But it is extremely serious incompatibility.

--
Kevin Walzer
Code by Kevin/Mobile Code by Kevin
http://www.codebykevin.com
http://www.wtmobilesoftware.com

I have done some additional research and discovered this patch:

https://bugs.ruby-lang.org/attachments/download/4612/0001-Fix-tk-crash-with-Tk-8.6.1-on-Ubuntu.patch

I have not tested it, but if it addresses the issue of Ruby crashing with Tk 8.6, then I am puzzled why it has not been included, although I fully admit that I may not have all the information here.

One point, however, is important. Tcl/Tk 8.5 is still being maintained along with 8.6, but it is likely to be end-of-lifed in the not-too-distant future. (I am one of the core Tcl/Tk developers, the maintainer of Tk on the Mac.) Tcl/Tk 8.6 has been out for two years now. Forcing Ruby developers to use an unsupported version of Tcl/Tk is not the best solution IMO. Maintaining a stance of "'never work' is better than 'partial work'" is tantamount to removing Tk support from Ruby's core. Is that really what you are proposing?

--Kevin

···

On 10/8/14, 6:29 PM, Hidetoshi NAGAI wrote:

I know that some kind rubyists provide patches for Tcl8.6.
But partial support may cause unexpected trouble.
I think that "never work" is better than "partial work".

--
Kevin Walzer
Code by Kevin/Mobile Code by Kevin
http://www.codebykevin.com
http://www.wtmobilesoftware.com

Indeed, and I went to the trouble to clean up and apply the patch I referenced earlier in this thread (removing the Ubuntu-specific bits). I was able to build Ruby 2.1.3, and it could even load a simple Tk script ("hello world"), but anything more complex raised a runtime error because Ruby didn't understand Tcl's "namespace" command.

I acknowledge the apparent complexity of integrating 8.6, but it is a disappointment that Ruby has hit this barrier when Python and Perl integrate Tk 8.6 without any issue. I lack the knowledge of Ruby to provide a more useful patch, however, so I will hereby cease further comment on this matter and hope that someone will be able to move ahead with the integration.

--Kevin

···

On 10/8/14, 6:29 PM, Hidetoshi NAGAI wrote:

I know that some kind rubyists provide patches for Tcl8.6.

--
Kevin Walzer
Code by Kevin/Mobile Code by Kevin
http://www.codebykevin.com
http://www.wtmobilesoftware.com

Hi,

···

From: Kevin Walzer <kw@codebykevin.com>
Subject: Re: Ruby-Tk and Tk 8.6
Date: Fri, 10 Oct 2014 09:06:58 -0400
Message-ID: <5437D9F2.7090007@codebykevin.com>

On 10/8/14, 6:29 PM, Hidetoshi NAGAI wrote:

I know that some kind rubyists provide patches for Tcl8.6.

Indeed, and I went to the trouble to clean up and apply the patch I
referenced earlier in this thread (removing the Ubuntu-specific
bits). I was able to build Ruby 2.1.3, and it could even load a simple
Tk script ("hello world"), but anything more complex raised a runtime
error because Ruby didn't understand Tcl's "namespace" command.

The following is an experimental patch for Ruby-2.1.3 with Tcl/Tk-8.6.2.
It will fix the "namespace" trouble only.
I've committed it to trunk also.
--------------------------------------------------------------
diff -urN ruby-2.1.3.orig/ext/tk/extconf.rb ruby-2.1.3.mod/ext/tk/extconf.rb
--- ruby-2.1.3.orig/ext/tk/extconf.rb 2013-11-30 11:46:47.000000000 +0900
+++ ruby-2.1.3.mod/ext/tk/extconf.rb 2014-10-14 02:26:29.875902167 +0900
@@ -9,10 +9,10 @@
   # %w[8.9 8.8 8.7 8.6 8.5 8.4 8.3 8.2 8.1 8.0 7.6 4.2]
   # %w[8.7 8.6 8.5 8.4 8.3 8.2 8.1 8.0]
   # %w[8.7 8.6 8.5 8.4 8.0] # to shorten search steps
- %w[8.5 8.4] # At present, Tcl/Tk8.6 is not supported.
+ %w[8.5 8.4 8.6] # Tcl/Tk8.6 support is experimental.

TkLib_Config['unsupported_versions'] =
- %w[8.8 8.7 8.6] # At present, Tcl/Tk8.6 is not supported.
+ %w[8.8 8.7] # Tcl/Tk8.6 support is experimental.

TkLib_Config['major_nums'] = '87'

diff -urN ruby-2.1.3.orig/ext/tk/tcltklib.c ruby-2.1.3.mod/ext/tk/tcltklib.c
--- ruby-2.1.3.orig/ext/tk/tcltklib.c 2014-02-10 20:45:14.000000000 +0900
+++ ruby-2.1.3.mod/ext/tk/tcltklib.c 2014-10-14 01:31:09.757932244 +0900
@@ -6012,7 +6012,12 @@
     Tcl_CmdInfo info;
     int ret;

+ DUMP1("call ip_rbNamespaceObjCmd");
+ DUMP2("objc = %d", objc);
+ DUMP2("objv[0] = '%s'", Tcl_GetString(objv[0]));
+ DUMP2("objv[1] = '%s'", Tcl_GetString(objv[1]));
     if (!Tcl_GetCommandInfo(interp, "__orig_namespace_command__", &(info))) {
+ DUMP1("fail to get __orig_namespace_command__");
         Tcl_ResetResult(interp);
         Tcl_AppendResult(interp,
                          "invalid command name \"namespace\"", (char*)NULL);
@@ -6020,15 +6025,37 @@
     }

     rbtk_eventloop_depth++;
- /* DUMP2("namespace wrapper enter depth == %d", rbtk_eventloop_depth); */
+ DUMP2("namespace wrapper enter depth == %d", rbtk_eventloop_depth);

     if (info.isNativeObjectProc) {
+#if TCL_MAJOR_VERSION == 8 && TCL_MINOR_VERSION < 6
+ DUMP1("call a native-object-proc");
         ret = (*(info.objProc))(info.objClientData, interp, objc, objv);
+#else
+ /* Tcl8.6 or later */
+ int i;
+ Tcl_Obj **cp_objv;
+ char org_ns_cmd_name = "__orig_namespace_command__";
+
+ DUMP1("call a native-object-proc for tcl8.6 or later");
+ cp_objv = RbTk_ALLOC_N(Tcl_Obj *, (objc + 1));
+
+ cp_objv[0] = Tcl_NewStringObj(org_ns_cmd_name, strlen(org_ns_cmd_name));
+ for(i = 1; i < objc; i++) {
+ cp_objv[i] = objv[i];
+ }
+ cp_objv[objc] = (Tcl_Obj *)NULL;
+
+ ret = Tcl_EvalObjv(interp, objc, cp_objv, TCL_EVAL_DIRECT);
+
+ ckfree((char*)cp_objv);
+#endif
     } else {
         /* string interface */
         int i;
         char **argv;

+ DUMP1("call with the string-interface");
         /* argv = (char **)Tcl_Alloc(sizeof(char *) * (objc + 1)); */
         argv = RbTk_ALLOC_N(char *, (objc + 1));
#if 0 /* use Tcl_Preserve/Release */
@@ -6056,9 +6083,10 @@
#endif
     }

- /* DUMP2("namespace wrapper exit depth == %d", rbtk_eventloop_depth); */
+ DUMP2("namespace wrapper exit depth == %d", rbtk_eventloop_depth);
     rbtk_eventloop_depth--;

+ DUMP1("end of ip_rbNamespaceObjCmd");
     return ret;
}
#endif
@@ -6068,6 +6096,8 @@
     Tcl_Interp *interp;
{
#if TCL_MAJOR_VERSION >= 8
+
+#if TCL_MAJOR_VERSION == 8 && TCL_MINOR_VERSION < 6
     Tcl_CmdInfo orig_info;

     if (!Tcl_GetCommandInfo(interp, "namespace", &(orig_info))) {
@@ -6084,6 +6114,11 @@
                           orig_info.deleteProc);
     }

+#else /* tcl8.6 or later */
+ Tcl_Eval(interp, "rename namespace __orig_namespace_command__");
+
+#endif
+
     Tcl_CreateObjCommand(interp, "namespace", ip_rbNamespaceObjCmd,
                          (ClientData) 0, (Tcl_CmdDeleteProc *)NULL);
#endif
@@ -8454,9 +8489,12 @@
     char **argv = (char **)NULL;
#endif

+ DUMP1("call invoke_tcl_proc");
+
     /* memory allocation for arguments of this command */
#if TCL_MAJOR_VERSION >= 8
     if (!inf->cmdinfo.isNativeObjectProc) {
+ DUMP1("called proc is not a native-obj-proc");
         /* string interface */
         /* argv = (char **)ALLOC_N(char *, argc+1);*/ /* XXXXXXXXXX */
         argv = RbTk_ALLOC_N(char *, (argc+1));
@@ -8470,11 +8508,13 @@
     }
#endif

+ DUMP1("reset result of tcl-interp");
     Tcl_ResetResult(inf->ptr->ip);

     /* Invoke the C procedure */
#if TCL_MAJOR_VERSION >= 8
     if (inf->cmdinfo.isNativeObjectProc) {
+ DUMP1("call tcl_proc as a native-obj-proc");
         inf->ptr->return_value
             = (*(inf->cmdinfo.objProc))(inf->cmdinfo.objClientData,
                                         inf->ptr->ip, inf->objc, inf->objv);
@@ -8483,6 +8523,7 @@
#endif
     {
#if TCL_MAJOR_VERSION >= 8
+ DUMP1("call tcl_proc as not a native-obj-proc");
         inf->ptr->return_value
             = (*(inf->cmdinfo.proc))(inf->cmdinfo.clientData, inf->ptr->ip,
                                      argc, (CONST84 char **)argv);
@@ -8505,6 +8546,7 @@
#endif
     }

+ DUMP1("end of invoke_tcl_proc");
     return Qnil;
}

@@ -8644,7 +8686,9 @@
#endif

     /* invoke tcl-proc */
+ DUMP1("invoke tcl-proc");
     rb_protect(invoke_tcl_proc, (VALUE)&inf, &status);
+ DUMP2("status of tcl-proc, %d", status);
     switch(status) {
     case TAG_RAISE:
         if (NIL_P(rb_errinfo())) {
--------------------------------------------------------------
--
Hidetoshi NAGAI (nagai@ai.kyutech.ac.jp)
Department of Artificial Intelligence, Kyushu Institute of Technology

Thank you so much for this!

The patch applies cleanly and I was able to build Ruby 2.1.3 against Tk 8.6. Just as importantly, Ruby did not crash when I ran the Arcadia IDE against Tk 8.6 (https://github.com/angal/arcadia\)--the most complicated Ruby-Tk app that I know of.

Any support for Tk 8.6, even "experimental," is welcome and will allow Ruby-Tk developers to keep working with Tk once 8.5 is end-of-lifed, which will likely be soon.

Thank you again for taking the time to do this. It is much appreciated!

--Kevin

···

On 10/13/14, 1:44 PM, Hidetoshi NAGAI wrote:

The following is an experimental patch for Ruby-2.1.3 with Tcl/Tk-8.6.2.
It will fix the "namespace" trouble only.
I've committed it to trunk also.

--
Kevin Walzer
Code by Kevin/Mobile Code by Kevin
http://www.codebykevin.com
http://www.wtmobilesoftware.com