[ANN] Syck 0.08 -- Next-generation of YAML.rb

citizens,

Here’s the deal. I became sick and delirious about a month ago. In my
deranged state, I felt it imperative that I should recode YAML.rb’s
parser in C. I have just now come to my senses, incident to the
powerful fumes of a passing onion truck. And yet, I am sitting before a
new YAML parser that is as wide and wonderful as any of the world’s
finest onion trucks!

It is like I am an Olympic diver who has developed a serious case of amnesia
mid-pike. There is nothing I can do now.

So I am please to present Syck, the new YAML parser/toy on the block.
I have included Ruby, Python and PHP extensions, though the last two
need some work. An OCaml extension is in progress as well.

Download here:
Download syck-0.08.tar.gz (YAML.rb: YAML for Ruby)

To compile the Ruby extension:
cd ext/ruby/
ruby extconf.rb
make
sudo make install

To load a document:

require ‘syck’
Syck.load( <<EOY )

  • one
  • two
  • three

Much is done but much is left to be done. The point is: I’m at version
0.08. And it’s time to start releasing early and releasing often.

Please jump in and voice your support for YAML in the Ruby distribution!
YAML.rb has only been a module for eight months and has seen widespread
usage. Wouldn’t you like to <require ‘yaml’> with no worries?

Best of luck. And I can feel my amnesia wearing off already. Time for
a dive.

_why

Works great under OS X and Ruby 1.8!

-rich

···

On Friday, March 14, 2003, at 02:04 AM, why the lucky stiff wrote:

citizens,

Here’s the deal. I became sick and delirious about a month ago. In my
deranged state, I felt it imperative that I should recode YAML.rb’s
parser in C. I have just now come to my senses, incident to the
powerful fumes of a passing onion truck. And yet, I am sitting before
a
new YAML parser that is as wide and wonderful as any of the world’s
finest onion trucks!

It is like I am an Olympic diver who has developed a serious case of
amnesia
mid-pike. There is nothing I can do now.

So I am please to present Syck, the new YAML parser/toy on the block.
I have included Ruby, Python and PHP extensions, though the last two
need some work. An OCaml extension is in progress as well.

Download here:
Download syck-0.08.tar.gz (YAML.rb: YAML for Ruby)

To compile the Ruby extension:
cd ext/ruby/
ruby extconf.rb
make
sudo make install

To load a document:

require ‘syck’
Syck.load( <<EOY )

  • one
  • two
  • three

Much is done but much is left to be done. The point is: I’m at version
0.08. And it’s time to start releasing early and releasing often.

Please jump in and voice your support for YAML in the Ruby
distribution!
YAML.rb has only been a module for eight months and has seen widespread
usage. Wouldn’t you like to <require ‘yaml’> with no worries?

Best of luck. And I can feel my amnesia wearing off already. Time for
a dive.

_why

I took a look at Syck's interface. So you do have your own hashing built
in. It's interesting because Syck is more than a parser, but less than a
loader. But you could easily build a push/pull wrapper over it, or build
a loader out of it. I kinda like it. (Especially if it works!)

I could actually see doing the Perl extension to Syck as
YAML::Parser::Syck or YAML::Loader::Syck. Eventually I'd want both. I
think I'll start with the Loader though.

Good work, Why. I'll be your Perl guy on this. (Aren't I always :slight_smile:

Cheers, Brian

PS This gives me a good reason to get back into Inline again. All my
projects will some day achieve oneness. Ingytopia. Pass the Dr Bronner's.

Cheers, Brian

···

On 14/03/03 00:16 -0700, why the lucky stiff wrote:

citizens,

Here's the deal. I became sick and delirious about a month ago. In my
deranged state, I felt it imperative that I should recode YAML.rb's
parser in C. I have just now come to my senses, incident to the
powerful fumes of a passing onion truck. And yet, I am sitting before a
new YAML parser that is as wide and wonderful as any of the world's
finest onion trucks!

It is like I am an Olympic diver who has developed a serious case of amnesia
mid-pike. There is nothing I can do now.

So I am please to present Syck, the new YAML parser/toy on the block.
I have included Ruby, Python and PHP extensions, though the last two
need some work. An OCaml extension is in progress as well.

You’re joking. Open up that Apple and make sure for me will ya? Wow. Gotta
love autoconf/automake!

Can anybody help get this sucker compiled on Windows? Solaris?

I’m already testing on:

  • Intel P3-800, FreeBSD 4.7, Ruby 1.6.8
  • Intel P-166, Deb Linux 2.2.19, Ruby 1.7.3
  • HP Server, FreeBSD 4.7, Ruby 1.6.6

_why

···

On Friday 14 March 2003 12:03 pm, Richard Kilmer wrote:

Works great under OS X and Ruby 1.8!

-rich

It’s interesting because Syck is more than a parser, but less than a
loader. But you could easily build a push/pull wrapper over it, or build
a loader out of it. I kinda like it. (Especially if it works!)

You can turn implicit typing on or off as you parse, so it really strives to
be both a parser and a loader, if you want it to. The parsing of implicit
types is separate, so you can use that independantly as well. (In an XML
parser, for instance. :D)

I could actually see doing the Perl extension to Syck as
YAML::Parser::Syck or YAML::Loader::Syck. Eventually I’d want both. I
think I’ll start with the Loader though.

YAML.rb + Syck sports similiar fashion. YAML::Loader::Syck is the base class
of YAML::Parser::Syck. A different handler is used in the loader and
implicits are turned off.

_why

···

On Friday 14 March 2003 12:48 pm, Brian Ingerson wrote:

In article 20030314114816.B29888@ttul.org,

citizens,

Here’s the deal. I became sick and delirious about a month ago. In my
deranged state, I felt it imperative that I should recode YAML.rb’s
parser in C. I have just now come to my senses, incident to the
powerful fumes of a passing onion truck. And yet, I am sitting before a
new YAML parser that is as wide and wonderful as any of the world’s
finest onion trucks!

It is like I am an Olympic diver who has developed a serious case of amnesia
mid-pike. There is nothing I can do now.

So I am please to present Syck, the new YAML parser/toy on the block.
I have included Ruby, Python and PHP extensions, though the last two
need some work. An OCaml extension is in progress as well.

I took a look at Syck’s interface. So you do have your own hashing built
in. It’s interesting because Syck is more than a parser, but less than a
loader. But you could easily build a push/pull wrapper over it, or build
a loader out of it. I kinda like it. (Especially if it works!)

It does indeed look cool. If illness is the inspiration that leads to
creation then perhaps… uhhh, well no, I don’t want to go through what
why had.

I could actually see doing the Perl extension to Syck as
YAML::Parser::Syck or YAML::Loader::Syck. Eventually I’d want both. I
think I’ll start with the Loader though.

Good work, Why. I’ll be your Perl guy on this. (Aren’t I always :slight_smile:

One of these days, Brian, you’ll be the Ruby guy on one of these projects
:wink:

Cheers, Brian

PS This gives me a good reason to get back into Inline again. All my
projects will some day achieve oneness. Ingytopia. Pass the Dr Bronner’s.

The 18-in-one Hemp Peppermint please!

Phil

···

Brian Ingerson ingy@ttul.org wrote:

On 14/03/03 00:16 -0700, why the lucky stiff wrote:

why the lucky stiff wrote:

You’re joking. Open up that Apple and make sure for me will ya? Wow. Gotta
love autoconf/automake!

Can anybody help get this sucker compiled on Windows? Solaris?

I’m already testing on:

  • Intel P3-800, FreeBSD 4.7, Ruby 1.6.8
  • Intel P-166, Deb Linux 2.2.19, Ruby 1.7.3
  • HP Server, FreeBSD 4.7, Ruby 1.6.6

_why

Works great under OS X and Ruby 1.8!

-rich

Dies during ‘make’ on Solaris 9, sparc
GNU bison 1.50, bison-1.875-sol9-sparc-local package (the latest on
sunfreeware)
autoconf (GNU Autoconf) 2.54
automake (GNU automake) 1.7
gcc 3.2

bison -y -d -t -v gram.y && mv y.tab.c gram.c
gram.y:47.9-52.4: type clash (nodeId' ‘) on default action
gram.y:52.6: parse error, unexpected “:”, expecting “;” or “|”
gram.y:55.9-61.62: $1 of doc' has no declared type gram.y:64.9-69.61: $1 of doc’ has no declared type
gram.y:90.13-91.41: $1 of doc' has no declared type gram.y:102.13-103.31: $1 of doc’ has no declared type
gram.y:108.13-109.23: $1 of doc' has no declared type gram.y:117.13-118.42: $1 of doc’ has no declared type
gram.y:121.4-126.23: $1 of doc' has no declared type gram.y:147.17-148.41: $1 of doc’ has no declared type
gram.y:151.5-152.36: $1 of doc' has no declared type gram.y:151.5-152.40: $3 of doc’ has no declared type
gram.y:151.5-153.27: $1 of doc' has no declared type gram.y:169.17-170.83: $1 of doc’ has no declared type
gram.y:173.5-174.36: $1 of doc' has no declared type gram.y:173.5-174.82: $3 of doc’ has no declared type
gram.y:173.5-175.27: $1 of doc' has no declared type gram.y:187.17-189.68: $1 of doc’ has no declared type
gram.y:187.17-190.68: $3 of doc' has no declared type gram.y:202.17-205.68: invalid $ value gram.y:202.17-205.68: $5 of doc’ has no declared type
gram.y:209.5-210.27: $1 of doc' has no declared type gram.y:213.17-214.39: $1 of doc’ has no declared type
gram.y:213.17-214.43: $3 of doc' has no declared type gram.y:213.17-215.27: $1 of doc’ has no declared type
gram.y:222.17-224.68: $1 of doc' has no declared type gram.y:222.17-225.68: $3 of doc’ has no declared type
gram.y:238.5-239.12: $1 of doc' has no declared type gram.y:242.5-243.39: $1 of doc’ has no declared type
gram.y:242.5-243.43: $3 of doc' has no declared type gram.y:242.5-244.27: $1 of doc’ has no declared type
make[2]: *** [gram.c] Error 1
make[2]: Leaving directory
/home/djberge/local/modules/ruby/syck-0.08/lib' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory /home/djberge/local/modules/ruby/syck-0.08’
make: *** [all] Error 2

I haven’t tried building bison from scratch, but I can do it if you
think it will help.

Regards,

Dan

···

On Friday 14 March 2003 12:03 pm, Richard Kilmer wrote:

When I get this with lynx or with wget I get something that claims
to be ascii but I can’t edit with vim. So I can’t test solaris.

    Hugh
···

On Sat, 15 Mar 2003, why the lucky stiff wrote:

Can anybody help get this sucker compiled on Windows? Solaris?

> It's interesting because Syck is more than a parser, but less than a
> loader. But you could easily build a push/pull wrapper over it, or build
> a loader out of it. I kinda like it. (Especially if it works!)
>

You can turn implicit typing on or off as you parse, so it really strives to
be both a parser and a loader, if you want it to. The parsing of implicit
types is separate, so you can use that independantly as well. (In an XML
parser, for instance. :D)

So how do you think that specific type handling is going to be
cross-language? If you can fully load a !foo {} from Syck, there's a
problem. Because it assuredly won't work from perl. Your syck node
should contain a kind and a type. And the program using Syck should call
the class method to create the foo object.

I would argue against putting Ruby only features in Syck. Put those in a
separate C library.

> I could actually see doing the Perl extension to Syck as
> YAML::Parser::Syck or YAML::Loader::Syck. Eventually I'd want both. I
> think I'll start with the Loader though.
>

YAML.rb + Syck sports similiar fashion. YAML::Loader::Syck is the base class
of YAML::Parser::Syck. A different handler is used in the loader and
implicits are turned off.

Actually I do things a bit differently in Perl. A YAML::Parser is
definitely not a subclass of YAML::Loader. A Loader object contains a
parser object, which keeps them totally orthoganal. So my end user
interface is something like this:

    use YAML;

    $y = YAML->new;
    $y->loader('YAML::Loader::Standard');
    $y->loader->parser('YAML::Parser::Syck');

    $y->loader->parser->string(<<END_YAML);

···

On 14/03/03 13:20 -0700, why the lucky stiff wrote:

On Friday 14 March 2003 12:48 pm, Brian Ingerson wrote:

    ---
    foo: barney
    END_YAML
    
    $data = $y->loader->next;

That's long for:

    use YAML;

    $YAML::Loader = 'YAML::Loader::Standard';
    $YAML::Parser = 'YAML::Parser::Syck';

    $data = Load <<END_YAML;
    ---
    foo: barney
    END_YAML

The nice thing is that I can switch out any component. When this really
pays is when end users want to create their own Loaders. Let me show you
how this would happen in Perl:

Say that a user needed a Loader that loaded scalars as strings except
for dates and currency. Say they also wanted to load strings that looked
like RGB values as color objects:

This is what their entire Perl YAML Loader module would look like:

    package My::YAML::Loader;

    # set up inheritance
    use base 'YAML::Loader::Date',
             'YAML::Loader::Currency',
             'YAML::Loader::Standard';

    # associate this class with certain implicit values
    sub yaml_match {
        my ($self, $node) = @_;
        # match strings like 'ffcc55'
        return ($node->value =~ /[0-9a-f]{6}/);
    }

    sub yaml_load {
        my ($self, $node) = @_;
        return Color->new($node->value);
    }

    1;

And they use it like this:

    use YAML;

    $YAML::Loader = 'My::YAML::Loader';

    $data = Load <<END_YAML;
    ---
    red: FF0000
    white: FFFFFF
    blue: 0000FF
    END_YAML

I hope that gives a clearer indication of where I want to go with this.

Cheers, Brian

“why the lucky stiff” ruby-talk@whytheluckystiff.net wrote in message

Can anybody help get this sucker compiled on Windows? Solaris?

How does one “make” it on Windows ? Or are you implying cygwin ?

I was using bison 1.28. I just upgraded to 1.75 and am getting the same list
of errors that you’re getting. It appears that I need to add some
definitions to my parser.

Next release won’t require bison or re2c. Sorry about that. Thanks for
trying. Didn’t expect an answer so quickly. (Ah, the benefits of
ruby-talk.)

_why

···

On Friday 14 March 2003 01:40 pm, Daniel Berger wrote:

Dies during ‘make’ on Solaris 9, sparc
GNU bison 1.50, bison-1.875-sol9-sparc-local package (the latest on
sunfreeware)
autoconf (GNU Autoconf) 2.54
automake (GNU automake) 1.7
gcc 3.2

bison -y -d -t -v gram.y && mv y.tab.c gram.c
gram.y:47.9-52.4: type clash (nodeId' ') on default action
……

So how do you think that specific type handling is going to be
cross-language? If you can fully load a !foo {} from Syck, there’s a
problem. Because it assuredly won’t work from perl. Your syck node
should contain a kind and a type. And the program using Syck should call
the class method to create the foo object.

A SyckNode does include a kind and type.

[from syck.h]
struct _syck_node {
// Symbol table ID
SYMID id;
// Underlying kind
enum syck_kind_tag kind;
// Fully qualified tag-uri for type
char *type_id;
// Anchor name
char *anchor;
union { … } data;
}

If implicits are turned on then only the types from the central repository are
tagged.

  • [ “12”, ! “12”, !foo “12” ]

In the above example, you’d have both scalar nodes listed as kind
syck_str_kind'. The nodes would be typed tag:yaml.org,2002:str’,
tag:yaml.org,2002:int' and tag:yaml.org,2002:foo’, respectively. It’s an
optimization.

One of the bottlenecks in YAML.rb has been using regular expressions to detect
implicits.

I would argue against putting Ruby only features in Syck. Put those in a
separate C library.

Absolutely. I have wondered about the possibility of having !syck/* types,
though. A user recently asked if Ruby symbols could be added to the central
type repository. I don’t know if symbols belong in the central type
repository or in a repository shared by languages who use a similiar loader.

Actually I do things a bit differently in Perl. A YAML::Parser is
definitely not a subclass of YAML::Loader. A Loader object contains a
parser object, which keeps them totally orthoganal.

Hmm. Hmmmm. I’ll think about this. I can see the advantages. My loader
classes are just shells that farm the loading off to callbacks registered by
the user. I don’t know what the implications are of having the loader become
so easily decoupled. If a user changes the loader, then who knows how
documents would be loaded. I guess that’s the idea but I can see it being
the source of bugs.

require ‘yaml’ # Defaults to YAML::Loader::Syck
require ‘okay/rpc’
require ‘jackslib’ # Switches to YAML::Loader::Jacks
# for a spell.

Are the !okay/rpc types registered with Jack’s loader?

orc = Okay::RPC::Client.new( ‘whytheluckystiff.net’, ‘/okayRpc/’ )
p orc.call( ‘system.listMethods’ )

User emails why…

I’d prefer it if Jack subclassed the YAML::Parser and used his loader class
directly rather than affecting the loading of YAML throughout the system. I
can see wanting to switch loaders out (Syck, libyaml, pure Ruby).

Anyway, you’re a BDUFer and bless you for it, BI. I’m a
see-you-at-the-finish-liner.

This kid just walked into the room with a tae-kwon-do outfit. I’m going to
see what he wants.

_why

···

On Friday 14 March 2003 02:54 pm, Brian Ingerson wrote:

bison -y -d -t -v gram.y && mv y.tab.c gram.c
gram.y:47.9-52.4: type clash (nodeId' ') on default action
……

Ok. These errors are caused by missing semicolons at the end of each of rule.
So:

implicit_map : IOPEN in_implicit_map IEND
{
$$ = $2;
}

Should be:

implicit_map : IOPEN in_implicit_map IEND
{
$$ = $2;
}
;

See gram.y below.

_why

//
// gram.y
//
// $Author: whythluckystiff $
// $Date: 2003/03/14 05:29:26 $
//
// Copyright (C) 2003 why the lucky stiff
//

%start doc
%pure-parser

%{

#include “syck.h”

#define YYPARSE_PARAM parser
#define YYLEX_PARAM parser

%}

%union {
SYMID nodeId;
SyckNode *nodeData;
char *name;
};

%token ANCHOR ALIAS TRANSFER ITRANSFER
%token WORD PLAIN BLOCK
%token DOCSEP IOPEN INDENT IEND

%type doc basic_seq
%type atom word_rep struct_rep atom_or_empty
%type implicit_seq inline_seq implicit_map inline_map
%type in_implicit_seq in_inline_seq basic_mapping basic_mapping2
%type in_implicit_map in_inline_map complex_mapping

%left ‘-’ ‘:’
%left ‘+’ ‘[’ ‘]’ ‘{’ ‘}’ ‘,’ ‘?’

%%

doc : struct_rep
{
((SyckParser *)parser)->root = syck_hdlr_add_node( (SyckParser
*)parser, $1 );
}
> DOCSEP atom
{
((SyckParser *)parser)->root = syck_hdlr_add_node( (SyckParser
*)parser, $2 );
}
;

atom : word_rep
> struct_rep
> ANCHOR atom
{
/*
* Anchors: The language binding must keep a separate symbol
table
* for anchors. The actual ID in the symbol table is returned to
the
* higher nodes, though.
*/
$$ = syck_hdlr_add_anchor( (SyckParser )parser, $1, $2 );
}
> ALIAS
{
/

* Aliases: The anchor symbol table is scanned for the anchor
name.
* The anchor’s ID in the language’s symbol table is returned.
*/
$$ = syck_hdlr_add_alias( (SyckParser *)parser, $1 );
}
> IOPEN atom IEND
{
$$ = $2;
}
;

atom_or_empty : atom
>
{
SyckNode *n = syck_new_str( “” );
n->type_id = “null”;
$$ = n;
}
;

//
// Words are broken out to distinguish them
// as keys in implicit maps and valid elements
// for the inline structures
//
word_rep : TRANSFER word_rep
{
$$ = syck_add_transfer( $1, $2 );
}
> ITRANSFER word_rep
{
if ( ((SyckParser *)parser)->implicit_typing == 1 )
{
try_tag_implicit( $2 );
}
$$ = $2;
}
> WORD
{
SyckNode *n = $1;
n->type_id = “str”;
$$ = n;
}
> PLAIN
{
$$ = $1;
}
;

//
// Any of these structures can be used as
// complex keys
//
struct_rep : TRANSFER struct_rep
{
$$ = syck_add_transfer( $1, $2 );
}
> BLOCK
{
//SyckNode *n = $3;
//syck_fold_format( $1, $3 );
//n->type_id = “str”;
//$$ = n;
$$ = $1;
}
> implicit_seq
> inline_seq
> implicit_map
> inline_map
;

//
// Implicit sequence
//
implicit_seq : IOPEN in_implicit_seq IEND
{
$$ = $2;
}
;

basic_seq : ‘-’ atom_or_empty
{
$$ = syck_hdlr_add_node( (SyckParser *)parser, $2 );
}
;

in_implicit_seq : basic_seq
{
$$ = syck_new_seq( $1 );
}
> in_implicit_seq INDENT basic_seq
{
syck_seq_add( $1, $3 );
$$ = $1;
}
;

//
// Inline sequences
//
inline_seq : ‘[’ in_inline_seq ‘]’
{
$$ = $2;
}
> ‘[’ ‘]’
{
$$ = syck_alloc_seq();
}
;

in_inline_seq : atom
{
$$ = syck_new_seq( syck_hdlr_add_node( (SyckParser
*)parser, $1 ) );
}
> in_inline_seq ‘,’ atom
{
syck_seq_add( $1, syck_hdlr_add_node( (SyckParser
*)parser, $3 ) );
$$ = $1;
}
;

//
// Implicit maps
//
implicit_map : IOPEN in_implicit_map IEND
{
$$ = $2;
}
;

basic_mapping : word_rep ‘:’ atom_or_empty
{
$$ = syck_new_map(
syck_hdlr_add_node( (SyckParser *)parser, $1 ),
syck_hdlr_add_node( (SyckParser *)parser, $3 ) );
}
;

complex_mapping : basic_mapping
> ‘?’ atom INDENT ‘:’ atom_or_empty
{
$$ = syck_new_map(
syck_hdlr_add_node( (SyckParser *)parser, $2 ),
syck_hdlr_add_node( (SyckParser *)parser, $5 ) );
}
;

in_implicit_map : complex_mapping
{
$$ = $1;
}
> in_implicit_map INDENT complex_mapping
{
syck_map_update( $1, $3 );
$$ = $1;
}
;

//
// Inline maps
//
basic_mapping2 : atom ‘:’ atom_or_empty
{
$$ = syck_new_map(
syck_hdlr_add_node( (SyckParser *)parser, $1 ),
syck_hdlr_add_node( (SyckParser *)parser, $3 ) );
}
;

inline_map : ‘{’ in_inline_map ‘}’
{
$$ = $2;
}
> ‘{’ ‘}’
{
$$ = syck_alloc_map();
}
;

in_inline_map : basic_mapping2
{
$$ = $1;
}
> in_inline_map ‘,’ basic_mapping2
{
syck_map_update( $1, $3 );
$$ = $1;
}
;

%%

···

On Friday 14 March 2003 01:40 pm, Daniel Berger wrote:

That's sourceforge nonsense. When you see a link with a target of
'syck-0.08.tar.gz' you think that would be the actual file, wouldn't you? It
isn't. It's a splash page giving you a choice of download locations.

Follow the link using right-arrow (not 'd') in lynx. You'll get to another
link like
http://prdownloads.sourceforge.net/yaml4r/syck-0.08.tar.gz?use_mirror=unc
which in turn links to
http://unc.dl.sourceforge.net/sourceforge/yaml4r/syck-0.08.tar.gz
which is the actual download on one particular mirror.

Regards,

Brian.

···

On Sat, Mar 15, 2003 at 05:47:20AM +0900, Hugh Sasse Staff Elec Eng wrote:

On Sat, 15 Mar 2003, why the lucky stiff wrote:
> Can anybody help get this sucker compiled on Windows? Solaris?

When I get this with lynx or with wget I get something that claims
to be ascii but I can't edit with vim. So I can't test solaris.

why the lucky stiff wrote:

bison -y -d -t -v gram.y && mv y.tab.c gram.c
gram.y:47.9-52.4: type clash (nodeId' ') on default action
……

Ok. These errors are caused by missing semicolons at the end of each of rule.
So:

make check
Making check in lib

make[2]: Entering directory
`/home/djberge/local/modules/ruby/syck-0.08/tests’

OK (4 tests)

PASS: test-basic

OK (5 tests)

PASS: test-parse

···

On Friday 14 March 2003 01:40 pm, Daniel Berger wrote:
==================
All 2 tests passed
==================

Looks like that did the trick! You can add “Solaris 9 (Sparc)” to the
list of supported platforms. :slight_smile:

Regards,

Dan


a = [74, 117, 115, 116, 32, 65, 110, 111, 116, 104, 101, 114, 32, 82]
a.push(117,98, 121, 32, 72, 97, 99, 107, 101, 114)
puts a.pack(“C*”)

>
> So how do you think that specific type handling is going to be
> cross-language? If you can fully load a !foo {} from Syck, there's a
> problem. Because it assuredly won't work from perl. Your syck node
> should contain a kind and a type. And the program using Syck should call
> the class method to create the foo object.
>

A SyckNode does include a kind and type.

  [from syck.h]
  struct _syck_node {
    // Symbol table ID
    SYMID id;
    // Underlying kind
    enum syck_kind_tag kind;
    // Fully qualified tag-uri for type
    char *type_id;
    // Anchor name
    char *anchor;
    union { ... } data;
  }

If implicits are turned on then only the types from the central
repository are tagged.

cool.

  - [ "12", ! "12", !foo "12" ]

In the above example, you'd have both scalar nodes listed as kind
`syck_str_kind'. The nodes would be typed `tag:yaml.org,2002:str',
`tag:yaml.org,2002:int' and `tag:yaml.org,2002:foo', respectively. It's an
optimization.

One of the bottlenecks in YAML.rb has been using regular expressions
to detect implicits.

Interesting. I wouldn't have guessed this. At least in Perl, regexp
operations are usually close to pure C. Not often the source of
bottlenecks unless the regexps are poorly written.

> I would argue against putting Ruby only features in Syck. Put those in a
> separate C library.

Absolutely. I have wondered about the possibility of having !syck/* types,
though. A user recently asked if Ruby symbols could be added to the central
type repository. I don't know if symbols belong in the central type
repository or in a repository shared by languages who use a similiar loader.

What are symbols?

I see no reason for not putting lots of types in the repository. That's
what it is there for. The common wisdom is that type checking is done at
the loader level though. If your loader is designed for multiple
inheritance, then you can simply add a new type class as new types are
added, without have to change the parser or core loader.

> Actually I do things a bit differently in Perl. A YAML::Parser is
> definitely not a subclass of YAML::Loader. A Loader object contains a
> parser object, which keeps them totally orthoganal.

Hmm. Hmmmm. I'll think about this. I can see the advantages. My loader
classes are just shells that farm the loading off to callbacks registered by
the user. I don't know what the implications are of having the loader become
so easily decoupled. If a user changes the loader, then who knows how
documents would be loaded. I guess that's the idea but I can see it being
the source of bugs.

  require 'yaml' # Defaults to YAML::Loader::Syck
  require 'okay/rpc'
  require 'jackslib' # Switches to YAML::Loader::Jacks
                       # for a spell.

  # Are the !okay/rpc types registered with Jack's loader?
  orc = Okay::RPC::Client.new( 'whytheluckystiff.net', '/okayRpc/' )
  p orc.call( 'system.listMethods' )

  # User emails why...

I'd prefer it if Jack subclassed the YAML::Parser and used his loader class
directly rather than affecting the loading of YAML throughout the system. I
can see wanting to switch loaders out (Syck, libyaml, pure Ruby).

I am not suggesting that requiring a loader class effect the whole
system. That's ludicrous. Each YAML Load operation is done from a
particular YAML::Loader object. That Loader object may inherit certain
properties (like implicits) from various Loading classes, but it has one
predictable beahviour.

In fact if a load operation involved another load operation (like with
!include processing) then a separate loader object would be used. And
that separate object might be from a completely different class.

Anyway, you're a BDUFer and bless you for it, BI. I'm a
see-you-at-the-finish-liner.

Silly programmer, there's no basement in the Alamo, er I mean, there's
no finish line.

Anyway, you and I are the only ones currently implementing YAML stuff,
so we really shouldn't fight. It's all about peace and love.

This kid just walked into the room with a tae-kwon-do outfit. I'm going to
see what he wants.

KICK HIS ASS.

Cheers, Brian

···

On 14/03/03 15:57 -0700, why the lucky stiff wrote:

On Friday 14 March 2003 02:54 pm, Brian Ingerson wrote:

syck doesn’t build out of the box under FreeBSD-4.7 with its standard byacc.
I applied your new gram.y first. Compilation stops at:

byacc -d -d -t -v gram.y && mv y.tab.c gram.c
byacc: e - line 11 of “gram.y”, syntax error
%pure-parser
^
*** Error code 1

If I comment this line out, it builds without further errors. But ‘make
check’ then fails:

$ make check
Making check in lib
Making check in tests
make check-TESTS

OK (4 tests)

PASS: test-basic
Segmentation fault (core dumped)
FAIL: test-parse

···

===================
1 of 2 tests failed

*** Error code 1

Here’s the backtrace from the core dump:

$ gdb -c test-parse.core test-parse
GNU gdb 4.18 (FreeBSD)
Copyright 1998 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type “show copying” to see the conditions.
There is absolutely no warranty for GDB. Type “show warranty” for details.
This GDB was configured as “i386-unknown-freebsd”…Deprecated bfd_read called at /usr/src/gnu/usr.bin/binutils/gdb/…/…/…/…/contrib/gdb/gdb/dbxread.c line 2627 in elfstab_build_psymtabs
Deprecated bfd_read called at /usr/src/gnu/usr.bin/binutils/gdb/…/…/…/…/contrib/gdb/gdb/dbxread.c line 933 in fill_symbuf

Core was generated by `test-parse’.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/lib/libc.so.4…done.
Reading symbols from /usr/libexec/ld-elf.so.1…done.
#0 syck_hdlr_add_node (p=0x8057000, n=0x0) at handler.c:16
16 if ( ! n->id ) n->id = (p->handler)( p, n );
(gdb) bt
#0 syck_hdlr_add_node (p=0x8057000, n=0x0) at handler.c:16
#1 0x804a906 in yyparse (parser=0x8057000) at gram.y:230
#2 0x8048bc0 in TestSyckParseString (tc=0x80500a0) at Parse.c:95
#3 0x80491dd in CuTestRun (tc=0x80500a0) at CuTest.c:166
#4 0x80492c3 in CuSuiteRun (testSuite=0x8053000) at CuTest.c:211
#5 0x8048db2 in main () at Parse.c:155
#6 0x8048815 in _start ()
(gdb) p n
$1 = (SyckNode *) 0x0
(gdb) p p
$2 = (SyckParser *) 0x8057000

Regards,

Brian.

> When I get this with lynx or with wget I get something that claims
> to be ascii but I can't edit with vim. So I can't test solaris.

That's sourceforge nonsense. When you see a link with a target of
'syck-0.08.tar.gz' you think that would be the actual file, wouldn't you? It
isn't. It's a splash page giving you a choice of download locations.

Oh, I see, thanks. An htm(l?) extension would help, but still.

Regards,

Brian.

        Hugh

···

On Sat, 15 Mar 2003, Brian Candler wrote:

On Sat, Mar 15, 2003 at 05:47:20AM +0900, Hugh Sasse Staff Elec Eng wrote:

One of the bottlenecks in YAML.rb has been using regular expressions
to detect implicits.

Interesting. I wouldn’t have guessed this. At least in Perl, regexp
operations are usually close to pure C. Not often the source of
bottlenecks unless the regexps are poorly written.

See the problem is having to use the ten or twenty regexps that handle
int, float, timestamps, nulls, nools, etc. The tokenizer in C can
whittle that down to what can be thought of as a single expression.

What are symbols?

A symbol is an entry in the symbol table represented by a name. In
Ruby, symbols are designated by a colon followed by word characters.

For sumbols, the following is true: :Fred.id == :Fred.id
Whereas for strings: “Fred”.id != “Fred”.id

(ID is the entry number in the symbol table.)

Symbols are very efficient and often serialized. (In comparison to
classes like Range and Regexp, which are serialized less often.)

I see no reason for not putting lots of types in the repository. That’s
what it is there for. The common wisdom is that type checking is done at
the loader level though. If your loader is designed for multiple
inheritance, then you can simply add a new type class as new types are
added, without have to change the parser or core loader.

Cool.

I am not suggesting that requiring a loader class effect the whole
system. That’s ludicrous. Each YAML Load operation is done from a
particular YAML::Loader object. That Loader object may inherit certain
properties (like implicits) from various Loading classes, but it has one
predictable beahviour.

In fact if a load operation involved another load operation (like with
!include processing) then a separate loader object would be used. And
that separate object might be from a completely different class.

Ok. I see your angle. YAML.rb is similar.

_why

···

Brian Ingerson (ingy@ttul.org) wrote:

Yeah, that directive is essential at the moment. I didn’t realize it was
exclusive to bison. I can think of a couple ways to make this work with
byacc and automake. I’ll play with it and let ya know.

_why

···

On Saturday 15 March 2003 02:37 am, Brian Candler wrote:

byacc -d -d -t -v gram.y && mv y.tab.c gram.c
byacc: e - line 11 of “gram.y”, syntax error
%pure-parser
^
*** Error code 1

If I comment this line out, it builds without further errors. But ‘make
check’ then fails: