[Q] specify start postion of Regexp matching

Hi, all.

Is it possible to specify start position of Regexp matching?

    str = "foo bar baz"
    m = /ba/.match(str)
    p m.begin(0) #=> 4
    m = /ba/.match(str, 5) # is it possible?
    p m.begin(0) #=> 8 (if possible)

If it is possible, some kind of parser or scanner can be
implemented easily.
# StringScanner is a litte too big, I think.

···

--
makoto kuwata

You could try something like this:

    m = /^.{5,}(ba)/.match(str)
    p m.begin(1)

In the regular expression, you're saying start at the beginning and
skip at least 5 characters. But then we have to use parens to "note"
the part you're interested in, and then we have to pass 1 rather than
0 to begin, so it reports the location of the first noted match (0
would report where the entire Regexp matched, and that would be the
beginning of the line).

An alternative would be to slice the first n characters off the front
of the string and then do the match.

Eric

···

On Nov 25, 10:18 am, makoto kuwata <k...@kuwata-lab.com> wrote:

Hi, all.

Is it possible to specify start position of Regexp matching?

    str = "foo bar baz"
    m = /ba/.match(str)
    p m.begin(0) #=> 4
    m = /ba/.match(str, 5) # is it possible?
    p m.begin(0) #=> 8 (if possible)

If it is possible, some kind of parser or scanner can be
implemented easily.
# StringScanner is a litte too big, I think.

====

Interested in hands-on, on-site Ruby training? See http://LearnRuby.com
for information about a well-reviewed class.

-------- Original-Nachricht --------

Datum: Mon, 26 Nov 2007 00:20:25 +0900
Von: makoto kuwata <kwa@kuwata-lab.com>
An: ruby-talk@ruby-lang.org
Betreff: [Q] specify start postion of Regexp matching

Hi, all.

Is it possible to specify start position of Regexp matching?

    str = "foo bar baz"
    m = /ba/.match(str)
    p m.begin(0) #=> 4
    m = /ba/.match(str, 5) # is it possible?
    p m.begin(0) #=> 8 (if possible)

If it is possible, some kind of parser or scanner can be
implemented easily.
# StringScanner is a litte too big, I think.

--
makoto kuwata

Dear Makoto,

what about :

class Regexp
  def match_index_offset(string,start_pos)
    temp=string[start_pos..-1]
    ref=self.match(temp)
    return temp.index(ref[0])+start_pos
  end
end

str = "foo bar baz"
m = /ba/.match_index_offset(str,5)
p m

Best regards,

Axel

···

--
Psssst! Schon vom neuen GMX MultiMessenger gehört?
Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger

Hi,

···

In message "Re: [Q] specify start postion of Regexp matching" on Mon, 26 Nov 2007 00:20:25 +0900, makoto kuwata <kwa@kuwata-lab.com> writes:

Is it possible to specify start position of Regexp matching?

   str = "foo bar baz"
   m = /ba/.match(str)
   p m.begin(0) #=> 4
   m = /ba/.match(str, 5) # is it possible?
   p m.begin(0) #=> 8 (if possible)

str.index(/ba/, 5) ?

              matz.

Another alternative is to use String#scan - we would have to know what the OP really wants to parse though to decide whether it's a feasible solution.

Kind regards

  robert

···

On 25.11.2007 16:39, Eric I. wrote:

On Nov 25, 10:18 am, makoto kuwata <k...@kuwata-lab.com> wrote:

Hi, all.

Is it possible to specify start position of Regexp matching?

    str = "foo bar baz"
    m = /ba/.match(str)
    p m.begin(0) #=> 4
    m = /ba/.match(str, 5) # is it possible?
    p m.begin(0) #=> 8 (if possible)

If it is possible, some kind of parser or scanner can be
implemented easily.
# StringScanner is a litte too big, I think.

You could try something like this:

    m = /^.{5,}(ba)/.match(str)
    p m.begin(1)

In the regular expression, you're saying start at the beginning and
skip at least 5 characters. But then we have to use parens to "note"
the part you're interested in, and then we have to pass 1 rather than
0 to begin, so it reports the location of the first noted match (0
would report where the entire Regexp matched, and that would be the
beginning of the line).

An alternative would be to slice the first n characters off the front
of the string and then do the match.

Thank you, all.

Eric l wrote:

You could try something like this:
    m = /^.{5,}(ba)/.match(str)
    p m.begin(1)

In my program, start position is variable such as
     def f(n)
       m = /^.{n,}(ba)/.match(str)
       ...
     end
In this case, /^.{n,}(ba)/ is created for each time.
It is not effective.

Robert Klemme wrote:

Another alternative is to use String#scan -

String#scan is useful only when regexp pattern is fixed.
    input.scan(/FIXED-REGEXP/) do ... end
Using String#scan, it is not able to change regexp pattern
in the loop.

Axel Etzold wrote:

               temp=string[start_pos..-1]
               ref=self.match(temp)
               return temp.index(ref[0])+start_pos

In this solution, temp substring is created every time.
If input string is long, it is not efficient.

Thanks to all your advices.
I'm going to propose to support start position in Regexp#match().

···

--
makoto kuwata

No, String#index returns Fixnum (position), but I want MatchData.

Regexp#match(string, start=0) in Ruby1.9 is the best solution I want.
Is there any plan to implement it into Ruby1.8?

···

Yukihiro Matsumoto <m...@ruby-lang.org> wrote:

str.index(/ba/, 5) ?

--
makoto kuwata

Robert Klemme wrote:

Another alternative is to use String#scan -

String#scan is useful only when regexp pattern is fixed.
    input.scan(/FIXED-REGEXP/) do ... end
Using String#scan, it is not able to change regexp pattern
in the loop.

But in various situations it is possible to use a unified regexp for scanning or a regexp that comprises all other patterns.

Axel Etzold wrote:

               temp=string[start_pos..-1]
               ref=self.match(temp)
               return temp.index(ref[0])+start_pos

In this solution, temp substring is created every time.
If input string is long, it is not efficient.

This is not true. Creating a substring is fairly cheap because the character buffer is not copied (copy on write).

I'm going to propose to support start position in Regexp#match().

For the time being it's faster to use one of the other alternatives. Also, with the new regexp engine in 1.9 your feature might be present already.

Kind regards

  robert

···

On 25.11.2007 18:58, makoto kuwata wrote:

I found that it is able to get MatchData by Regexp.last_match()
after String#index().
Well, I think Regexp#match(string, start=0) is the natural way,
but String#index(regexp, start) can be the good solution.

Thank you, Matz.

···

makoto kuwata <k...@kuwata-lab.com> wrote:

> str.index(/ba/, 5) ?

No, String#index returns Fixnum (position), but I want MatchData.

--
makoto kuwata

> In this solution, temp substring is created every time.
> If input string is long, it is not efficient.

This is not true. Creating a substring is fairly cheap because the
character buffer is not copied (copy on write).

You are right. If input string is not modified, creating substring
doesn't copy anything.
Creating substring may be the solution I wanted.

> I'm going to propose to support start position in Regexp#match().

For the time being it's faster to use one of the other alternatives.
Also, with the new regexp engine in 1.9 your feature might be present
already.

I found that Regexp#match() can take optional 2nd argument which
specifies matching start position in Ruby1.9. Good news.

Thank you, Robert.

···

Robert Klemme <shortcut...@googlemail.com> wrote:

--
makoto kuwata

What's the difference between 1.9 Regexp#match(string, start=n) and
1.8 Regexp#match(string[n..-1])?? You have to create a sub-string with
the 1.8 version, but according to Robert Klemme (above) it's just
creating a pointer into the original string if you're not changing the
substring or original string. Besides, even if you did get a copy,
it's anonymous and should be garbage collected soon. If I understand
everything correctly, the 1.9 version would just basically be a
convenience feature over the 1.8 way?

$ irb19
irb(main):001:0> RUBY_VERSION
=> "1.9.0"
irb(main):002:0> m = /oo/.match("foo", start=1)
=> #<MatchData "oo">
irb(main):003:0> m[0]
=> "oo"

$ irb
irb(main):001:0> RUBY_VERSION
=> "1.8.6"
irb(main):002:0> m = /oo/.match("foo"[1..-1])
=> #<MatchData:0xb78777a8>
irb(main):003:0> m[0]
=> "oo"

Regards,
Jordan

Jordan Callicoat wrote:

You have to create a sub-string with
the 1.8 version, but according to Robert Klemme (above) it's just
creating a pointer into the original string if you're not changing the
substring or original string.

I'm having a hard time confirming that:

str = "hello"
sub_str = str[1, 2]

puts str.object_id
-->76750

puts sub_str.object_id
-->76740

puts sub_str.class
-->String

···

--
Posted via http://www.ruby-forum.com/\.

I'm not sure how to confirm it, other than just looking at the source,
and since I'm very poor at C programming, it probably wouldn't help
for me to try that. I'm sure Robert can demonstrate. But I will say
that I'm not suprised that they have different object_id, because they
are different objects. The copy on write is just a back-end
optimization where you pretend that two objects that point to the same
data are unique copies in the front-end, but you don't actually move
any data in the back-end until you have to (i,e., when one of the
objects is changed).

Regards,
Jordan

···

On Nov 26, 12:07 am, 7stud -- <bbxx789_0...@yahoo.com> wrote:

Jordan Callicoat wrote:
> You have to create a sub-string with
> the 1.8 version, but according to Robert Klemme (above) it's just
> creating a pointer into the original string if you're not changing the
> substring or original string.

I'm having a hard time confirming that:

A new ruby object is created, but the string buffer that it points to
is only copied on write.

···

On Nov 26, 2007 5:07 PM, 7stud -- <bbxx789_05ss@yahoo.com> wrote:

Jordan Callicoat wrote:
> You have to create a sub-string with
> the 1.8 version, but according to Robert Klemme (above) it's just
> creating a pointer into the original string if you're not changing the
> substring or original string.

I'm having a hard time confirming that:

str = "hello"
sub_str = str[1, 2]

puts str.object_id
-->76750

puts sub_str.object_id
-->76740

puts sub_str.class
-->String

Well, I did anyhow...

http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8/ruby.h
http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8/string.c

And I think the functions of interest are str_new3 and str_new4
(called from rb_str_substr). Specifically, the assignment of
RSTRING(str2)->aux.shared. But like I said, I'm not great with C, so I
could be mistaken.

Regards,
Jordan

···

On Nov 26, 12:45 am, MonkeeSage <MonkeeS...@gmail.com> wrote:

I'm not sure how to confirm it, other than just looking at the source,
and since I'm very poor at C programming, it probably wouldn't help
for me to try that.

Here's a test to show that my reading of the source, and Robert's
assertion, is correct (there is probably a better way to do this...):

#!/usr/bin/env ruby

# disable GC to get fair reading of actual allocation cost
GC.disable

def free_megs
   (`free -o`.split("\n")[1].split(' ')[3].to_i/1024).to_s
end

puts "Free megabytes " + free_megs
# make a one megabyte string
s1 = "a" * 1048576
s100 = "" # placeholder to be filled in below
# make 100 substrings of it
0.upto(101) { |i| eval("s#{i}=s1[0..-1]") }

puts s100.length.to_s
puts "Free megabytes " + free_megs

Output:

Free megabytes 588
1048576
Free megabytes 587

Only one meg is used, which is the length of the original string. So,
by inductive inference, the substrings are only pointers back to the
original string rather than copies of the data.

Regards,
Jordan