How does gem deal with extensions?

Hi, all!

I'm building a gem that contains some C extensions. Here's
the directory hierarchy:

lib/
  rmmseg.rb
  rmmseg/
    foo.rb
    bar.rb
ext/
  rmmseg/
    extconf.rb
    ext.c

After installing the gem. The ext.so get built in ext/rmmseg/ . It is OK
since the require path contains ext. I can require it using 'rmmseg/ext'.
However, I found the ext.so also gets copied to lib/ directory. What's the
purpose? Can I disable this? Thanks!