Project Directory Structure

ruby-talk
21

Hi Jim

Questions for a small ruby application. The application will not result
in a ruby library at all.

Hope your “Ruby Project Directory Structure” is also valid for a simple
application.

Good question. I have this scenario all the time.

a) There are a couple of .rb files which hold the ruby classes for the
application. Where do these files go to? bin/ or lib/

lib/

The executable can be installed in the ruby_path/bin or some
other path. When it calls require ‘my_lib’, it will find your .rb
classes installed under ruby_path/lib/ruby/site_ruby/1.x/ .

b) A ruby file is called start1.rb and start2.rb which starts the
applications (each creates an instance object of the main application,
sort of client-server). Does this file go into bin/ or does it go into
examples as it showes how to start the application or is it a
replacement for the setup.rb file?

When I used setup.rb I was thinking of the app written by Minero
Aoki. From what I can tell, most raa packages are installed via:
ruby install.rb setup
ruby install.rb config
ruby install.rb install

I would say that if your startx apps are absolutely required to
run your app, then they should go in bin/. If, as you say, they
are examples, the they should go under examples.

  1. Also there are a couple of data (input files) for the application.
    Where do those data (input files) go?

That’s a good question. If the data files are configurable by
the user, it might be best to put them under $HOME/.myapprc,
or to read them from the command line.

In the case where they are static and part of the app database,
you may wnat to store them in the ruby_lib path.
I have a situation similar to this at work where we have an
app that requires a database of netlist templates. We want this
to be invisible to the user, so we have a class that controls
these templates. We put the templates in a directory under the
class and it finds them with:

 template_path = File.join(File.dirname(__FILE__), "template")

 ruby_path/lib/site_ruby/1.8/my_app/template.rb
 ruby_path/lib/site_ruby/1.8/my_app/template/template1
 ruby_path/lib/site_ruby/1.8/my_app/template/template2
···

On Tuesday, 27 May 2003 at 15:53:06 +0900, Sacha Schlegel wrote:

–
Jim Freeze

Cahn’s Axiom:
When all else fails, read the instructions.

22

Thanks. I’ll add TODO to the list. (I think it is optional, unlike
Manifest, which seems to be required by some package installers.)

···

On Tuesday, 27 May 2003 at 18:26:53 +0900, Robert Feldt wrote:

On Tue, 27 May 2003, Jim Freeze wrote:

Minor points if this is going to be a “complete” recommendation:

I sometimes/often use a TODO file at top-level.
LICENSE is sometimes called COPYING. I think LICENSE is better though.

–
Jim Freeze

A man said to the Universe: “Sir, I exist!”

“However,” replied the Universe, “the fact has not created in me a
sense of obligation.”
– Stephen Crane

23

CHANGELOG
INSTALL # optional
LICENSE
Manifest
README # if multi lang, README.en, README.ja, etc
TODO
setup.rb
bin/
doc/ # If multi lang, then use doc.en, doc.ja, etc
examples/
ext/
/ext_name?
extconf.rb
lib/
src/
/<other_lang_name>
test/

abbreviated names is NOT userfriendly to newbies.

I think your statement is a little dogmatic, but
I see your point. For a converse point, as a newby
I would be confused to see binary/ instead of bin/,
simply because I expected to see bin/.

I prefer long names (at least in the top-level dir):

src => source
doc => documents/text
bin => binary
ext => config/recipy/setup

I suppose you could swap the above list and use links:

source → src
documents → doc
binary → bin

(Personally, I like it when the names line up. :sunglasses: )

BTW: ‘data’ is also a useful top-level dir :slight_smile:

True.

···

On Tuesday, 27 May 2003 at 15:04:10 +0900, Simon Strandgaard wrote:

On Tue, 27 May 2003 09:57:30 +0900, Jim Freeze wrote:

–
Jim Freeze

The doctrine of human equality reposes on this: that there is no man
really clever who has not found that he is stupid.
– Gilbert K. Chesterson

24

abbreviated names is NOT userfriendly to newbies.
I prefer long names (at least in the top-level dir):

src => source
doc => documents/text
bin => binary
ext => config/recipy/setup

What does “recipy” mean? It’s not an English word.

= recipe… (im not a native english speaker)

I don’t think there’s any problem with ‘src’, ‘doc’, ‘bin’, ‘ext’. If you’re
a newbie, you’re going to have to learn what those things stand for anyway.

Yes you are going to learn those anyway.
Abbreviations might have saved a few keystrokes in the past.
But now people have either tab-completion or uses a GUI file manager.

I think Jim is documenting current best practice, not trying to invent
something which is different and incompatible with current practice.

3 letter abbreviations is NOT good practice :slight_smile:

···

On Tue, 27 May 2003 18:07:12 +0900, Brian Candler wrote:

On Tue, May 27, 2003 at 03:04:10PM +0900, Simon Strandgaard wrote:

–
Simon Strandgaard

25

Thanks for all the input. A description of the Project
Directory Structure and a quick ‘hack’ of a program to
build the skeleton are listed below.

Below is a suggested directory hierarchy.
All of the directories, except possibly test/ ;),
are optional. Most of the filenames listed
are optional but are listed for consistent
spelling and capitalization.

CHANGELOG
INSTALL
LICENSE
Manifest
README # if multi lang, README.en, README.ja, etc
TODO # optional
setup.rb
bin/
# ruby executables
# other executables
contrib/
# third party contributions
doc/ # If multi lang, then use doc.en, doc.ja, etc
man/ # for unix man
api/ # rdoc output
rd/ # if rd based docs
tutorial/
examples/
ext/
ext_name/
extconf.rb
… other source
lib/
# .rb files that go into RUBY_LIB
# may want to use parallel directory structure below
# site_ruby//
src/
<other_lang_name>/
test/
# test suites

···

#! /usr/bin/env ruby -w

require ‘fileutils’

def usage
<<-EOT
Usage: #{File.basename($0)} project_path
EOT
end

unless 1 == ARGV.size
STDERR.puts usage
exit 1
end

root = File.expand_path(ARGV.shift)
FileUtils.mkdir_p(root)

files = %w{
CHANGELOG
INSTALL
LICENSE
Manifest
README
TODO
setup.rb
}

dirs = %w{
bin
contrib
doc
doc/api
examples
ext
lib
src
test
}

files.each { |file|
path = File.join(root, file)
puts "Creating file: #{path}"
File.open(path,“w”) {}
}

dirs.each { |dir|
path = File.join(root, dir)
puts "Creating directory: #{path}"
FileUtils.mkdir_p(path)
}

% ./project_config.rb my_proj
Creating file: /usr/home/jfreeze/my_proj/CHANGELOG
Creating file: /usr/home/jfreeze/my_proj/INSTALL
Creating file: /usr/home/jfreeze/my_proj/LICENSE
Creating file: /usr/home/jfreeze/my_proj/Manifest
Creating file: /usr/home/jfreeze/my_proj/README
Creating file: /usr/home/jfreeze/my_proj/TODO
Creating file: /usr/home/jfreeze/my_proj/setup.rb
Creating directory: /usr/home/jfreeze/my_proj/bin
Creating directory: /usr/home/jfreeze/my_proj/contrib
Creating directory: /usr/home/jfreeze/my_proj/doc
Creating directory: /usr/home/jfreeze/my_proj/doc/api
Creating directory: /usr/home/jfreeze/my_proj/examples
Creating directory: /usr/home/jfreeze/my_proj/ext
Creating directory: /usr/home/jfreeze/my_proj/lib
Creating directory: /usr/home/jfreeze/my_proj/src
Creating directory: /usr/home/jfreeze/my_proj/test
% ls -F my_proj/
CHANGELOG Manifest bin/ examples/ setup.rb
INSTALL README contrib/ ext/ src/
LICENSE TODO doc/ lib/ test/

–
Jim Freeze

Some people are born mediocre, some people achieve mediocrity, and some
people have mediocrity thrust upon them.
– Joseph Heller, “Catch-22”

26

In article 20030527112738.C62761@freeze.org,

···

Jim Freeze jim@freeze.org wrote:

On Tuesday, 27 May 2003 at 15:04:10 +0900, Simon Strandgaard wrote:

On Tue, 27 May 2003 09:57:30 +0900, Jim Freeze wrote:

CHANGELOG
INSTALL # optional
LICENSE
Manifest
README # if multi lang, README.en, README.ja, etc
TODO
setup.rb
bin/
doc/ # If multi lang, then use doc.en, doc.ja, etc
examples/
ext/
/ext_name?
extconf.rb
lib/
src/
/<other_lang_name>
test/

abbreviated names is NOT userfriendly to newbies.

I think your statement is a little dogmatic, but
I see your point. For a converse point, as a newby
I would be confused to see binary/ instead of bin/,
simply because I expected to see bin/.

Yeah, I’m with you. I guess I’m thinking what’s the problem with bin and
src, they seem quite natural. But I suppose that’s because our worldview
is shaped by Unix, not Windows.

Phil

27

I beg to differ.

3 letter abbreviations USED TO BE good practise, because disk reads took
in the order of seconds and users doing a directory listing wanted to
load as few blocks from the disk as possible, which is achieved by making
the directory/command/file names as short as possible (eg usr, ls and
fstab).*

3 letter abbreviations are CURRENTLY good practise because standards are
good practise and those have been the standards for a long time. It is
far better, IMO, to put out the newbies for a short time while they get
a grasp on these cryptic names than it is to put out everyone else for a
long time while they change their long-standing habits.

Or, as someone I know once put it, there’s a world shortage of
keystrokes and you should do all you can to conserve them.

  • Short for “user”, “list” and “file system table”.

Tim Bates

···

On Tue, May 27, 2003 at 08:06:33PM +0900, Simon Strandgaard wrote:

3 letter abbreviations is NOT good practice :slight_smile:

–
tim@bates.id.au

28

If you have scripts that help pack the tarball etc, ie. scripts that
are used in the project but not included when shipped, where should they
go? (/bin?)

I tend to use a top-level dir “helpers” for these scripts currently but it
would be nice to see how others handle this.

Another comment is that I don’t like “examples” in pluralis but “test” in
singularis; I tend to use “tests”. However, most of the others are
singularis I guess… :wink:

Yet another issue is whether people recommend structure within test dir? I
have started using the same structure as in lib and ext ie. the unit tests
for the code in lib/cryptor/public_key goes into
tests/unit/cryptor/public_key etc (I use “tests/unit” since I have
acceptance tests in tests/acceptance but maybe that is taking things too
far…). I have felt the need to add structure to the tests dir since
test-first design often lead to as much (or more) test code as the code
itself. With one flat, big tests dir it was hard to get an overview. I
also have a single script tests/run_all_tests.rb so that the command

ruby -Ilib tests/run_all_tests.rb

runs them all.

Just my 2 cents…

Regards,

Robert

···

On Wed, 28 May 2003, Jim Freeze wrote:

CHANGELOG
INSTALL
LICENSE
Manifest
README # if multi lang, README.en, README.ja, etc
TODO # optional
setup.rb
bin/
# ruby executables
# other executables
contrib/
# third party contributions
doc/ # If multi lang, then use doc.en, doc.ja, etc
man/ # for unix man
api/ # rdoc output
rd/ # if rd based docs
tutorial/
examples/
ext/
ext_name/
extconf.rb
… other source
lib/
# .rb files that go into RUBY_LIB
# may want to use parallel directory structure below
# site_ruby//
src/
<other_lang_name>/
test/
# test suites

29

3 letter abbreviations are CURRENTLY good practice because standards are
good practice and those have been the standards for a long time.

There may be some marginal de facto naming standards, but, given that
we’re having this discussion, there is room for clarification and improvement.

I have a hard time believing that anyone would be baffled
by the use of ‘documents’ instead of ‘docs’, or ‘binary’
for ‘bin,’ and, in general, I prefer more-obvious naming conventions.

It is
far better, IMO, to put out the newbies for a short time while they get
a grasp on these cryptic names than it is to put out everyone else for a
long time while they change their long-standing habits.

Sounds like an argument for everyone to learn C with Hungarian notation.

:slight_smile:

James

30

If you have scripts that help pack the tarball etc, ie. scripts that
are used in the project but not included when shipped, where should they
go? (/bin?)

Good question. My intended use of bin is for what gets installed into
PREFIX/bin or ruby_path/bin.

I have not addressed project dependent packaging code.
I have not looked recently, but I assume that the package managers
can be configured to ignore certain directories. For example,
I think CVS is usually ignored. What is your opinion of using
directory names that begin with ‘.’ for this purpose?

I tend to use a top-level dir “helpers” for these scripts currently but it
would be nice to see how others handle this.

Another comment is that I don’t like “examples” in pluralis but “test” in
singularis; I tend to use “tests”. However, most of the others are
singularis I guess… :wink:

Yes, it is inconsistent. I have changed examples to example in my
script.

Yet another issue is whether people recommend structure within test dir? I
have started using the same structure as in lib and ext ie. the unit tests
for the code in lib/cryptor/public_key goes into
tests/unit/cryptor/public_key etc (I use “tests/unit” since I have
acceptance tests in tests/acceptance but maybe that is taking things too

Sorry to be stupid, but how do acceptance tests differ from unit
tests and why do you want to keep them seperate? Would not all
of them be necessary for full validation?

far…). I have felt the need to add structure to the tests dir since
test-first design often lead to as much (or more) test code as the code
itself. With one flat, big tests dir it was hard to get an overview. I

I have to agree. For larger projects, this would be a real benefit.
I will add that it is suggested that the directories under test
parallel the directories under lib.

also have a single script tests/run_all_tests.rb so that the command

ruby -Ilib tests/run_all_tests.rb

runs them all.

I use a top level make inside test and just do:

make

which recursively calls all my tests. I find this very useful,
but have not proved it for multiple OS’s (ie, may not be windows
friendly).

···

On Wednesday, 28 May 2003 at 1:45:56 +0900, Robert Feldt wrote:

On Wed, 28 May 2003, Jim Freeze wrote:

–
Jim Freeze

“Always try to do things in chronological order; it’s less confusing
that way.”

31

Another comment is that I don’t like “examples” in pluralis but “test” in
singularis; I tend to use “tests”. However, most of the others are
singularis I guess… :wink:

Well, “test” can be a verb, too, though suspect the use here is
as a noun. Seems, though, that if you have more than one of
something it should be plural. Seeing a directory named ‘example’
leads me to think there will be one example, and if there are
multiple files then they are all part of the same example.

James

32

I have a hard time believing that anyone would be baffled
by the use of ‘documents’ instead of ‘docs’, or ‘binary’
for ‘bin,’ and, in general, I prefer more-obvious naming
conventions.

Conversely, would anyone be baffled really by “doc” or “bin”?

···

Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.

33

Yet another issue is whether people recommend structure within test dir? I
have started using the same structure as in lib and ext ie. the unit tests
for the code in lib/cryptor/public_key goes into
tests/unit/cryptor/public_key etc (I use “tests/unit” since I have
acceptance tests in tests/acceptance but maybe that is taking things too

Sorry to be stupid, but how do acceptance tests differ from unit
tests and why do you want to keep them seperate? Would not all
of them be necessary for full validation?

Sure. I’m not sure its very meaningful but I tend to call tests with
externally spec’ed test data acceptance tests and rest unit tests.
Example is in my cryptography library where acceptance tests use
test data vectors for RSA encryption specified in PKCS #1 RSA paper from
RSA labs while unit tests for RSA encryption has a large number of tests
on random data, exceptional cases etc. I’m not 100% sure this separation
adds enough value to warrant the added “complexity” but I kind of like
it… :wink:

I think I like it because when writing the acceptance tests I typically
think about the externally visible API, the “end-product” delivered by the
app/lib/extension. I put on “a different hat” so to speak. Unit tests are
more nitty-gritty. Often the division is not meaningful though and
especially not in a dynamic language like Ruby where there is often no
idea in thinking about external and internal parts/classes; everything is
malleable.

also have a single script tests/run_all_tests.rb so that the command

ruby -Ilib tests/run_all_tests.rb

runs them all.

I use a top level make inside test and just do:

make

which recursively calls all my tests. I find this very useful,
but have not proved it for multiple OS’s (ie, may not be windows
friendly).

Pro with having a ruby script is that anyone installing a ruby extension
can be assumed to have Ruby while not everyone will have make (ok,
many/most have). Also the run_all_tests script can do “smart” things like
searching for the tests to load/run, only run tests that matches certain
regexps. No biggie of course and I’m sure you can also do it nicely with
make.

Regards,

Robert

···

On Wed, 28 May 2003, Jim Freeze wrote:

34

Jim Freeze wrote:

Sorry to be stupid, but how do acceptance tests differ from unit
tests?

The short answer is that:

  • acceptance tests are generally specified or written by
    the customer to ensure that the program does what it’s
    supposed to do, i.e. it meets it acceptance criteria.
  • unit tests are generally written by the code developer(s)
    to ensure that individual classes or components of the
    software are doing the right thing.

For some more ruminations, google for either “acceptance tests” or “unit
tests”. As usual, Ward Cunningham’s WikiWiki
(http://c2.com/cgi/wiki?WelcomeVisitors) has a lot of interesting pages
discussing these ideas…

35

Another comment is that I don’t like “examples” in pluralis but “test” in
singularis; I tend to use “tests”. However, most of the others are
singularis I guess… :wink:

Well, “test” can be a verb, too, though suspect the use here is
as a noun. Seems, though, that if you have more than one of

Yes, I think of it as the ‘test’ directory where ‘tests’ go. :slight_smile:

something it should be plural. Seeing a directory named ‘example’
leads me to think there will be one example, and if there are
multiple files then they are all part of the same example.

I agree, but I’m not sure I am correct. I can’t say why I am
comfortable with ‘test’ as singular and ‘examples’ as plural,
even though they are not parallel. Probably because I DON’T
think of it as the ‘example’ directory where ‘examples’ go. :slight_smile:

How do you feel about pluralism in the other dirs?

bin/ => bins/
lib/ => libs/
doc/ => docs/
contrib => contribs/

or are these ok without the ‘s’ suffix since they are abbreviations?

···

On Wednesday, 28 May 2003 at 14:31:49 +0900, james_b@neurogami.com wrote:

–
Jim Freeze

The rhino is a homely beast,
For human eyes he’s not a feast.
Farewell, farewell, you old rhinoceros,
I’ll stare at something less prepoceros.
– Ogden Nash

36

This means in practice that acceptance tests “imply” integration tests,
as they consider the whole system at once, for the customer judges the
behavior of the program, not of a single unit. Right?

···

On Wed, May 28, 2003 at 08:12:45AM +0900, Lyle Johnson wrote:

Jim Freeze wrote:

Sorry to be stupid, but how do acceptance tests differ from unit
tests?

The short answer is that:

  • acceptance tests are generally specified or written by
    the customer to ensure that the program does what it’s
    supposed to do, i.e. it meets it acceptance criteria.
  • unit tests are generally written by the code developer(s)
    to ensure that individual classes or components of the
    software are doing the right thing.

–
_ _

__ __ | | ___ _ __ ___ __ _ _ __
'_ \ / | __/ __| '_ _ \ / ` | ’ \
) | (| | |__ \ | | | | | (| | | | |
.__/ _,|_|/| || ||_,|| |_|
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

  • JHM wonders what Joey did to earn “I’d just like to say, for the record,
    that Joey rules.”
    – Seen on #Debian
37

Bins are garbage receptacles :slight_smile:

Hmm:

$ /usr/bins/pwd
/usr/homes/brian
$ grep brian /etcs/passwds

No, I don’t think so…

Regards,

Brian.

···

On Wed, May 28, 2003 at 07:20:40PM +0900, Jim Freeze wrote:

How do you feel about pluralism in the other dirs?

bin/ => bins/

38

How do you feel about pluralism in the other dirs?

bin/ => bins/
lib/ => libs/
doc/ => docs/
contrib => contribs/

or are these ok without the ‘s’ suffix since they are abbreviations

It’s mixing apples and orange; keeping needless brevity while trying to
maintain some formal grammatical correctness.

binaries
libraries
documents (or documentation)
contributions

James

39

Hi –

How do you feel about pluralism in the other dirs?

bin/ => bins/
lib/ => libs/
doc/ => docs/
contrib => contribs/

or are these ok without the ‘s’ suffix since they are abbreviations?

Disclaimer: I’m a naysayer on this; I think it’s basically impossible
that there will be consensus and uniformity in practice. So ignore me
:slight_smile: But for what it’s worth, I think if you want even a shot at
consensus, this is probably not the place to introduce an overhaul of
traditional usage. And yes, they’re abbreviations of (in some cases)
plurals already :slight_smile:

David

···

On Wed, 28 May 2003, Jim Freeze wrote:

–
David Alan Black
home: dblack@superlink.net
work: blackdav@shu.edu
Web: http://pirate.shu.edu/~blackdav

40

Saluton!

  • Jim Freeze; 2003-05-28, 11:15 UTC:

How do you feel about pluralism in the other dirs?

bin/ => bins/
lib/ => libs/
doc/ => docs/
contrib => contribs/

or are these ok without the ‘s’ suffix since they are
abbreviations?

One argument for using singular is that this is familiar to many
people:

/usr/bin
/usr/include
/usr/lib
/usr/man

are typical directory names you find on a Unix or Linux system.
Besides that there is the reason already pointed out. The directories
are outnumbered by the files that reside in them. The above results
in files like

/usr/bin/grep
/usr/include/stdio.h
/usr/lib/libruby.so
/usr/man/man1/ruby.1

The binary ‘grep’ that sits in the user tree.
The include file ‘stdio.h’ that sits in the user tree.
The library ‘libruby.so’ that sits in the user tree.
The man page ‘ruby.1’ that sits in the user tree and belongs to
section 1 of the manual.

Use of plural would be more confusing. I am not sure if that has
really been the reason to choose the given directory names. Another
explanation would be that singulars are in most cases shorter and
IIRC old Unix systems had severe restrictions on the length of file
names, directory names and even on the total length of a path.

Gis,

Josef ‘Jupp’ Schugt

···

–
e-mails that do not contain plain text, are larger than 50 KiB, are
unsolicited, or contain binarys are ignored unless payment from your
side or technical reasons give rise to a non-standard treatment.
Schroedinger’s cat is not alive.