A gem for handling temporary file(s)?

ruby-talk
1

I'm writing a program that needs to generate two or three temporary
files.

(Specifically: my program runs a shell command and I need to pass the
shell command a path to a non-existing file which it will dump data to.)

Is there a 'gem' that manages these things? Preferably it should remove
the files when the script finishes or whatever.

Of course, if Ruby supports this built-in that's fine too (I did 'ri
File' and 'ri FileUtils' but founds nothing).

···

--
Posted via http://www.ruby-forum.com/.

2

Albert Schlef wrote:

I'm writing a program that needs to generate two or three temporary
files.

(Specifically: my program runs a shell command and I need to pass the
shell command a path to a non-existing file which it will dump data to.)

Is there a 'gem' that manages these things? Preferably it should remove
the files when the script finishes or whatever.

Of course, if Ruby supports this built-in that's fine too (I did 'ri
File' and 'ri FileUtils' but founds nothing).

ri Tempfile

that'll get you started

···

--
Posted via http://www.ruby-forum.com/\.

3

I'm writing a program that needs to generate two or three temporary
files.

(Specifically: my program runs a shell command and I need to pass the
shell command a path to a non-existing file which it will dump data to.)

man mktemp
man tempfile

Is there a 'gem' that manages these things? Preferably it should remove
the files when the script finishes or whatever.

it is builtin in ruby. but in this case, you'd better do it all in ruby...

best regards -botp

···

On Wed, Mar 3, 2010 at 10:19 AM, Albert Schlef <albertschlef@gmail.com> wrote:

4

Paul Harrington wrote:

Albert Schlef wrote:

I'm writing a program that needs to generate two or three temporary
files.

(Specifically: my program runs a shell command and I need to pass the
shell command a path to a non-existing file which it will dump data to.)

[...]

ri Tempfile

that'll get you started

Thanks! I didn't know about Tempfile.

Though I have a little problem: Tempfile let me *open* a new temporary
file. But I just need to generate a temporary file *name*, which I'll
pass to a shell command.

I guess I'll immediately close the handle Tempfile.new() returns and
pass its path() to the shell command.

···

--
Posted via http://www.ruby-forum.com/\.

5

Bah. Use file-temp.

gem install file-temp

Regards,

Dan

···

On Mar 3, 12:28 am, botp <botp...@gmail.com> wrote:

On Wed, Mar 3, 2010 at 10:19 AM, Albert Schlef <albertsch...@gmail.com> wrote:
> I'm writing a program that needs to generate two or three temporary
> files.

> (Specifically: my program runs a shell command and I need to pass the
> shell command a path to a non-existing file which it will dump data to.)

man mktemp
man tempfile

> Is there a 'gem' that manages these things? Preferably it should remove
> the files when the script finishes or whatever.

it is builtin in ruby. but in this case, you'd better do it all in ruby...

6

Paul Harrington wrote:

Albert Schlef wrote:

I'm writing a program that needs to generate two or three temporary
files.

(Specifically: my program runs a shell command and I need to pass the
shell command a path to a non-existing file which it will dump data to.)

[...]

ri Tempfile

that'll get you started

Thanks! I didn't know about Tempfile.

Though I have a little problem: Tempfile let me *open* a new temporary file. But I just need to generate a temporary file *name*, which I'll pass to a shell command.

You could use a dirty hack and abuse a private method:

irb(main):005:0> Tempfile.open('/tmp') {|tf| p tf, tf.send(:make_tmpname,'a','o')}
#<File:/tmp/tmp20100303-4173-rdy9aq-0>
"a20100303-4173-kvk0d3-o"
=> [#<File:/tmp/tmp20100303-4173-rdy9aq-0 (closed)>, "a20100303-4173-kvk0d3-o"]
irb(main):006:0>

I guess I'll immediately close the handle Tempfile.new() returns and pass its path() to the shell command.

What do you want the external program to do with the tempfile?

Kind regards

  robert

···

On 03/03/2010 05:07 AM, Albert Schlef wrote:

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

7

I'm not expert enough to be certain about this, but by doing this
you'll be creating a tempfile race condition security hole in your
program. I think the same goes for Robert's suggestion as well. There
may be a way to do it securely... but it's probably tricky. One
advantage of Tempfile (and similar facilities in other languages) is
that it avoids this subtle and nassty security hole. But you have to
use it the way it wants to be used, otherwise you defeat the security.
This is why you're better off rewriting this external command in ruby,
if that's possible. Or rewriting your ruby script to make it an
integral part of the external program.

None of this may actually matter in your case... but you're the only
one with enough information to make that judgment.

···

On 3/2/10, Albert Schlef <albertschlef@gmail.com> wrote:

Paul Harrington wrote:

ri Tempfile

that'll get you started

Thanks! I didn't know about Tempfile.

Though I have a little problem: Tempfile let me *open* a new temporary
file. But I just need to generate a temporary file *name*, which I'll
pass to a shell command.

8

I can't get file-temp to install at the moment, (for reasons unrelated
to file-temp, apparently). Does file-temp allow one to create
temporary _directories_? That's a feature I've often missed in the
past.

···

On 3/3/10, Daniel Berger <djberg96@gmail.com> wrote:

Bah. Use file-temp.

gem install file-temp

9

Robert Klemme wrote:

You could use a dirty hack and abuse a private method:
tf.send(:make_tmpname,'a','o')}

Thanks.

Anyway, it turned out that was my smallest problems. I ended up writing
a wrapper class that remembers a set of related Tempfile object (or else
the files get deleted too soon for me).

···

--
Posted via http://www.ruby-forum.com/\.

10

Paul Harrington wrote:

ri Tempfile

that'll get you started

Thanks! I didn't know about Tempfile.

Though I have a little problem: Tempfile let me *open* a new temporary
file. But I just need to generate a temporary file *name*, which I'll
pass to a shell command.

I'm not expert enough to be certain about this, but by doing this
you'll be creating a tempfile race condition security hole in your
program. I think the same goes for Robert's suggestion as well.

Do you mean there is a robustness issue or a security issue? I don't see a security issue here. Robustness would only be at risk if the file name generation algorithm is bad. What else am I missing?

There
may be a way to do it securely... but it's probably tricky. One
advantage of Tempfile (and similar facilities in other languages) is
that it avoids this subtle and nassty security hole. But you have to
use it the way it wants to be used, otherwise you defeat the security.
This is why you're better off rewriting this external command in ruby,
if that's possible. Or rewriting your ruby script to make it an
integral part of the external program.

None of this may actually matter in your case... but you're the only
one with enough information to make that judgment.

Albert still did not disclose what the external program should do with the temporary file. We do not even know whether it is an option to rewrite the external program.

Kind regards

  robert

···

On 03/03/2010 08:35 PM, Caleb Clausen wrote:

On 3/2/10, Albert Schlef <albertschlef@gmail.com> wrote:

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

11

I _think_ that if you create a temporary file with Tempfile, and then
DON'T CLOSE IT, you can safely pass the tempfile's name to an external
command. (I'm assuming that the tempfile is an output from the
external command.... if it's an input, none of my security anxieties
apply. I think.) Only close the tempfile once the external command has
finished (and you've read out of it whatever information you need).
This will fail if the external command balks at writing to an already
existing file.

Alternatively, you could put your temp file in $HOME/tmp rather than
the system-wide /tmp, which is another way to sidestep the race. I'm
pretty sure. If you go this way, Tempfile is useless to you, tho.

···

On 3/3/10, Caleb Clausen <vikkous@gmail.com> wrote:

There
may be a way to do it securely... but it's probably tricky.

12

No, but there's Dir.mktmpdir.

Regards,

Dan

···

On Mar 3, 2:13 pm, Caleb Clausen <vikk...@gmail.com> wrote:

On 3/3/10, Daniel Berger <djber...@gmail.com> wrote:

> Bah. Use file-temp.

> gem install file-temp

I can't get file-temp to install at the moment, (for reasons unrelated
to file-temp, apparently). Does file-temp allow one to create
temporary _directories_? That's a feature I've often missed in the
past.

13

I'm not expert enough to be certain about this, but by doing this
you'll be creating a tempfile race condition security hole in your
program. I think the same goes for Robert's suggestion as well.

Do you mean there is a robustness issue or a security issue? I don't
see a security issue here. Robustness would only be at risk if the file
name generation algorithm is bad. What else am I missing?

It's a security issue. See:
http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/avoid-race.html
and scroll down to "7.10.1.2. Temporary Files"

As I implied above, I may not know what I'm talking about here, but
I'm fairly sure. Also, I didn't contemplate the snippet you
contributed closely; perhaps it avoids the race condition in some
clever way that I'm unaware of.

There
may be a way to do it securely... but it's probably tricky. One
advantage of Tempfile (and similar facilities in other languages) is
that it avoids this subtle and nassty security hole. But you have to
use it the way it wants to be used, otherwise you defeat the security.
This is why you're better off rewriting this external command in ruby,
if that's possible. Or rewriting your ruby script to make it an
integral part of the external program.

None of this may actually matter in your case... but you're the only
one with enough information to make that judgment.

Albert still did not disclose what the external program should do with
the temporary file. We do not even know whether it is an option to
rewrite the external program.

Yes, indeed. Which is why I said "if that's possible".

···

On 3/3/10, Robert Klemme <shortcutter@googlemail.com> wrote:

On 03/03/2010 08:35 PM, Caleb Clausen wrote:

14

Another way is Dir.mktmpdir in tmpdir.

require 'tmpdir'
Dir.mktmpdir {|d|
  tmppath = File.join(d, "foo")
  ... use tmppath ...
}

···

2010/3/4 Caleb Clausen <vikkous@gmail.com>:

Alternatively, you could put your temp file in $HOME/tmp rather than
the system-wide /tmp, which is another way to sidestep the race. I'm
pretty sure. If you go this way, Tempfile is useless to you, tho.

--
Tanaka Akira

15

Robert Klemme wrote:

one with enough information to make that judgment.

Albert still did not disclose what the external program should do with
the temporary file [...]

Thank you all, this task is behind me already.

If it interests you, then the external command I'm using is 'dot' (from
the graphviz package).

The command I'm using is something like this:

  dot "%s" -Tgif -o "%s" -Tsvg -o "%s"

(I need three temporary files, in this example. 'dot' reads the first
file and writes images to the next two files.)

Smart guys would say that instead, on POSIX systems, I could write the
command as...

  dot /dev/fd/10 -Tgif -o /dev/fd/11" -Tsvg -o /dev/fd/12

...but then I'd need to read simultaneously from fd/11 and fd/12 because
I'm not sure which is written first. But I need my program to work on
Windows too and I'm not sure it supports this "sophistication".

Caleb Clausen wrote:

I'm not expert enough to be certain about this, but by doing this
you'll be creating a tempfile race condition [...]

I don't care much about robustness because the command is run from an
interactive application by a single user and I'm using the files
immediately. There isn't much a chance for a race condition.

Caleb Clausen wrote:

I _think_ that if you create a temporary file with Tempfile, and then
DON'T CLOSE IT, you can safely pass the tempfile's name to an external
command.

For some reason, the files aren't always deleted (well, it seems they're
never deleted). I don't know why. At first I suspected 'dot' re-creates
the files, but when I do "ls -i" I see that the inodes haven't changed.
But I can live with this bug. Maybe I could add some code to the
"destructor" of my class to explicitly delete these files. Does Ruby
support "destructors"?

···

On 03/03/2010 08:35 PM, Caleb Clausen wrote:

--
Posted via http://www.ruby-forum.com/\.

16

Thanks to you and to Tanaka Akira for pointing this out, since I was
not aware of it. I'll definitely remember it next time I need that.

···

On 3/4/10, Daniel Berger <djberg96@gmail.com> wrote:

On Mar 3, 2:13 pm, Caleb Clausen <vikk...@gmail.com> wrote:

to file-temp, apparently). Does file-temp allow one to create
temporary _directories_? That's a feature I've often missed in the
past.

No, but there's Dir.mktmpdir.

17

Robert Klemme wrote:

one with enough information to make that judgment.

Albert still did not disclose what the external program should do with
the temporary file [...]

Thank you all, this task is behind me already.

Considering your last comment about files not being deleted I'd say
it's not done yet. :slight_smile:

If it interests you, then the external command I'm using is 'dot' (from
the graphviz package).

The command I'm using is something like this:

dot "%s" -Tgif -o "%s" -Tsvg -o "%s"

(I need three temporary files, in this example. 'dot' reads the first
file and writes images to the next two files.)

Smart guys would say that instead, on POSIX systems, I could write the
command as...

dot /dev/fd/10 -Tgif -o /dev/fd/11" -Tsvg -o /dev/fd/12

...but then I'd need to read simultaneously from fd/11 and fd/12 because
I'm not sure which is written first. But I need my program to work on
Windows too and I'm not sure it supports this "sophistication".

Yeah, probably not a good idea to go into that direction.

Btw, do you need to read those files while they are written or is it
sufficient to wait for process termination (e.e. use system) before
you access them?

Caleb Clausen wrote:

I'm not expert enough to be certain about this, but by doing this
you'll be creating a tempfile race condition [...]

I don't care much about robustness because the command is run from an
interactive application by a single user and I'm using the files
immediately. There isn't much a chance for a race condition.

Caleb Clausen wrote:

I _think_ that if you create a temporary file with Tempfile, and then
DON'T CLOSE IT, you can safely pass the tempfile's name to an external
command.

For some reason, the files aren't always deleted (well, it seems they're
never deleted). I don't know why. At first I suspected 'dot' re-creates
the files, but when I do "ls -i" I see that the inodes haven't changed.
But I can live with this bug. Maybe I could add some code to the
"destructor" of my class to explicitly delete these files. Does Ruby
support "destructors"?

We have

begin
...
ensure
...
end

The usual way to do this is to write a method. See
http://blog.rubybestpractices.com/posts/rklemme/002_Writing_Block_Methods.html

Cheers

robert

···

2010/3/4 Albert Schlef <albertschlef@gmail.com>:

On 03/03/2010 08:35 PM, Caleb Clausen wrote:

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

18

It's not (just) a robustness issue, but a security issue. Even if
there's not much chance for a race condition, that can create a very
large security hole. I'd suggest you should take a look at the link I
posted in reply to Robert above. I just want to make sure you're
aware.

···

On 3/4/10, Albert Schlef <albertschlef@gmail.com> wrote:

Caleb Clausen wrote:

I'm not expert enough to be certain about this, but by doing this
you'll be creating a tempfile race condition [...]

I don't care much about robustness because the command is run from an
interactive application by a single user and I'm using the files
immediately. There isn't much a chance for a race condition.

19

I'm not expert enough to be certain about this, but by doing this
you'll be creating a tempfile race condition security hole in your
program. I think the same goes for Robert's suggestion as well.

Do you mean there is a robustness issue or a security issue? I don't
see a security issue here. Robustness would only be at risk if the file
name generation algorithm is bad. What else am I missing?

It's a security issue. See:
http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/avoid-race.html
and scroll down to "7.10.1.2. Temporary Files"

As I implied above, I may not know what I'm talking about here, but
I'm fairly sure. Also, I didn't contemplate the snippet you
contributed closely; perhaps it avoids the race condition in some
clever way that I'm unaware of.

Thanks for the link. If I understand all this right this cannot be
fixed as long as a) there are two processes involved or b) the second
process (dot) cannot be made inherit the file descriptor. An
alternative approach would avoid a shared directory like /tmp and
write the output in $HOME/.../somewhere - which might be slower
because /tmp is often mounted in RAM. Maybe it helps to create a
directory in /tmp which is owned by and only accessible to $USER; then
create the tempfiles in that directory.

There
may be a way to do it securely... but it's probably tricky. One
advantage of Tempfile (and similar facilities in other languages) is
that it avoids this subtle and nassty security hole. But you have to
use it the way it wants to be used, otherwise you defeat the security.
This is why you're better off rewriting this external command in ruby,
if that's possible. Or rewriting your ruby script to make it an
integral part of the external program.

None of this may actually matter in your case... but you're the only
one with enough information to make that judgment.

Albert still did not disclose what the external program should do with
the temporary file. We do not even know whether it is an option to
rewrite the external program.

Yes, indeed. Which is why I said "if that's possible".

Yep. I mainly included the remark to poke Albert so he would
eventually give the information. :wink:

Kind regards

robert

···

2010/3/3 Caleb Clausen <vikkous@gmail.com>:

On 3/3/10, Robert Klemme <shortcutter@googlemail.com> wrote:

On 03/03/2010 08:35 PM, Caleb Clausen wrote:

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

20

Robert Klemme wrote:

dot ... -Tgif -o ... -Tsvg -o ...

[...]

Btw, do you need to read those files while they are written or is it
sufficient to wait for process termination (e.e. use system) before
you access them?

(I read the files after termination (I'm using system()).)

Maybe I could add some code to the
"destructor" of my class to explicitly delete these files. Does Ruby
support "destructors"?

We have

begin
...
ensure
...
end

The usual way to do this is to write a method. See
http://blog.rubybestpractices.com/posts/rklemme/002_Writing_Block_Methods.html

Thanks. I know this scheme (and the article), but for some reason it
didn't come up to my mind to use it. You just gave me a nice idea. I
need to read more Ruby code to train myself. The problem is that I love
to program so I prefer writing to reading :frowning:

Caleb Clausen wrote:

It's not (just) a robustness issue, but a security issue. Even if
there's not much chance for a race condition, that can create a very
large security hole. I'd suggest you should take a look at the link I
posted in reply to Robert above. I just want to make sure you're
aware.

You mean a privacy issue. Yes, I fixed that. Thanks.

···

2010/3/4 Albert Schlef <albertschlef@gmail.com>:

--
Posted via http://www.ruby-forum.com/\.