Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

g.tempfile: allow creation of temporary directories #4397

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 44 additions & 15 deletions general/g.tempfile/g.tempfile.html
Original file line number Diff line number Diff line change
@@ -1,19 +1,25 @@
<h2>DESCRIPTION</h2>

<em>g.tempfile</em>
is designed for shell scripts that need to use large temporary files.
GRASS provides a mechanism for temporary files that does not depend on
/tmp. GRASS temporary files are created in the data base with the assumption
that there will be enough space under the data base for large files.
creates a unique temporary file or directory (<b>-f</b>) and prints the path.
It is designed for shell scripts that need to use large temporary files.
GRASS provides a mechanism for temporary files or directories that does not
depend on /tmp. GRASS temporary files and directories are created in the
GRASS GIS database with the assumption that there will be enough space for
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can consider talking only about the project. See #3121 Phase 3, bullet point De-emphasize GISDBASE as its own thing from user perspective.

Suggested change
GRASS GIS database with the assumption that there will be enough space for
GRASS GIS project with the assumption that there will be enough space for

You anyway specify only $PROJECT below.

large files. The base directory is: $PROJECT/$MAPSET/.tmp/$HOSTNAME/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if it is clear to the reader what host name is. Elsewhere we talk about node or node name. Hard to say, maybe it's fine as is.


<p>
GRASS periodically removes temporary files that have been left behind
by programs that failed to remove them before terminating.

<p>
<em>g.tempfile</em>
creates an unique file and prints the name. The user is required to provide
a process-id which will be used as part of the name of the file.
wenzeslaus marked this conversation as resolved.
Show resolved Hide resolved
Most Unix shells provide a way to get the process id of the current shell.
For /bin/sh and /bin/csh this is $$.
<em>g.tempfile</em> requires the user to provide an integer number,
usually a process-id (<b>pid</b>), which will be used as part of the file or
directory name of the resulting path, suffixed by a dot and the 0-indexed
number of temporary files with that PID.

Most Unix shells provide a way to get the
process id of the current shell. For /bin/sh and /bin/csh this is $$.
It is recommended that $$ be specified as the process-id for
<em>g.tempfile</em>.

Expand All @@ -23,6 +29,11 @@ <h2>EXAMPLE</h2>
<div class="code"><pre>
temp1=`g.tempfile pid=$$`
temp2=`g.tempfile pid=$$`
# Get the tempile path but do not create it
temp3=`g.tempfile -d pid=$$`
# Create a temporary directory
temp3=`g.tempfile -f pid=$$`

</pre></div>
For /bin/csh scripts, the following can be used:
<div class="code"><pre>
Expand All @@ -36,12 +47,30 @@ <h2>NOTES</h2>
creates a different (i.e. unique) name.

Although GRASS does eventually get around to removing
tempfiles that have been left behind, the programmer should
make every effort to remove these files. They often get
large and take up disk space. If you write /bin/sh scripts,
learn to use the /bin/sh <em>trap</em> command. If you
write /bin/csh scripts, learn to use the /bin/csh
<em>onintr</em> command.
temporary files and directories that have been left behind,
the programmer should make every effort to remove these files.
They often get large and take up disk space. If you write /bin/sh
scripts, learn to use the /bin/sh <em>trap</em> command. If you
write /bin/csh scripts, learn to use the /bin/csh <em>onintr</em>
command. In Python, use an <em>atexit</em> procedure.

<p>
If the size of temporary files is not an issue, it is recommended
to use <i>NamedTemporaryFile</i> with a context manager to create a
temporary file in Python.<br>
Comment on lines +58 to +60
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be clear, you can pass dir to both NamedTemporaryFile and TemporaryDirectory, so you can (could) use the 100% way with the project-based tmp dir if you get the project-mapset-hostname path in some way even without creating subdirectories there.

In this example below, we open a temporary file for writing,
write something and then we can use it in another tool.
Once we do not need it anymore, we need to delete it ourselves.

<div class="code"><pre>
import tempfile

with tempfile.NamedTemporaryFile(mode="w", delete=False) as tmp_file:
file_path = tmp_file.name
tmp_file.write(...)

gs.try_remove(file_path)
</pre></div>

<h2>AUTHOR</h2>

Expand Down
16 changes: 14 additions & 2 deletions general/g.tempfile/main.c
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ int main(int argc, char *argv[])
struct GModule *module;
struct Option *pid;
struct Flag *dry_run;
struct Flag *directory;
char *tempfile;
int p;

Expand All @@ -52,6 +53,13 @@ int main(int argc, char *argv[])
dry_run->description =
_("Dry run - don't create a file, just prints it's file name");

directory = G_define_flag();
directory->key = 'f';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

d is unfortunately used, but f is not great: folder, file, or directory?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I felt the same but had no better idea. I considered -D but that did not seem better than -f for folder mode...
Any suggestions?

directory->description =
_("Folder mode - create a temporary directory, not a file");

G_option_exclusive(dry_run, directory, NULL);

G_disable_interactive();
if (G_parser(argc, argv))
exit(EXIT_FAILURE);
Expand All @@ -63,8 +71,12 @@ int main(int argc, char *argv[])
tempfile = G_tempfile_pid(p);

/* create tempfile so next run of this program will create a unique name */
if (!dry_run->answer)
close(creat(tempfile, 0666));
if (!dry_run->answer) {
if (directory->answer)
G_mkdir(tempfile);
else
close(creat(tempfile, 0666));
}
fprintf(stdout, "%s\n", tempfile);

exit(EXIT_SUCCESS);
Expand Down
10 changes: 6 additions & 4 deletions python/grass/script/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -955,11 +955,13 @@ def tempfile(create=True, env=None):


def tempdir(env=None):
"""Returns the name of a temporary dir, created with g.tempfile."""
tmp = tempfile(create=False, env=env)
os.mkdir(tmp)
"""Returns the name of a temporary directory, created with g.tempfile.

return tmp
:param env: environment

:return: path to a temporary directory
"""
return read_command("g.tempfile", flags="f", pid=os.getpid(), env=env).strip()


def tempname(length, lowercase=False):
Expand Down
Loading