🔧 Python: Generate Temporary Files And Directories Using tempfile

This article was originally published on GeekPython

Python has a rich collection of standard libraries to carry out various tasks. In Python, there is a module called tempfile that allows us to create and manipulate temporary files and directories.

We can use tempfile to create temporary files and directories for storing temporary data during the program execution. The module has functions that allow us to create, read, and manipulate temporary files and directories.

In this article, we'll go over the fundamentals of the tempfile module, including creating, reading, and manipulating temporary files and directories, as well as cleaning up after we're done with them.

Creating temporary file

The tempfile module includes a function called TemporaryFile() that allows us to create a temporary file for use as temporary storage. Simply enter the following code to create a temporary file.

import tempfile

# Creating a temporary file
temp_file = tempfile.TemporaryFile()

print(f'Tempfile object: {temp_file}')
print(f'Tempfile name: {temp_file.name}')

# Closing the file 
temp_file.close()

First, we imported the module, and because it is part of the standard Python library, we did not need to install it.

Then we used the TemporaryFile() function to create a temporary file, which we saved in the temp_file variable. Then we added print statements to display the tempfile object and temporary file name.

Finally, we used the close() function to close the file, just as we would any other file. This will automatically clean up and delete the file.

Output

Tempfile object: <tempfile._TemporaryFileWrapper object at 0x0000015E54545AE0>
Tempfile name: C:\Users\SACHIN\AppData\Local\Temp\tmp5jyjavv2

We got the location of the object in memory when we printed the object temp_file, and we got the random name tmp5jyjavv2 with the entire path when we printed the name of the temporary file.

We can specify dir parameter to create a temporary file in the specified directory.

import tempfile

# Creating a temporary file in the cwd
named_file = tempfile.TemporaryFile(
    dir='./'
)

print('File created in the cwd')
print(named_file.name)

# Closing the file
named_file.close()

The file will be created in the current working directory.

File created in the cwd
D:\SACHIN\Pycharm\tempfile_module\tmptbg9u6wk

Writing text and reading it

The TemporaryFile() function sets the mode parameter 'w+b' to the default value to read and write files in a binary mode. We can also use 'w+t' mode to write text data into the temporary file.

import tempfile

# Creating a temporary file
file = tempfile.TemporaryFile()

try:
    # Writing into the temporary file
    file.write(b"Welcome to GeekPython.")
    # Position to start reading the file
    file.seek(0)  # Reading will start from beginning
    # Reading the file
    data = file.read()
    print(data)
finally:
    # Closing the file
    file.close()

----------
b'Welcome to GeekPython.'

In the above code, we created a temporary file and then used the write() function to write data into it, passing the string in bytes format (as indicated by the prefix 'b' before the content).

Then we used the read() function to read the content, but first, we specified the position from which to begin reading the content using the seek(0) (start from the beginning) function, and finally, we closed the file.

Using context manager

We can also create temporary files using the context managers such as with keyword. Also, the following example will show us how to write and read text data into the temporary file.

import tempfile

# Using with statement creating a temporary file
with tempfile.TemporaryFile(mode='w+t') as file:
    # Writing text data into the file
    file.write('Hello Geeks!')
    # Specifying the position to start reading
    file.seek(0)
    # Reading the content
    print(file.read())
    # Closing the file
    file.close()

In the above code, we used the with statement to create a temporary file and then opened it in text mode('w+t') and did everything the same as we do for handling other files in Python.

Hello Geeks!

Named temporary file

To take more control over making the temporary file like naming the temporary file as we want or keeping it or deleting it, then we can use NamedTemporaryFile().

When creating a temporary file, we can specify a suffix and a prefix, and we can choose whether to delete or keep the temporary file. It has a delete parameter that defaults to True, which means that the file will be deleted automatically when we close it, but we can change this value to False to keep it.

import tempfile

# Creating a temporary file using NamedTemporaryFile()
named_file = tempfile.NamedTemporaryFile()

print('Named file:', named_file)
print('Named file name:', named_file.name)

# Closing the file
named_file.close()

----------
Named file: <tempfile._TemporaryFileWrapper object at 0x000001E70CDEC310>
Named file name: C:\Users\SACHIN\AppData\Local\Temp\tmpk6ycz_ek

The output of the above code is the same as when we created the temporary file using the TemporaryFile() function, it's because NamedTemporaryFile() operates exactly as TemporaryFile() but the temporary file created using the NamedTemporaryFile() is guaranteed to have a visible name in the file system.

How to customize the name of a temporary file?

As you can see, the name of our temporary file is generated at random, but we can change that with the help of the prefix and suffix parameters.

import tempfile

# Creating a temporary file using NamedTemporaryFile()
named_file = tempfile.NamedTemporaryFile(
    prefix='geek-',
    suffix='-python'
)

print('Named file name:', named_file.name)

# Closing the file
named_file.close()

The temporary file will now be created when the code is run, and its name will be prefixed and suffixed with the values we specified in the code above.

Named file name: C:\Users\SACHIN\AppData\Local\Temp\geek-n6luwuwx-python

Creating temporary directory

To create a temporary directory, we can use TemporaryDirectory() function from the tempfile module.

import tempfile

# Creating a temporary directory inside the TempDir
tempdir = tempfile.TemporaryDirectory(
    dir='TempDir'
)

# Printing the name of temporary dir
print('Temp Directory:', tempdir.name)

# Cleaning up the dir
tempdir.cleanup()

The above code will create a temporary directory inside the TempDir directory that we created in the root directory. Then we printed the name of the temporary directory and then called the cleanup() method to explicitly clean the temporary directory.

Temp Directory: TempDir\tmplfu2ooew

We can use with statement to create a temporary directory and we can also specify suffix and prefix parameters.

with tempfile.TemporaryDirectory(
        dir='TempDir',
        prefix='hey-',
        suffix='-there'
) as tempdir:
    print("Temp Dir:", tempdir)

When we create a temporary directory using the with statement the name of the directory will be assigned to the target of the as clause. To get the name of the temporary directory, we passed tempdir rather than tempdir.name in the print statement.

Temp Dir: TempDir\hey-q14i1hc4-there

You will notice that this time we didn't use the cleanup() method, that's because the temporary directory and its content are removed after the completion of the context of the temporary directory object.

SpooledTemporaryFile

The SpooledTemporaryFile() function is identical to TemporaryFile(), with the exception that it stores the data in memory up until the maximum size is reached or fileno() method is called.

import tempfile

# Creating a temporary file to spool data into it
sp_file = tempfile.SpooledTemporaryFile(max_size=10)
print(sp_file)

# Writing into the file
sp_file.write(b'Hello from GeekPython')

# Printing the size of the file
print(sp_file.__sizeof__())

# Printing if data is written to the disk
print(sp_file._rolled)
print(sp_file._file)

With the max_size parameter set to 10, we used the SpooledTemporaryFile() function to create a spooled temporary file. It indicates that file size up to 10 can be stored in the memory before it is rolled over to the on-disk file.

Then we wrote some data into the file and printed the size of the file using the __sizeof__ attribute.

There is a rollover() method by which we can see whether the data is in the memory or rolled over to the on-disk temporary file. This method returns a boolean value, True means the data is rolled over and False means the data is still in the memory of the spooled temporary file.

<tempfile.SpooledTemporaryFile object at 0x0000023FFBD0C370>
32
True
<tempfile._TemporaryFileWrapper object at 0x0000023FFBD0D930>

The temporary file object is SpooledTemporaryFile, as can be seen. Because the amount of data we wrote into the file exceeded the allowed size, the data was rolled over, and we received the boolean value True.

Depending on the mode we specified, this function returns a file-like object whose _file attribute is either an io.BytesIO or an io.TextIOWrapper (binary or text). Up until the data exceeded the max size, this function holds the data in memory using the io.BytesIO or io.TextIOWrapper buffer.

import tempfile

with tempfile.SpooledTemporaryFile(mode="w+t", max_size=50) as sp_file:
    print(sp_file)

    # Running a while loop until the data gets rolled over
    while sp_file._rolled == False:
        sp_file.write("Welcome to GeekPython")
        print(sp_file._rolled, sp_file._file)

Output

<tempfile.SpooledTemporaryFile object at 0x000001A8901CC370>
False <_io.TextIOWrapper encoding='cp1252'>
False <_io.TextIOWrapper encoding='cp1252'>
True <tempfile._TemporaryFileWrapper object at 0x000001A8906388E0>

Until the data is rolled over, we ran a while loop and printed the _file attribute of the SpooledTemporaryFile object. As we can see the data was stored in the memory using the io.TextIOWrapper buffer(mode was set to text) and when the data was rolled over, the code returned the temporary file object.

We can do the same with data in binary format.

import tempfile

with tempfile.SpooledTemporaryFile(mode="w+b", max_size=50) as sp_file:
    print(sp_file)

    # Running a while loop until the data gets rolled over
    while sp_file._rolled == False:
        sp_file.write(b"Welcome to GeekPython")
        print(sp_file._rolled, sp_file._file)

Output

<tempfile.SpooledTemporaryFile object at 0x00000223C633C370>
False <_io.BytesIO object at 0x00000223C6331580>
False <_io.BytesIO object at 0x00000223C6331580>
True <tempfile._TemporaryFileWrapper object at 0x00000223C67A8850>

We can also call fileno() to roll over the file content.

import tempfile

with tempfile.SpooledTemporaryFile(mode="w+b", max_size=500) as sp_file:
    print(sp_file)

    for _ in range(3):
        sp_file.write(b'Hey, there welcome')
        print(sp_file._rolled, sp_file._file)

    print('Data is written to the disk before reaching the max size.')
    # Calling the fileno() method
    sp_file.fileno()
    print(sp_file._rolled, sp_file._file)

Output

<tempfile.SpooledTemporaryFile object at 0x000001F51D12C370>
False <_io.BytesIO object at 0x000001F51D121580>
False <_io.BytesIO object at 0x000001F51D121580>
False <_io.BytesIO object at 0x000001F51D121580>
Data is written to the disk before reaching the max size.
True <tempfile._TemporaryFileWrapper object at 0x000001F51D5A0970>

Low-level functions

There are some low-level functions included in the tempfile module. We'll explore them one by one.

mkstemp and mkdtemp

mkstemp is used for creating a temporary file in the most secure manner possible and mkdtemp is used to create a temporary directory in the most secure manner possible.

They also have parameters like suffix, prefix and dir but mkstemp has one additional text parameter which defaults to False(binary mode), if we make it True then the file will be opened in text mode.

import tempfile

# Using mkstemp
file = tempfile.mkstemp(prefix='hello-', suffix='-world')
print('Created File:', file)

# Using mkdtemp
directory = tempfile.mkdtemp(dir='TempDir')
print('Directory Created:', directory)

----------
Created File: (3, 'C:\\Users\\SACHIN\\AppData\\Local\\Temp\\hello-xmtqn88f-world')
Directory Created: TempDir\tmp_6coyqq2

Note: There is a function called mktemp() also which is deprecated.

gettempdir and gettempdirb

gettempdir() is used to return the name of the directory used for temporary files and gettempdirb() also return the name of the directory but in bytes.

import tempfile

print(tempfile.gettempdir())
print(tempfile.gettempdirb())

----------
C:\Users\SACHIN\AppData\Local\Temp
b'C:\\Users\\SACHIN\\AppData\\Local\\Temp'

Conclusion

The tempfile module, which offers functions to create temporary files and directories, was covered in this article. We have used various functions from the tempfile module in code examples, which has helped us better understand how these functions operate.

🏆Other articles you might be interested in if you liked this one

✅Use match case statement for pattern matching in Python.

✅Types of class inheritance in Python with examples.

✅An improved and modern way of string formatting in Python.

✅Integrate PostgreSQL database with Python.

✅Argmax function in NumPy and TensorFlow - Are they the same?

✅Take multiple inputs from the user in a single line in Python.

✅Perform a parallel iteration over multiple iterables using zip() function in Python.

That's all for now

Keep Coding✌✌