Tag: tmp

Write to /tmp directory in aws lambda with python

Goal

I’m trying to write a zip file to the /tmp folder in a python aws lambda, so I can extract manipulate before zipping, and placing it in s3 bucket.

Problem

Os Errno30 Read Only FileSystem

This code was tested locally on my computer to make sure the file would write to my working directory before I uploaded it to aws. This is the code i’m trying to use.

file = downloadFile() #This is api call that returns binary zip object
newFile = open('/tmp/myZip.zip','wb')
newFile.write(file)
extractAll('/tmp/myZip.zip')

here is the code that is trying to extract the zip file

def extractAll(self,source):
        with zipfile.ZipFile(source, 'r') as archive:
            archive.extractall()

here is the trace

[Errno 30] Read-only file system: '/var/task/test-deploy': OSError
Traceback (most recent call last):
File "/var/task/web.py", line 31, in web
performAction(bb, eventBody)
File "/var/task/src/api/web.py", line 42, in performAction
zipHelper.extractAll('/tmp/myZip.zip')
File "/var/task/src/shared/utils/zipfilehelper.py", line 24, in extractAll
archive.extractall()
File "/var/lang/lib/python3.6/zipfile.py", line 1491, in extractall
self.extract(zipinfo, path, pwd)
File "/var/lang/lib/python3.6/zipfile.py", line 1479, in extract
return self._extract_member(member, path, pwd)
File "/var/lang/lib/python3.6/zipfile.py", line 1538, in _extract_member
os.mkdir(targetpath)
OSError: [Errno 30] Read-only file system: '/var/task/test-deploy'

Solution:

extractAll() will extract files in the current directory, which is /var/task/test-deploy in your case.

You need to use os.chdir() to change the current directory:

import os, zipfile

os.chdir('/tmp')
with zipfile.ZipFile(source, 'r') as archive:
    archive.extractall()

Even better, you can create a temporary directory and extract the files there, to avoid polluting /tmp:

import os, tempfile, zipfile

with tempfile.TemporaryDirectory() as tmpdir:
    os.chdir(tmpdir)
    with zipfile.ZipFile(source, 'r') as archive:
        archive.extractall()

If you want to restore the current working directory after extracting the file, consider using this decorator:

import os, tempfile, zipfile, contextlib

@contextlib.context_manager
def temporary_work_dir():
    old_work_dir = os.getcwd()
    with tempfile.TemporaryDirectory() as tmp_dir:
        os.chdir(tmp_dir)
        try:
            yield
        finally:
            os.chdir(old_work_dir)

with temporary_work_dir():
    with zipfile.ZipFile(source, 'r') as archive:
        archive.extractall()

cannot create directory : Permission denied

I had created a directory mkdir xyz with user centos

The following is the output of ll command:
drw——-. 2 centos centos 6 Oct 6 06:11 xyz

When I executed cd xyz I received the following error:
bash: cd: xyz/: Permission denied

I could not access the directory in any case
mkdir xyz/abc
touch xyz/test.txt
All returned the same error.

The solution for this is to add +x permission on the directory.
chmod 700 xyz/
Then I could cd into the directory and create files and folders.

The logic behind this is as belows:

Read bit = If set, you can read this list. So, for example, if you have a directory named poems:

  • You can ls poems and you’ll get a list of items living within (-l won’t reveal any details!).
  • You can use command-line completion i.e. touch poems/so <TAB> poems/somefile.
  • You cannot make poems your working directory (i.e. cd into it).

Write bit = If set, you can modify this list i.e. you can {add,rename,delete} names on it. But! You can actually do it only if the execute bit is set too.

Execute bit = Make this directory your working directory i.e. cd into it. You need this permission if you want to:

  • access (read, write, execute) items living within.
  • modify the list itself i.e. add, rename, delete names on it (of course the write bit must be set on the directory).

Interesting case 1: If you have write + execute permissions on a directory, you can {delete,rename} items living within even if you don’t have write perimission on those items. (use sticky bit to prevent this)

Interesting case 2: If you have execute (but not write) permission on a directory AND you have write permission on a file living within, you cannot delete the file (because it involves removing it from the list). However, you can erase its contents e.g. if it’s a text file you can use vi to open it and delete everything. The file will still be there, but it will be empty.

So the solution is to add +x to the directory.
You will be able to cd into the directory even if you don’t have +x if you are the root user.