Unix permissions

Every day; I deal with permissions. Permissions come in all formats; anything from a manual authorization request to complex network ACL, to mutual authentication at the SSL layer.

Today though; we're going over POSIX permissions and how they apply to various situations.

At work, we enforce a separation between the NGINX/PHP-FPM user and the general user developers use to connect over SSH. It's far from a perfect model; and it adds complications like the one I'm going to talk about; but it's a much better approach than running the same user or, god forbid, the old

chmod 777 /var/www/html -R

...shudders

How numbers work

Basic POSIX permissions are broken down into three blocks of three grants:

  • User (aka "u")
  • Group (aka "g")
  • Other (aka "o")

And three basic permissions:

  • Read (aka "r")
  • Write (aka "w")
  • eXecute (aka "x")

You'll commonly see file permissions written out in full; here's some examples:

[[email protected] ausernamedjim]# ls -lah elasticsearch-5.3.0.tar.gz
-rw-r--r-- 1 root root 33M Apr  4  2017 elasticsearch-5.3.0.tar.gz
[[email protected] ausernamedjim]# getfacl elasticsearch-5.3.0.tar.gz
# file: elasticsearch-5.3.0.tar.gz
# owner: root
# group: root
user::rw-
group::r--
other::r--

This should be a pretty simple to follow. This starts to get complicated to people when we start introducing umask values. (which we'll get to in a few minutes!)

Linux distributions specify permissions in octal. Mostly for historical reasons, octal is typed with a leading zero. (Other modern implementations typically prefix strings with the lowercase letter "o").

There are three object types, and three sets of permissions, this matrix can be represented in Octal by mapping each permission to each object type. Because base 8 (comprised of the digits 0-7) can be simply translated to base 2 (binary) we'll do this in binary so it makes more sense.

Screen-Shot-2017-12-08-at-10.58.02-PM

  • The binary string 0100 translates to the octal (or decimal) number 4.
  • The binary string 0010 translates to the number 2.
  • The binary string 0001 translates to the number 1.

Permissions are made up of hierarchical bits; so the most-significant-bit should always be the first set. These numbers can then be put together (added) to simplify the number down to a single base 8 digit:

  • 4 -> Read permission only
  • 5 -> Read and eXecute permission
  • 6 -> Read and Write permission
  • 7 -> Read, Write, and eXecute permission

and finally are bound together into the three objects; to yield the common permission scheme you'll typically find in various guides online:

  • 0750 -> Read+Write+eXecute for Owner, Read+eXecute for Group, and no permissions for Other
  • 666 -> Read + Write for all users (User, Group, Other)
  • 444 -> Readable by anyone; only writable by root

Application

Here's where things start getting interesting.

Permissions are great; they define what you can; and can't read; write; or execute. On Linux everything is a file, this means that using a fairly simple permissions scheme a systems administrator can control access to anything from text files to control sockets to hardware devices.

The first two raw permission values are pretty easy to understand based on their name.

  • The Read permission let's applications.. read the file. This let's you open the file; read it's contents, copy the information in it, etc.
  • The Write permission let's you write to the file. You can write contents to the file; append; delete the file, etc.

The third permission is a little more complex to explain; I'll let some wonderful person from Wikipedia explain:

The execute permission grants the ability to execute a file. This permission must be set for executable programs, including shell scripts, in order to allow the operating system to run them. When set for a directory, the execute permission is interpreted as the search permission: it grants the ability to access file contents and meta-information if its name is known, but not list files inside the directory, unless read is set also.

Using this information effectively

When you create a file on a Linux machine via pretty well any method imaginable, you actually create a structure located at an inode containing meta-data about the file. The inode creation process allocates the pointer(s) that reference the blocks-on-disk that will store the actual bytes you intend to write to the filesystem.

Let's take a look at an example here; (don't worry about the length; we'll go over the important parts!):

[[email protected] ausernamedjim tmp]# strace -t bash -c echo 'testing' > /tmp/a-test-file.tmp
06:21:51 execve("/bin/bash", ["bash", "-c", "echo", "testing"], [/* 30 vars */]) = 0
06:21:51 brk(0)                         = 0xf86000
06:21:51 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd480e97000
06:21:51 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
06:21:51 open("/etc/ld.so.cache", O_RDONLY) = 3
06:21:51 fstat(3, {st_mode=S_IFREG|0644, st_size=40578, ...}) = 0
06:21:51 mmap(NULL, 40578, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fd480e8d000
06:21:51 close(3)                       = 0
06:21:51 open("/lib64/libtinfo.so.5", O_RDONLY) = 3
06:21:51 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\[email protected]\310\0\0\0\0\0\0"..., 832) = 832
06:21:51 fstat(3, {st_mode=S_IFREG|0755, st_size=132408, ...}) = 0
06:21:51 mmap(NULL, 2228832, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fd480a58000
06:21:51 mprotect(0x7fd480a75000, 2093056, PROT_NONE) = 0
06:21:51 mmap(0x7fd480c74000, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c000) = 0x7fd480c74000
06:21:51 mmap(0x7fd480c78000, 608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fd480c78000
06:21:51 close(3)                       = 0
06:21:51 open("/lib64/libdl.so.2", O_RDONLY) = 3
06:21:51 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\r\0\0\0\0\0\0"..., 832) = 832
06:21:51 fstat(3, {st_mode=S_IFREG|0755, st_size=19536, ...}) = 0
06:21:51 mmap(NULL, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fd480854000
06:21:51 mprotect(0x7fd480856000, 2097152, PROT_NONE) = 0
06:21:51 mmap(0x7fd480a56000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7fd480a56000
06:21:51 close(3)                       = 0
06:21:51 open("/lib64/libc.so.6", O_RDONLY) = 3
06:21:51 read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0000\356\1\0\0\0\0\0"..., 832) = 832
06:21:51 fstat(3, {st_mode=S_IFREG|0755, st_size=1923352, ...}) = 0
06:21:51 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd480e8c000
06:21:51 mmap(NULL, 3750184, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fd4804c0000
06:21:51 mprotect(0x7fd48064a000, 2097152, PROT_NONE) = 0
06:21:51 mmap(0x7fd48084a000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x18a000) = 0x7fd48084a000
06:21:51 mmap(0x7fd480850000, 14632, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fd480850000
06:21:51 close(3)                       = 0
06:21:51 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd480e8b000
06:21:51 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd480e8a000
06:21:51 arch_prctl(ARCH_SET_FS, 0x7fd480e8b700) = 0
06:21:51 mprotect(0x7fd48084a000, 16384, PROT_READ) = 0
06:21:51 mprotect(0x7fd480a56000, 4096, PROT_READ) = 0
06:21:51 mprotect(0x7fd480e98000, 4096, PROT_READ) = 0
06:21:51 munmap(0x7fd480e8d000, 40578)  = 0
06:21:51 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
06:21:51 open("/dev/tty", O_RDWR|O_NONBLOCK) = 3
06:21:51 close(3)                       = 0
06:21:51 brk(0)                         = 0xf86000
06:21:51 brk(0xfa7000)                  = 0xfa7000
06:21:51 open("/usr/lib/locale/locale-archive", O_RDONLY) = 3
06:21:51 fstat(3, {st_mode=S_IFREG|0644, st_size=99164480, ...}) = 0
06:21:51 mmap(NULL, 99164480, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fd47a62d000
06:21:51 close(3)                       = 0
06:21:51 getuid()                       = 0
06:21:51 getgid()                       = 0
06:21:51 geteuid()                      = 0
06:21:51 getegid()                      = 0
06:21:51 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
06:21:51 open("/proc/meminfo", O_RDONLY|O_CLOEXEC) = 3
06:21:51 fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
06:21:51 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd480e96000
06:21:51 read(3, "MemTotal:       32737480 kB\nMemF"..., 1024) = 1024
06:21:51 close(3)                       = 0
06:21:51 munmap(0x7fd480e96000, 4096)   = 0
06:21:51 rt_sigaction(SIGCHLD, {SIG_DFL, [], SA_RESTORER|SA_RESTART, 0x7fd4804f2660}, {SIG_DFL, [], 0}, 8) = 0
06:21:51 rt_sigaction(SIGCHLD, {SIG_DFL, [], SA_RESTORER|SA_RESTART, 0x7fd4804f2660}, {SIG_DFL, [], SA_RESTORER|SA_RESTART, 0x7fd4804f2660}, 8) = 0
06:21:51 rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7fd4804f2660}, {SIG_DFL, [], 0}, 8) = 0
06:21:51 rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7fd4804f2660}, {SIG_DFL, [], SA_RESTORER, 0x7fd4804f2660}, 8) = 0
06:21:51 rt_sigaction(SIGQUIT, {SIG_DFL, [], SA_RESTORER, 0x7fd4804f2660}, {SIG_DFL, [], 0}, 8) = 0
06:21:51 rt_sigaction(SIGQUIT, {SIG_DFL, [], SA_RESTORER, 0x7fd4804f2660}, {SIG_DFL, [], SA_RESTORER, 0x7fd4804f2660}, 8) = 0
06:21:51 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
06:21:51 rt_sigaction(SIGQUIT, {SIG_IGN, [], SA_RESTORER, 0x7fd4804f2660}, {SIG_DFL, [], SA_RESTORER, 0x7fd4804f2660}, 8) = 0
06:21:51 uname({sys="Linux", node="somenode.servers.domain.tld", ...}) = 0
06:21:51 stat("/tmp", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=266240, ...}) = 0
06:21:51 stat(".", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=266240, ...}) = 0
06:21:51 getpid()                       = 19228
06:21:51 open("/usr/lib64/gconv/gconv-modules.cache", O_RDONLY) = 3
06:21:51 fstat(3, {st_mode=S_IFREG|0644, st_size=26060, ...}) = 0
06:21:51 mmap(NULL, 26060, PROT_READ, MAP_SHARED, 3, 0) = 0x7fd480e90000
06:21:51 close(3)                       = 0
06:21:51 getppid()                      = 19225
06:21:51 stat(".", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=266240, ...}) = 0
06:21:51 stat("/opt/nvm/versions/node/v6.11.1/bin/bash", 0x7fff90f48f40) = -1 ENOENT (No such file or directory)
06:21:51 stat("/sbin/bash", 0x7fff90f48f40) = -1 ENOENT (No such file or directory)
06:21:51 stat("/bin/bash", {st_mode=S_IFREG|0755, st_size=906248, ...}) = 0
06:21:51 stat("/bin/bash", {st_mode=S_IFREG|0755, st_size=906248, ...}) = 0
06:21:51 geteuid()                      = 0
06:21:51 getegid()                      = 0
06:21:51 getuid()                       = 0
06:21:51 getgid()                       = 0
06:21:51 access("/bin/bash", X_OK)      = 0
06:21:51 stat("/bin/bash", {st_mode=S_IFREG|0755, st_size=906248, ...}) = 0
06:21:51 geteuid()                      = 0
06:21:51 getegid()                      = 0
06:21:51 getuid()                       = 0
06:21:51 getgid()                       = 0
06:21:51 access("/bin/bash", R_OK)      = 0
06:21:51 stat("/bin/bash", {st_mode=S_IFREG|0755, st_size=906248, ...}) = 0
06:21:51 stat("/bin/bash", {st_mode=S_IFREG|0755, st_size=906248, ...}) = 0
06:21:51 geteuid()                      = 0
06:21:51 getegid()                      = 0
06:21:51 getuid()                       = 0
06:21:51 getgid()                       = 0
06:21:51 access("/bin/bash", X_OK)      = 0
06:21:51 stat("/bin/bash", {st_mode=S_IFREG|0755, st_size=906248, ...}) = 0
06:21:51 geteuid()                      = 0
06:21:51 getegid()                      = 0
06:21:51 getuid()                       = 0
06:21:51 getgid()                       = 0
06:21:51 access("/bin/bash", R_OK)      = 0
06:21:51 getpgrp()                      = 19225
06:21:51 rt_sigaction(SIGCHLD, {0x43f860, [], SA_RESTORER|SA_RESTART, 0x7fd4804f2660}, {SIG_DFL, [], SA_RESTORER|SA_RESTART, 0x7fd4804f2660}, 8) = 0
06:21:51 getrlimit(RLIMIT_NPROC, {rlim_cur=127800, rlim_max=127800}) = 0
06:21:51 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
06:21:51 getpeername(0, 0x7fff90f492c0, [16]) = -1 ENOTSOCK (Socket operation on non-socket)
06:21:51 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
06:21:51 rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
06:21:51 fstat(1, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
06:21:51 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd480e8f000
06:21:51 write(1, "\n", 1)              = 1
06:21:51 exit_group(0)                  = ?
06:21:51 +++ exited with 0 +++
  • Everything up to the third instance of "No such file or directory" is just libraries loading. Good to know if you're wondering where your environment is loading values from; but not useful in our case.
  • Eventually execution proceeds to the first "fstat", this is where all our magic happens. The structure itself is well described and returns a pile of detail about where/how/when the file needs inputs and outputs / etc.
  • The fstat returns a File Descriptor Number (fd) that we can use to write contents to.
  • You'll notice on this line that the fstat returned the file permission 0644; we'll get into that in a sec!
  • The mmap then copies the streamed contents into the file (This is a topic for another time; it actually maps the content into the kernel's dentry cache; and the exit triggers the SYNC call to the filesystem; but I digress)
  • We 'write' a single newline character to FD 1
  • time to exit and let the OS clean up.

When a file is created; applications use some hard-coded default values to determine what permissions should be set in the structure; namely:

  • 0666 for Files
  • 0777 for Directories (folders)

These are then altered by the umask value set in the operating context.

Whoh; lots of numbers

For Redhat / Centos users; the UMASK value is typically set in /etc/profile at login or appication start time (though for services / daemons; these are typically set in /etc/sysconfig/[application_name]) by the following code:

if [ $UID -gt 199 ] && [ "`id -gn`" = "`id -un`" ]; then
    umask 002
else
    umask 022
fi

This block set's all "regular" users (UID's > 200 who's primary group is equal to their username) to a UMASK value of 002; and all othere ('system' users) to 002.

Umasks are subtracted (they're actually NAND'd at the binary level) from the permissions set on a file. This means that because we ran the above example code as the root user; (UID 0) we got stuck with the 022 default umask.

0666 - 022 => 0644

Exactly the number we got when creating our file.

Directories are created in a very similar way, they would end up UMASK'd against the same value yeilding 755 folders; permitting All rights to the owner; but only read + execute to the group or other users.

Back to how we got here

In this specific circumstance; I was attempting to explain some unusual behaviour to a developer; they have an application that is uploading files. In this case; that application uses the Laravel PHP framework. The developer had crafted a simple file upload form using the frameworks included Filesystem drivers. (this in turn actually implements the Composer accessible package/library: Flysystem )

The upload form works great; but the files land on the filesystem with incorrect permissions. As mentioned earlier; the webserver (NGINX) runs as a limited user; with no shell and very minimal filesystem permissions. All paths that needs to be read/written by NGINX are owned by the user of the same name.

The developer connects to the server as some other user; that user is a member of the NGINX group. We need to ensure that files NGINX creates are available to group users; so NGINX(actually PHP-FPM in this case; NGINX never actually serves any files; it just handles requests and hands them to PHP-FPM) runs with a UMASK value of 0002.

In theory; this should work. About a year ago; when I first saw this problem; I was left scratching my head for nearly an hour.

I began tracing the php-FPM execution path; and learned that the file was actually being created in the temp directory with the correct permissions (664) but was being CHMOD'd by the framework itself to 644 and losing it's group write bit. This was due to the default configuration array present in Flysystem.

To verify and confirm that I wasn't totally crazy; I threw together the following one-file-script to test operation:

<?php
if($_SERVER['REQUEST_METHOD'] === 'POST') {

$target_dir = "uploads/";
$target_file = $target_dir . basename($_FILES["fileToUpload"]["name"]);
$uploadOk = 1;
$imageFileType = pathinfo($target_file,PATHINFO_EXTENSION);
// Check if image file is a actual image or fake image
if(isset($_POST["submit"])) {
    $check = getimagesize($_FILES["fileToUpload"]["tmp_name"]);
    if($check !== false) {
        echo "File is an image - " . $check["mime"] . ".";
        $uploadOk = 1;
    } else {
        echo "File is not an image.";
        $uploadOk = 0;
    }
}
// Check if file already exists
if (file_exists($target_file)) {
    echo "Sorry, file already exists.";
    $uploadOk = 0;
}
// Check file size
if ($_FILES["fileToUpload"]["size"] > 500000) {
    echo "Sorry, your file is too large.";
    $uploadOk = 0;
}
// Allow certain file formats
if($imageFileType != "jpg" && $imageFileType != "png" && $imageFileType != "jpeg"
&& $imageFileType != "gif" ) {
    echo "Sorry, only JPG, JPEG, PNG & GIF files are allowed.";
    $uploadOk = 0;
}
// Check if $uploadOk is set to 0 by an error
if ($uploadOk == 0) {
    echo "Sorry, your file was not uploaded.";
// if everything is ok, try to upload file
} else {
    if (move_uploaded_file($_FILES["fileToUpload"]["tmp_name"], $target_file)) {
        echo "The file ". basename( $_FILES["fileToUpload"]["name"]). " has been uploaded.";
    } else {
        echo "Sorry, there was an error uploading your file.";
    }
}

} else {
?>
<!DOCTYPE html>
<html>
<body>

<form action="uploadtest.php" method="post" enctype="multipart/form-data">
    Select image to upload:
    <input type="file" name="fileToUpload" id="fileToUpload">
    <input type="submit" value="Upload Image" name="submit">
</form>

</body>
</html>

<?php
}

Throw that file into /your/web/root/uploadtest.php, mkdir /your/web/root/uploads and chmod 775 /your/web/root/uploads. Access the page using a web browser; and upload an image. You'll confirm what your umask value is set to pretty darn quickly.

LOTS more to come on this topic; I deal with permissions a lot.

Happy December!!