Standard library · System and tools

os

Files, directories, system commands, program arguments and environment variables.

import os

The os package is the simplest way to talk to the system: whole-file reads and writes, folders, shell commands, the program's arguments and its environment. Each function does one thing and takes paths as plain strings. Reach for it in scripts, build tools and small servers. When you need to know why something failed, or to read a big file piece by piece, the fs family (fs for files, fs.dir, fs.path) does more; for building paths use filepath, and for configuration from the environment env.

Results follow two conventions. Functions that change something (Remove, Mkdir, Move, Copy, MkdirAll, RemoveAll) return 0 on success and -1 on failure, with no reason attached. WriteFile and AppendFile return the number of bytes written, or -1. Functions that read return an empty value when they fail: ReadFile gives empty text and ListDir an empty vector for a missing folder.

There are sub-packages for more specialised work: os.process for running other programs with exact arguments, os.shell for capturing command output and errors, os.platform for identifying the system, os.temp for temporary files, os.user for the current user and os.error for error codes.

What people get wrong:

  • Building a shell command from user input. Run, RunCapture and Output hand the whole string to /bin/sh, so a name such as x; rm -rf ~ runs the second command too.
  • Expecting RunCapture to capture errors. It captures standard output only; the command's standard error still goes to the terminal. Add 2>&1 to the command when you want both.
  • Running Walk on a tree with symbolic links. It follows a link to a folder as if it were a real folder, so it lists what is behind it and never ends on a link loop. RemoveAll does not follow links: it deletes the link itself and leaves its target alone.
  • Using Move across disks. It is a rename, and a rename between file systems fails with -1; copy and remove instead.

The sections below walk through the jobs this package is used for most, each with a complete program and the output it printed. The programs work in the current directory and remove what they create.

Read a settings file with defaults

A small program usually keeps its settings in a text file next to it. Exists tells you whether the file is there yet, WriteFile creates it with sensible values the first time, and ReadFile gives you the whole text to pick apart. Every setting that is missing falls back to a default, so an old file keeps working after you add a new option.

os-guide-config.salam
import os

func setting(text: str, key: str, fallback: str): str:
    each line in text.Split("\n"):
        if line.StartsWith(key + "="):
            ret line.Substr(key.Len() + 1, line.Len() - key.Len() - 1)
        end
    end
    ret fallback
end

func main:
    path := "app.conf"
    if not os.Exists(path):
        os.WriteFile(path, "port=8080\nname=notes\n")
        println "wrote a default", path
    end
    text := os.ReadFile(path)
    println "port:", setting(text, "port", "80")
    println "name:", setting(text, "name", "app")
    println "debug:", setting(text, "debug", "off")
    os.Remove(path)
end
Outputwrote a default app.conf port: 8080 name: notes debug: off

ReadFile returns an empty string both for an empty file and for one it cannot read. When the difference matters, ask Exists first, as here. For a real format, read the text with encoding.json or a similar package instead of splitting lines by hand.

Keep a log and rotate it

AppendFile adds to the end of a file and creates it when it is missing, which is all a log needs. FileSize says how big the log has become; once it passes a limit, Move renames it out of the way and the next AppendFile starts a fresh one.

os-guide-log.salam
import os

func log(path: str, line: str):
    os.AppendFile(path, line + "\n")
end

func main:
    path := "app.log"
    log(path, "started")
    log(path, "user sara signed in")
    log(path, "stopped")
    println "size:", os.FileSize(path), "bytes"
    lines := os.ReadFile(path).Split("\n")
    println "first entry:", lines.Get(0)
    if os.FileSize(path) > 32:
        os.Move(path, "app.log.1")
        println "rotated:", os.Exists("app.log.1"), os.Exists(path)
    end
    log(path, "a fresh log")
    println "new size:", os.FileSize(path)
    os.Remove(path)
    os.Remove("app.log.1")
end
Outputsize: 36 bytes first entry: started rotated: true false new size: 12

Each call opens and closes the file, which is fine for a few lines a second. For a busy log, open it once with Open(path, "ab") and write to the handle.

Create, walk and remove a directory tree

MkdirAll creates a path with every parent it needs, so the order you create folders in does not matter. Walk lists every file below a directory, at any depth, and ListDirs only the folders directly inside it. RemoveAll deletes the whole tree at the end.

os-guide-tree.salam
import os
import sort

func main:
    os.MkdirAll("site/pages/blog")
    os.MkdirAll("site/assets")
    os.WriteFile("site/pages/index.salam", "func main: end")
    os.WriteFile("site/pages/blog/first.salam", "func main: end")
    os.WriteFile("site/assets/logo.svg", "<svg/>")
    mut count := 0
    mut bytes := 0 as i64
    each p in os.Walk("site"):
        if p.EndsWith(".salam"):
            count++
            bytes += os.FileSize(p)
        end
    end
    println "source files:", count, "with", bytes, "bytes"
    folders := os.ListDirs("site")
    sort.Sort(folders)
    each name in folders:
        println "folder:", name
    end
    os.RemoveAll("site")
    println "cleaned up:", not os.Exists("site")
end
Outputsource files: 2 with 28 bytes folder: assets folder: pages cleaned up: true

Walk returns full paths that start with the folder you gave it, while ListDir and ListDirs return bare names, hidden ones included and . and .. left out. The system returns directory entries in no particular order, and the order changes between file systems. Sort the names, as the program does with sort, before you print them or compare them with something.

Run shell commands and check them

Run, RunCapture and Output pass the command to the system shell, so pipes, redirection and quoting work just as they do in a terminal. Run only gives back the exit code, and the command prints straight to the program's own output; that is enough to ask whether a tool is installed. RunCapture gives the code and the standard output together, and Output only the text, final newline included, so trim it.

os-guide-commands.salam
import os

func tool_exists(name: str): bool:
    ret os.Run("command -v " + name + " > /dev/null") == 0
end

func main:
    println "sh available:", tool_exists("sh")
    println "made-up tool available:", tool_exists("no-such-tool-here")
    r := os.RunCapture("printf 'b\\na\\nc\\n' | sort")
    if r.code != 0:
        printerrln "sort failed with", r.code
        ret 1
    end
    print r.out
    count := os.Output("printf 'x\\ny\\n' | wc -l").Trim()
    println "lines counted by wc:", count
end
Outputsh available: true made-up tool available: false a b c lines counted by wc: 2

Because a shell reads the command, never build one from text a user typed: a name such as x; rm -rf ~ would run the second command too. To pass untrusted values, use os.process, which hands each argument to the program untouched.

Which function for which job

JobFunction
Read or replace a whole small fileReadFile, WriteFile
Add lines to a logAppendFile
Read a big file piece by pieceOpen, then the handle's readline and close
Know if a path is there, and what it isExists, IsDir, or Stat for everything at once
Make or delete foldersMkdirAll, RemoveAll (Mkdir and Rmdir for one level)
Find files below a folderWalk; ListDir for one level
Run a shell commandRun, RunCapture, Output
Run a program with exact argumentsos.process or os.shell
Find out why a file operation failedfs together with os.error

Types

FileInfo

struct FileInfo
What Stat returns.
NameTypeDescription
sizei64 = 0The size of the file in bytes.
is_dirbool = falseTrue when the path is a directory.
existsbool = falseTrue when the path exists.
os-Files.salam
import os

func main:
    os.WriteFile("note.txt", "first line\n")
    os.AppendFile("note.txt", "second line\n")
    print os.ReadFile("note.txt")
    println os.Exists("note.txt"), os.FileSize("note.txt")
    info := os.Stat("note.txt")
    println info.exists, info.is_dir, info.size
    os.Copy("note.txt", "copy.txt")
    os.Move("copy.txt", "new name.txt")
    println os.Exists("copy.txt"), os.Exists("new name.txt")
    os.Remove("new name.txt")
    println os.Exists("new name.txt")
    f := os.Open("note.txt", "rb")
    println f.Readline()
    f.Close()
end
Outputfirst line second line true 23 true false 23 false true false first line

CmdResult

struct CmdResult
The result of running a command with RunCapture.
NameTypeDescription
codeint = 0The command's exit code; zero means success.
outstr = ""The command's output.
os-Run.salam
import os

func main:
    code := os.Run("echo from the shell")
    println "exit code:", code
    r := os.RunCapture("echo hello")
    println r.code, "[" + r.out + "]"
    println "[" + os.Output("echo world") + "]"
    println os.Pid() > 0
end
Outputfrom the shell exit code: 0 0 [hello ] [world ] true

Functions

Run

func Run(cmd: str): int
Runs a command in the system shell (/bin/sh, or cmd on Windows) and returns its exit code; the command's output goes straight to the program's own output.

The example for this item is shown with the ones above: os-Run.salam

RunCapture

func RunCapture(cmd: str): CmdResult
Runs a command and returns its exit code and its standard output together in a CmdResult. Standard error is not captured; append 2>&1 for it. When the shell cannot be started, code is -1.

The example for this item is shown with the ones above: os-Run.salam

Output

func Output(cmd: str): str
Runs a command and returns only its standard output text, including the final newline. The exit code is lost, so use RunCapture when failure matters.

The example for this item is shown with the ones above: os-Run.salam

Pid

func Pid(): int
The process id of the running program.

The example for this item is shown with the ones above: os-Run.salam

Open

func Open(path: str, mode: str): File*
Opens a file in the given mode: "rb" to read, "wb" to rewrite and "ab" to append. It returns null when it cannot.

The example for this item is shown with the ones above: os-Files.salam

Exit

func Exit(code: int)
Ends the program immediately with the given code. Inside main write ret instead, because exiting does not run the defer statements (error E109).
os-Args.salam
import os

func fail(message: str):
    printerrln message
    os.Exit(2)
end

func give_up():
    os.Abort()
end

func main:
    args := os.Args()
    println "argument count:", args.Len()
    home := os.Env("HOME")
    println home.Len() > 0 or os.Env("USERPROFILE").Len() > 0
    if args.Len() > 10:
        fail("too many arguments")
    end
    if args.Len() > 100:
        give_up()
    end
end
Outputargument count: 1 true

Abort

func Abort()
Stops the program abruptly with an error, for a situation with no way to carry on.

The example for this item is shown with the ones above: os-Args.salam

Args

func Args(): Vector<str>
The program's command line arguments; element zero is the program's own name.

The example for this item is shown with the ones above: os-Args.salam

Env

func Env(name: str): str
The value of an environment variable, or an empty string when it is not set (so an empty and a missing variable look the same). The env package does more.

The example for this item is shown with the ones above: os-Args.salam

Remove

func Remove(path: str): int
Deletes a file; 0 on success, -1 on failure. Use Rmdir or RemoveAll for a folder.

The example for this item is shown with the ones above: os-Files.salam

Mkdir

func Mkdir(path: str): int
Creates a directory; the parent has to exist already, and an existing directory counts as a failure (-1).
os-Dirs.salam
import os

func main:
    os.Mkdir("project")
    os.MkdirAll("project/src/helpers")
    os.WriteFile("project/readme.txt", "hello")
    os.WriteFile("project/src/main.salam", "func main: end")
    println os.IsDir("project"), os.IsDir("project/readme.txt")
    println os.ListDir("project/src").Len(), os.ListDirs("project")
    println os.Walk("project").Len()
    os.CopyTree("project", "backup")
    println os.Exists("backup/src/main.salam")
    start := os.Cwd()
    os.Chdir("project")
    println os.Exists("readme.txt")
    os.Chdir(start)
    os.Rmdir("project/src/helpers")
    os.RemoveAll("project")
    os.RemoveAll("backup")
    println os.Exists("project"), os.TempDir().Len() > 0
end
Outputtrue false 2 ["src"] 2 true true false true

Rmdir

func Rmdir(path: str): int
Deletes an empty directory.

The example for this item is shown with the ones above: os-Dirs.salam

Copy

func Copy(src: str, dst: str): int
Copies a file's bytes, replacing the destination. Permissions such as the executable bit are not copied. 0 on success, -1 on failure.

The example for this item is shown with the ones above: os-Files.salam

Move

func Move(src: str, dst: str): int
Moves or renames a file or a directory. It is a rename, so it fails (-1) between two file systems.

The example for this item is shown with the ones above: os-Files.salam

ListDir

func ListDir(path: str): Vector<str>
The names of everything inside a directory, hidden entries included, without . and .. and without descending into subdirectories. A missing directory gives an empty vector.

The example for this item is shown with the ones above: os-Dirs.salam

IsDir

func IsDir(path: str): bool
True when the path is a directory the program can open, or a symbolic link to one.

The example for this item is shown with the ones above: os-Dirs.salam

Walk

func Walk(root: str): Vector<str>
The paths of every file inside a directory and its subdirectories, each starting with root; the directories themselves are not in the list. It follows symbolic links to directories, so a link loop never ends.

The example for this item is shown with the ones above: os-Dirs.salam

Exists

func Exists(path: str): bool
True when the path, file or directory, exists.

The example for this item is shown with the ones above: os-Files.salam

Cwd

func Cwd(): str
The program's current directory.

The example for this item is shown with the ones above: os-Dirs.salam

RealPath

func RealPath(path: str): str
The absolute path with every symbolic link resolved, or empty text when the path does not exist. Compare these, not the paths you were given, when checking that a file stays inside a folder.

The example for this item is shown with the ones above: os-Dirs.salam

Chdir

func Chdir(path: str): int
Changes the current directory; later relative paths are resolved from there.

The example for this item is shown with the ones above: os-Dirs.salam

TempDir

func TempDir(): str
The system's temporary directory: TEMP, TMP or TMPDIR from the environment, in that order, or /tmp.

The example for this item is shown with the ones above: os-Dirs.salam

ReadFile

func ReadFile(path: str): str
Reads the whole content of a file, or an empty string when it cannot. The text stops at the first zero byte, so it is for text, not binary files.

The example for this item is shown with the ones above: os-Files.salam

WriteFile

func WriteFile(path: str, data: str): int
Creates or replaces a file with data and returns the number of bytes written, or -1 when the file cannot be opened (for example when its folder does not exist).

The example for this item is shown with the ones above: os-Files.salam

AppendFile

func AppendFile(path: str, data: str): int
Adds to the end of a file, creating it when it is missing; returns the number of bytes written, or -1.

The example for this item is shown with the ones above: os-Files.salam

FileSize

func FileSize(path: str): i64
The size of a file in bytes, or minus one when it is not there or is a directory.

The example for this item is shown with the ones above: os-Files.salam

Stat

func Stat(path: str): FileInfo
Returns everything about a path at once; when the path is absent, exists is false.

The example for this item is shown with the ones above: os-Files.salam

MkdirAll

func MkdirAll(path: str): int
Creates a directory along with every parent it needs, and does nothing when it already exists. It splits the path on /.

The example for this item is shown with the ones above: os-Dirs.salam

RemoveAll

func RemoveAll(path: str): int
Deletes a file or a directory with everything inside it; a missing path counts as success. Take care: it cannot be undone. A symbolic link is deleted as a link; it is never followed, so its target stays.

The example for this item is shown with the ones above: os-Dirs.salam

CopyTree

func CopyTree(src: str, dst: str): int
Copies a directory with all its subdirectories and files.

The example for this item is shown with the ones above: os-Dirs.salam

ListDirs

func ListDirs(path: str): Vector<str>
Only the names of a directory's subdirectories.

The example for this item is shown with the ones above: os-Dirs.salam