Standard library · Encoding and data

encoding.value

A dynamic value for data whose shape you do not know in advance; the shared language of every format in this group.

import encoding.value

Value is the dynamic data type shared by the encoding packages. One Value is a null, a bool, a 64-bit int, a float, a string, an array of values or an object mapping strings to values, and its kind field says which. encoding.json, encoding.yaml, encoding.toml, encoding.msgpack and others decode into it and encode from it, and encoding.serde converts between formats through it.

Reach for a Value when the shape of the data is not known when you write the program: a webhook payload, a config with optional sections, a document you only pass along. When the shape is fixed, decoding straight into a struct (for example with json.Unmarshal) is shorter and checks the types for you.

The functions in five groups

JobFunctions
Make a valueValueNull, ValueBool, ValueInt, ValueFloat, ValueString, NewValueArray, NewValueObject
Ask the kindIsNull, IsBool, IsInt, IsFloat, IsNumber, IsString, IsArray, IsObject
ReadAsBool, AsInt, AsFloat, AsString, ArrayLen, ArrayGet, ObjectGet, ObjectGetOk, ObjectHas, ObjectKeys
Read and check, throwing on a wrong kind or a missing keythe methods field, at, int, float, str, bool
ChangeArrayPush, ArraySet, ObjectSet, ObjectDelete, ValueClone, ValueFree

What trips people up

  • The As functions never check the kind. AsString of a number is "" and AsInt of a string is 0. When the data comes from outside, use the checked methods instead: v.field("port").int() throws MissingKey or TypeMismatch with a message that names the key and both kinds, and try v.field("port").int() catch 8080 gives a default.
  • Missing is null, all the way down. ObjectGet on a missing key, or on something that is not an object, returns null, so a chain like ObjectGet(ObjectGet(doc, "a"), "b") is safe. ArrayGet is not: an index past ArrayLen stops the program.
  • Copies share parts. A copy of an object shares its map, so ObjectSet through the copy changes the original. A copy of an array shares its elements but not its length, so ArrayPush on an array you got from ObjectGet is lost until you ObjectSet it back. Use ValueClone for a real copy.
  • Key order depends on how the object was built. ObjectKeys returns keys in the order they were added with ObjectSet, and a decoded JSON document in the order of its text. An object made from a map by ValueObject follows that map's own order. Sort the keys, or encode with sort_keys, when output must be alphabetical.

Guide: walk a whole document

A recursive function over Value handles any shape: for an object, visit each key; for an array, each index; anything else is a leaf. This one prints the path of every leaf, which is handy for logging a webhook payload or comparing two documents line by line.

encoding-value-guide-walk.salam
import encoding.value
import encoding.json
import sort
import str

func walk(v: value.Value, path: str):
    if value.IsObject(v):
        keys := value.ObjectKeys(v)
        sort.QuickSort(keys)
        each k in keys:
            walk(value.ObjectGet(v, k), path + "." + k)
        end
    else value.IsArray(v):
        repeat value.ArrayLen(v) in i:
            walk(value.ArrayGet(v, i), path + "[" + str.FromInt(i) + "]")
        end
    else:
        println path, "=", value.ToDebugString(v)
    end
end

func main throws:
    doc := try json.Decode(`{"order": 1042, "customer": {"name": "Sara", "vip": true}, "lines": [{"sku": "BK-01", "qty": 2}, {"sku": "PN-07", "qty": 1}], "note": null}`)
    walk(doc, "$")
end
Output$.customer.name = string(Sara) $.customer.vip = bool(true) $.lines[0].qty = int(2) $.lines[0].sku = string(BK-01) $.lines[1].qty = int(1) $.lines[1].sku = string(PN-07) $.note = null $.order = int(1042)

ObjectKeys gives the keys in the order they were added, so a decoded document keeps the order of its text. This program sorts them with sort.QuickSort to print them alphabetically whatever order the input had. ToDebugString shows each leaf together with its kind.

Guide: build a payload with optional fields

A struct always writes all its fields. When some fields should only appear when they have a value, as many HTTP APIs expect, build a Value object instead and set only what you have. json.Encode with sort_keys prints it in a stable order.

encoding-value-guide-build.salam
import encoding.value
import encoding.json

func payload(name: str, email: str, age: int, tags: Vector<str>): value.Value:
    mut v := value.NewValueObject()
    value.ObjectSet(v, "name", value.ValueString(name))
    if email != "":
        value.ObjectSet(v, "email", value.ValueString(email))
    end
    if age > 0:
        value.ObjectSet(v, "age", value.ValueInt(age))
    end
    if tags.Len() > 0:
        mut arr := value.NewValueArray()
        each t in tags:
            value.ArrayPush(arr, value.ValueString(t))
        end
        value.ObjectSet(v, "tags", arr)
    end
    ret v
end

func main:
    opts := json.JSONOptions { sort_keys = true }
    mut tags := Vector {} as Vector<str>
    println json.Encode(payload("Sara", "", 0, tags), opts)
    tags.Push("beta")
    tags.Push("vip")
    println json.Encode(payload("Reza", "reza@example.com", 41, tags), opts)
end
Output{"name": "Sara"} {"age": 41, "email": "reza@example.com", "name": "Reza", "tags": ["beta", "vip"]}

Guide: change a nested field

To change something deep in a decoded document, take the inner part with ObjectGet, change it, and store it back with ObjectSet. For objects the store is not strictly needed, because the map is shared, but for arrays it is: the program below shows the new order line missing until the array is set back.

encoding-value-guide-edit.salam
import encoding.value
import encoding.json

func line(sku: str, qty: int): value.Value:
    mut l := value.NewValueObject()
    value.ObjectSet(l, "sku", value.ValueString(sku))
    value.ObjectSet(l, "qty", value.ValueInt(qty))
    ret l
end

func main throws:
    mut order := try json.Decode(`{"id": 1042, "customer": {"name": "Sara"}, "lines": [{"sku": "BK-01", "qty": 2}]}`)

    mut customer := value.ObjectGet(order, "customer")
    value.ObjectSet(customer, "tier", value.ValueString("gold"))

    mut lines := value.ObjectGet(order, "lines")
    value.ArrayPush(lines, line("PN-07", 1))
    println "before ObjectSet:", value.ArrayLen(value.ObjectGet(order, "lines")), "line(s)"
    value.ObjectSet(order, "lines", lines)
    println "after ObjectSet: ", value.ArrayLen(value.ObjectGet(order, "lines")), "line(s)"

    println json.Encode(order, json.JSONOptions { sort_keys = true })

    city := value.ObjectGet(value.ObjectGet(order, "shipping"), "city")
    println "shipping city given:", not value.IsNull(city), "[" + value.AsString(city) + "]"
    try:
        println order.field("shipping").field("city").str()
    catch e:
        println "checked read:", e.ToStr()
    end
end
Outputbefore ObjectSet: 1 line(s) after ObjectSet: 2 line(s) {"customer": {"name": "Sara", "tier": "gold"}, "id": 1042, "lines": [{"qty": 2, "sku": "BK-01"}, {"qty": 1, "sku": "PN-07"}]} shipping city given: false [] checked read: missing key 'shipping'

Always writing the changed part back with ObjectSet makes the code correct for both kinds, and keeps working if the representation changes. The last line reads a field of a shipping object that does not exist: the chain of ObjectGet calls returns null and AsString gives an empty string, with no crash. The checked form, order.field("shipping"), throws instead and names the missing key, which is what you want when the field is required.

Guide: deep merge of settings

Settings often come in layers: built-in defaults, then a file per site. A deep merge keeps every default the site file does not mention and replaces only what it does, one level at a time. Arrays and plain values in the override win as a whole.

encoding-value-guide-merge.salam
import encoding.value
import encoding.json

func merge(base: value.Value, over: value.Value): value.Value:
    if not value.IsObject(base) or not value.IsObject(over):
        ret value.ValueClone(over)
    end
    mut out := value.ValueClone(base)
    each k in value.ObjectKeys(over):
        o := value.ObjectGet(over, k)
        if value.ObjectHas(out, k):
            value.ObjectSet(out, k, merge(value.ObjectGet(out, k), o))
        else:
            value.ObjectSet(out, k, value.ValueClone(o))
        end
    end
    ret out
end

func main throws:
    defaults := try json.Decode(`{"server": {"host": "0.0.0.0", "port": 8080}, "log": {"level": "info", "json": false}, "plugins": ["auth"]}`)
    site := try json.Decode(`{"server": {"port": 9000}, "log": {"level": "debug"}, "plugins": ["auth", "cache"]}`)

    cfg := merge(defaults, site)
    println json.Encode(cfg, json.JSONOptions { indent = "  ", sort_keys = true })
    println "defaults untouched:", try defaults.field("server").field("port").int()
end
Output{ "log": { "json": false, "level": "debug" }, "plugins": [ "auth", "cache" ], "server": { "host": "0.0.0.0", "port": 9000 } } defaults untouched: 8080

The function clones what it keeps, so defaults is the same afterwards. Without the clones, the result would share its objects with the inputs, as the next section shows.

Guide: copies, conversions and missing keys

Assigning a Value to a new variable does not copy its contents: an object's fields live in a shared map, so a change through the copy shows up in the original. Call ValueClone when you need an independent copy, and ValueFree when you are done with it. Free only values nothing else shares, since freeing a plain copy also frees the original's map.

encoding-value-guide-pitfalls.salam
import encoding.value

func main:
    mut a := value.NewValueObject()
    value.ObjectSet(a, "n", value.ValueInt(1))

    mut shared := a
    value.ObjectSet(shared, "n", value.ValueInt(2))
    println "after editing a plain copy:", value.AsInt(value.ObjectGet(a, "n"))

    mut own := value.ValueClone(a)
    value.ObjectSet(own, "n", value.ValueInt(3))
    println "after editing a clone:", value.AsInt(value.ObjectGet(a, "n"))

    println "AsInt of 2.9:", value.AsInt(value.ValueFloat(2.9))
    println "AsFloat of 3:", value.AsFloat(value.ValueInt(3))
    println "AsInt of \"5\":", value.AsInt(value.ValueString("5"))
    println "1 equals 1.0:", value.Equal(value.ValueInt(1), value.ValueFloat(1.0))

    mut found := false
    _ := value.ObjectGetOk(a, "missing", found)
    println "missing key found:", found

    value.ValueFree(own)
end
Outputafter editing a plain copy: 2 after editing a clone: 2 AsInt of 2.9: 2 AsFloat of 3: 3 AsInt of "5": 0 1 equals 1.0: false missing key found: false
  • AsInt cuts a float toward zero and AsFloat widens an int, but neither parses text: a string gives 0. Check the kind first with IsInt, IsNumber or IsString.
  • Equal compares kinds too, so the integer 1 and the float 1.0 are different values.
  • ObjectGet on a missing key gives a null value; ObjectGetOk also tells you whether the key was there, and field throws MissingKey for it while an explicit null comes back as a null value.

Types

Value

struct Value
A dynamic value: null, boolean, whole number, decimal, string, array or object.
NameTypeDescription
kindintThe kind of the value; one of the Kind constants.
bbool = falseThe boolean value.
ii64 = 0The whole number value.
nf64 = 0The decimal value.
sstr = ""The string value.
arrVectorThe elements of an array.
objHashMapThe keys and values of an object.
orderi64 = 0An internal handle to the list that records the order in which keys were added to an object with ObjectSet. ObjectDelete keeps it up to date. Do not use it directly; ObjectKeys reads it.
binVectorThe raw bytes of a binary or extension value.
encoding-value-Value.salam
import encoding.value

func main:
    mut user := value.NewValueObject()
    value.ObjectSet(user, "name", value.ValueString("Narges"))
    value.ObjectSet(user, "age", value.ValueInt(34))
    value.ObjectSet(user, "height", value.ValueFloat(1.68))
    value.ObjectSet(user, "admin", value.ValueBool(false))
    value.ObjectSet(user, "phone", value.ValueNull())
    mut languages := value.NewValueArray()
    value.ArrayPush(languages, value.ValueString("fa"))
    value.ArrayPush(languages, value.ValueString("en"))
    value.ArraySet(languages, 1, value.ValueString("ar"))
    value.ObjectSet(user, "langs", languages)
    println value.ToDebugString(user)
    println value.ObjectLen(user), value.ObjectHas(user, "age"), value.ObjectKeys(user)
    println value.AsString(value.ObjectGet(user, "name")), value.AsInt(value.ObjectGet(user, "age")), value.AsFloat(value.ObjectGet(user, "height")), value.AsBool(value.ObjectGet(user, "admin"))
    langs := value.ObjectGet(user, "langs")
    println value.IsArray(langs), value.ArrayLen(langs), value.AsString(value.ArrayGet(langs, 1))
    mut present := false
    _ := value.ObjectGetOk(user, "email", present)
    println present
    phone := value.ObjectGet(user, "phone")
    println value.IsNull(phone), value.IsBool(phone), value.IsInt(value.ValueInt(1)), value.IsFloat(value.ValueFloat(1.0)), value.IsNumber(value.ValueInt(1))
    println value.IsString(value.ValueString("")), value.IsObject(user), phone.kind == value.KindNull, value.KindObject
    mut copy := value.ValueClone(user)
    println value.Equal(copy, user)
    removed := value.ObjectDelete(copy, "phone")
    println removed, value.Equal(copy, user)
    mut numbers := Vector {} as Vector<value.Value>
    numbers.Push(value.ValueInt(1))
    mut fields := HashMap {} as HashMap<str, value.Value>
    fields.Put("x", value.ValueArray(numbers))
    println value.ToDebugString(value.ValueObject(fields))
    value.ValueFree(copy)
end
Outputobject{name: string(Narges), age: int(34), height: float(1.68), admin: bool(false), phone: null, langs: array[string(fa), string(ar)]} 6 true ["name", "age", "height", "admin", "phone", "langs"] Narges 34 1.68 false true 2 ar false true false true true true true true true 6 true true false object{x: array[int(1)]}

Field

func Field(key: str): Throws<Value>

field

func field(key: str): Throws<Value>
The value of a key in an object. Throws MissingKey when the key is absent and TypeMismatch when the value is not an object, so a missing key and an explicit null are told apart.
encoding-value-Value.field.salam
import encoding.json

func main throws:
    doc := try json.Decode(`{"name": "Lamp", "price": 19.5, "stock": 12, "sale": true, "tags": ["home", "light"]}`)
    println try doc.field("name").str()
    println try doc.field("price").float()
    println try doc.field("stock").int()
    println try doc.field("sale").bool()
    println try doc.field("tags").at(1).str()
    try:
        println doc.field("color").str()
    catch e:
        println e.kind_name(), "->", e.message
    end
    try:
        println doc.field("tags").at(5).str()
    catch e:
        println e.kind_name(), "->", e.message
    end
    try:
        println doc.field("name").int()
    catch e:
        println e.kind_name(), "->", e.message
    end
    count := try doc.field("count").int() catch 0
    println "count or default:", count
end
OutputLamp 19.5 12 true light missing key -> missing key 'color' out of range -> index 5 is out of range for an array of 2 type mismatch -> expected an integer, got a string count or default: 0

At

func At(index: int): Throws<Value>

at

func at(index: int): Throws<Value>
Element number index of an array. Throws OutOfRange for an index outside the array and TypeMismatch when the value is not an array.

The example for this item is shown with the ones above: encoding-value-Value.field.salam

Int

func Int(): Throws<i64>

int

func int(): Throws<i64>
The whole number. Throws TypeMismatch for any other kind, a decimal included.

The example for this item is shown with the ones above: encoding-value-Value.field.salam

Float

func Float(): Throws<f64>

float

func float(): Throws<f64>
The number as a decimal; a whole number is converted. Throws TypeMismatch for any other kind.

The example for this item is shown with the ones above: encoding-value-Value.field.salam

Str

func Str(): Throws<str>

str

func str(): Throws<str>
The string. Throws TypeMismatch for any other kind.

The example for this item is shown with the ones above: encoding-value-Value.field.salam

Bool

func Bool(): Throws<bool>

bool

func bool(): Throws<bool>
The boolean. Throws TypeMismatch for any other kind.

The example for this item is shown with the ones above: encoding-value-Value.field.salam

Constants and variables

KindExt

const KindExt: i32 = 8
An extension value: a type number and a payload.

The example for this item is shown with the ones above: encoding-value-Value.salam

Functions

Equal

func Equal(a: Value, b: Value): bool
True when two values, with all their nested elements, are equal.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueBool

func ValueBool(b: bool): Value
A boolean value.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueInt

func ValueInt(i: i64): Value
A whole number value.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueFloat

func ValueFloat(n: f64): Value
A decimal value.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueString

func ValueString(s: str): Value
A string value.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueBinary

func ValueBinary(p: void*, n: int): Value
Binary data copied from a memory range.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueBinaryStr

func ValueBinaryStr(s: str): Value
Binary data copied from a string.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueExt

func ValueExt(typeId: int, p: void*, n: int): Value
An extension value copied from a memory range.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueExtStr

func ValueExtStr(typeId: int, s: str): Value
An extension value copied from a string.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueArray

func ValueArray(items: Vector<Value>): Value
An array from a vector.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueObject

func ValueObject(fields: HashMap<str, Value>): Value
An object from a map.

The example for this item is shown with the ones above: encoding-value-Value.salam

NewValueArray

func NewValueArray(): Value
An empty array.

The example for this item is shown with the ones above: encoding-value-Value.salam

NewValueObject

func NewValueObject(): Value
An empty object.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsNull

func IsNull(v: Value): bool
True when the value is null.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsBool

func IsBool(v: Value): bool
True when it is a boolean.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsInt

func IsInt(v: Value): bool
True when it is a whole number.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsFloat

func IsFloat(v: Value): bool
True when it is a decimal.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsNumber

func IsNumber(v: Value): bool
True when it is a whole number or a decimal.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsString

func IsString(v: Value): bool
True when it is a string.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsArray

func IsArray(v: Value): bool
True when it is an array.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsObject

func IsObject(v: Value): bool
True when it is an object.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsBytes

func IsBytes(v: Value): bool
True when the value holds binary data.

The example for this item is shown with the ones above: encoding-value-Value.salam

IsExt

func IsExt(v: Value): bool
True when the value is an extension value.

The example for this item is shown with the ones above: encoding-value-Value.salam

BytesLen

func BytesLen(v: Value): int
The number of bytes.

The example for this item is shown with the ones above: encoding-value-Value.salam

BytesGet

func BytesGet(v: Value, i: int): int
Byte number i; 0 when it is out of range.

The example for this item is shown with the ones above: encoding-value-Value.salam

BytesToBuf

func BytesToBuf(v: Value): compress.ByteBuf
A new buffer holding a copy of the bytes.

The example for this item is shown with the ones above: encoding-value-Value.salam

BytesToStr

func BytesToStr(v: Value): str
The bytes as a string; it stops at the first zero byte, so use the buffer for arbitrary data.

The example for this item is shown with the ones above: encoding-value-Value.salam

ExtType

func ExtType(v: Value): int
The extension type number, from -128 to 127.

The example for this item is shown with the ones above: encoding-value-Value.salam

ExtDataLen

func ExtDataLen(v: Value): int
The number of bytes in the extension payload.

The example for this item is shown with the ones above: encoding-value-Value.salam

ExtDataGet

func ExtDataGet(v: Value, i: int): int
Payload byte number i; 0 when it is out of range.

The example for this item is shown with the ones above: encoding-value-Value.salam

ExtDataToBuf

func ExtDataToBuf(v: Value): compress.ByteBuf
A new buffer holding a copy of the payload.

The example for this item is shown with the ones above: encoding-value-Value.salam

ExtDataToStr

func ExtDataToStr(v: Value): str
The payload as a string; it stops at the first zero byte.

The example for this item is shown with the ones above: encoding-value-Value.salam

AsBool

func AsBool(v: Value): bool
The boolean value; false for any other kind.

The example for this item is shown with the ones above: encoding-value-Value.salam

AsInt

func AsInt(v: Value): i64
The whole number value; a decimal is truncated toward zero, and any other kind gives 0.

The example for this item is shown with the ones above: encoding-value-Value.salam

AsFloat

func AsFloat(v: Value): f64
The decimal value; a whole number is converted, and any other kind gives 0.

The example for this item is shown with the ones above: encoding-value-Value.salam

AsString

func AsString(v: Value): str
The string value; an empty string for any other kind.

The example for this item is shown with the ones above: encoding-value-Value.salam

ArrayLen

func ArrayLen(v: Value): int
The number of elements in an array; 0 for any other kind.

The example for this item is shown with the ones above: encoding-value-Value.salam

ArrayGet

func ArrayGet(v: Value, i: int): Value
Element number i; an index outside 0 to ArrayLen - 1 stops the program.

The example for this item is shown with the ones above: encoding-value-Value.salam

ArraySet

func ArraySet(v &: Value, i: int, item: Value)
Replaces element number i.

The example for this item is shown with the ones above: encoding-value-Value.salam

ArrayPush

func ArrayPush(v &: Value, item: Value)
Adds an element at the end of an array. If the array came from ObjectGet or ArrayGet, store it back for the parent to see the new length.

The example for this item is shown with the ones above: encoding-value-Value.salam

ObjectLen

func ObjectLen(v: Value): int
The number of keys in an object.

The example for this item is shown with the ones above: encoding-value-Value.salam

ObjectHas

func ObjectHas(v: Value, key: str): bool
True when the key is there.

The example for this item is shown with the ones above: encoding-value-Value.salam

ObjectGet

func ObjectGet(v: Value, key: str): Value
The value for a key, or null when it is absent or v is not an object.

The example for this item is shown with the ones above: encoding-value-Value.salam

ObjectGetOk

func ObjectGetOk(v: Value, key: str, ok &: bool): Value
The value for a key, with ok saying whether the key was there; for telling "absent" apart from "null".

The example for this item is shown with the ones above: encoding-value-Value.salam

ObjectSet

func ObjectSet(v &: Value, key: str, item: Value)
Sets a key.

The example for this item is shown with the ones above: encoding-value-Value.salam

ObjectDelete

func ObjectDelete(v &: Value, key: str): bool
Removes a key and says whether it was there.

The example for this item is shown with the ones above: encoding-value-Value.salam

ObjectKeys

func ObjectKeys(v: Value): Vector<str>
The keys of an object. Keys added with ObjectSet come in the order they were added; the keys of an object made from a map by ValueObject come in that map's own order.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueClone

func ValueClone(v: Value): Value
A complete, independent copy of a value.

The example for this item is shown with the ones above: encoding-value-Value.salam

ValueFree

func ValueFree(v &: Value)
Releases the memory of a value and its elements.

The example for this item is shown with the ones above: encoding-value-Value.salam

ToDebugString

func ToDebugString(v: Value): str
A readable form with the kind of every element, such as object{name: string(Ali)}, for debugging.

The example for this item is shown with the ones above: encoding-value-Value.salam