Standard library · Encoding and data

encoding.url

Taking a web address apart, building and editing its parameters, and resolving relative addresses.

import encoding.url

This package works with whole web addresses. Parse splits a URL such as https://user@shop.example.co.uk:8443/path?page=2#top into a URL struct with the fields scheme, userinfo, host, port, path, raw_query and fragment, and String puts it back together. Around that come helpers to edit query parameters, resolve relative links with Join, and find the registrable domain of a host.

Use it whenever you have a full address. For the a=1&b=2 part on its own, such as a form body, use encoding.query; for percent-encoding a single string, encoding.URLEncode in encoding; to fetch the address, net.http.

Raw and decoded parts

The fields hold the text exactly as it appeared, percent escapes included, so String(Parse(x)) normally gives back x. Read the human form with DecodedPath, DecodedUserinfo or Query. When you set a field yourself, put encoded text in it: escape path segments with EncodePathSegment and build queries with SetQuery.

What trips people up

  • Parse throws on text that is not a URL. It is declared throws, so call it with try. It refuses an empty string, a space or control character, a scheme that is not a letter followed by letters, digits, +, - or ., a missing host after :// (only file:/// may have none), a host with characters such as <, a port that is not a number or is above 65535, an IPv6 address without its ], and a % not followed by two hex digits anywhere; the Error has the kind (Syntax, UnexpectedEnd, OutOfRange, InvalidArgument) and the column. A relative reference such as ../a.html or ?page=2 is fine, and text without :// (including mailto:sara@example.com) still has no scheme or host and lands whole in path, so check scheme and host before trusting an address from a user.
  • Nothing is lowercased. HTTPS://Shop.Example.COM keeps its case in scheme and host, and Domain returns Example.COM. Lowercase with str.Lower before you compare hosts or domains.
  • Editing one parameter rewrites the whole query. RemoveParam, SetParam and KeepParams decode the query and encode it again: %20 becomes +, characters such as * get escaped, a bare flag becomes flag=, and repeated keys are grouped together. The meaning stays the same; the text may not.
  • Everything that decodes throws too. Join parses its reference, and DecodedPath, DecodedUserinfo, Query, ParamNames, SetParam, RemoveParam and KeepParams decode a field. An address from Parse always decodes, but a field you set by hand may not.

Guide: strip tracking parameters

Links copied from email and social media carry utm_ and click id parameters that only serve analytics. Before you store or compare links, parse them, drop those parameters with RemoveParam, clear the fragment and print the result with String. The parameters that stay keep their order.

encoding-url-guide-clean.salam
import encoding.url

func clean(raw: str): str throws:
    mut u := try url.Parse(raw)
    names := try url.ParamNames(u)
    each n in names:
        if n.StartsWith("utm_") or n == "fbclid" or n == "gclid":
            _ := try url.RemoveParam(u, n)
        end
    end
    u.fragment = ""
    ret url.String(u)
end

func main:
    links := ["https://shop.example/item/42?color=red&utm_source=news&utm_medium=email#reviews", "https://blog.example/post?fbclid=IwAR0abc", "https://example.com/search?q=salam%20lang&utm_campaign=fall&page=2&debug", "https://example.com/sale?off=50%"]
    each link in links:
        try:
            println clean(link)
        catch e:
            println "skipped:", e.ToStr()
        end
    end
end
Outputhttps://shop.example/item/42?color=red https://blog.example/post https://example.com/search?q=salam+lang&page=2&debug= skipped: url: incomplete percent escape (line 1, column 32)

ParamNames returns a copy of the names, so removing parameters inside the loop is safe. The last link has a bare %, which is not a valid URL, so Parse throws and the loop skips it with the reason. The fields of URL are plain strings you can set directly, as u.fragment = "" does. The third line shows the rewrite described above: salam%20lang comes out as salam+lang and the bare debug as debug=. Both mean the same to a server, but if you use the cleaned link as a cache or database key, clean every link the same way so equal links give equal text.

A crawler or a link checker finds relative links like ../api/json.html in a page and needs the full address. Join resolves a reference against the page's own URL the way a browser does: ./ and ../ move through the folders, a leading / starts at the site root, //host keeps the scheme, a full URL replaces everything, and a reference of only ?query or #fragment keeps the page itself.

encoding-url-guide-links.salam
import encoding.url

func main throws:
    page := try url.Parse("https://docs.example.com/guide/start/index.html?lang=en")
    hrefs := ["install.html", "./setup/linux.html", "../api/json.html", "/blog/", "https://other.example/x", "//cdn.example.com/app.js", "?lang=fa", "#install"]
    each h in hrefs:
        println h, "->", url.String(try url.Join(page, h))
    end
end
Outputinstall.html -> https://docs.example.com/guide/start/install.html ./setup/linux.html -> https://docs.example.com/guide/start/setup/linux.html ../api/json.html -> https://docs.example.com/guide/api/json.html /blog/ -> https://docs.example.com/blog/ https://other.example/x -> https://other.example/x //cdn.example.com/app.js -> https://cdn.example.com/app.js ?lang=fa -> https://docs.example.com/guide/start/index.html?lang=fa #install -> https://docs.example.com/guide/start/index.html?lang=en#install

These are the rules of RFC 3986, section 5. A link that is not a valid reference, such as one with a broken % escape, makes Join throw, so a crawler should catch the error and skip that link.

Guide: is it the same site?

To allow redirects only within your own site, or to group links by owner, compare the registrable domain instead of the host name. Domain is the part a person registers, such as example.co.uk; PublicSuffix is the part under which anyone can register; Subdomain is what is left on the left. An address made of an IP has no domain, so compare the host name there. Host names are not case sensitive but the package keeps their case, so the program lowercases both addresses before parsing them.

encoding-url-guide-site.salam
import encoding.url
import str

func same_site(a: str, b: str): bool throws:
    ua := try url.Parse(str.Lower(a))
    ub := try url.Parse(str.Lower(b))
    if url.IsIP(ua) or url.IsIP(ub):
        ret url.Hostname(ua) == url.Hostname(ub)
    end
    ret url.Domain(ua) == url.Domain(ub)
end

func main throws:
    u := try url.Parse("https://api.eu.shop.example.co.uk:8443/v1/orders")
    println "host:", url.Hostname(u), "port:", u.port
    println "domain:", url.Domain(u), "suffix:", url.PublicSuffix(u), "sub:", url.Subdomain(u)

    println try same_site("https://www.example.com/a", "https://login.example.com/b")
    println try same_site("https://WWW.Example.com/", "https://example.com/cart")
    println try same_site("https://example.com/", "https://example.org/")
    println try same_site("http://10.0.0.5:8080/", "http://10.0.0.5/")
end
Outputhost: api.eu.shop.example.co.uk port: 8443 domain: example.co.uk suffix: co.uk sub: api.eu.shop true true false true

The package knows the common public suffixes such as com, co.uk and com.au, not the complete Public Suffix List, so hosting platforms that give every customer a subdomain may count as one site. Check those hosts by name when it matters.

Guide: build a URL from user input

When parts of the address come from users, escape each path segment with EncodePathSegment, which also escapes / so a file name cannot add folders. Put the parameters in a QueryMap and attach it with SetQuery; Add allows the same key more than once.

encoding-url-guide-build.salam
import encoding.url
import encoding.query

func main throws:
    folder := "My Files"
    name := "report 2026/Q3.pdf"
    mut u := try url.Parse("https://files.example.com")
    u.path = "/users/sara/" + url.EncodePathSegment(folder) + "/" + url.EncodePathSegment(name)

    mut q := query.NewQueryMap()
    query.Set(q, "download", "1")
    query.Add(q, "tag", "finance")
    query.Add(q, "tag", "سه‌ماهه")
    url.SetQuery(u, q)

    full := url.String(u)
    println full

    back := try url.Parse(full)
    println "path:", try url.DecodedPath(back)
    println "tags:", query.GetAll(back.raw_query, "tag")
end
Outputhttps://files.example.com/users/sara/My%20Files/report%202026%2FQ3.pdf?download=1&tag=finance&tag=%D8%B3%D9%87%E2%80%8C%D9%85%D8%A7%D9%87%D9%87 path: /users/sara/My Files/report 2026/Q3.pdf tags: ["finance", "سه‌ماهه"]

DecodedPath gives the readable path but turns %2F back into /, so it cannot tell a slash inside a name from a folder boundary. Split u.path at / first and decode each segment when you need the names exactly.

Types

URL

struct URL
A parsed web address.
NameTypeDescription
schemestr = ""The scheme, such as https.
userinfostr = ""The user name and password before the @, still encoded.
hoststr = ""The host name or numeric address.
portint = 0The port, or zero when none was given.
pathstr = ""The path, still encoded.
raw_querystr = ""The query after the ?, still encoded.
fragmentstr = ""The part after the #.
encoding-url-Parse.salam
import encoding.url
import encoding.query

func main throws:
    mut u := try url.Parse("https://user%20a:pw@shop.example.co.uk:8443/books/%D8%B3%D9%84%D8%A7%D9%85?page=2&sort=new#top")
    println u.scheme, u.host, u.port, u.path, u.raw_query, u.fragment
    println url.Hostname(u), url.HostPort(u), url.IsIP(u)
    println url.Labels(u), url.PublicSuffix(u), url.Domain(u), url.Subdomain(u)
    println try url.DecodedPath(u), try url.DecodedUserinfo(u)
    println try url.ParamNames(u), query.Size(try url.Query(u))
    try url.SetParam(u, "page", "3")
    println try url.RemoveParam(u, "sort"), url.String(u)
    mut keep := Vector {} as Vector<str>
    keep.Push("page")
    try url.KeepParams(u, keep)
    mut q := query.NewQueryMap()
    query.Add(q, "q", "hello world")
    url.SetQuery(u, q)
    println url.String(u)
    url.ClearQuery(u)
    println url.String(u)
    println url.String(try url.Join(u, "../img/a.png")), url.EncodePathSegment("a b/c")
    println url.IsIP(try url.Parse("http://127.0.0.1/"))
    try:
        _ := url.Parse("https://shop.example.com:80a/")
    catch e:
        println e.kind_name(), e.ToStr()
    end
end
Outputhttps shop.example.co.uk 8443 /books/%D8%B3%D9%84%D8%A7%D9%85 page=2&sort=new top shop.example.co.uk shop.example.co.uk:8443 false ["shop", "example", "co", "uk"] co.uk example.co.uk shop /books/سلام user a:pw ["page", "sort"] 2 true https://user%20a:pw@shop.example.co.uk:8443/books/%D8%B3%D9%84%D8%A7%D9%85?page=3#top https://user%20a:pw@shop.example.co.uk:8443/books/%D8%B3%D9%84%D8%A7%D9%85?q=hello+world#top https://user%20a:pw@shop.example.co.uk:8443/books/%D8%B3%D9%84%D8%A7%D9%85#top https://user%20a:pw@shop.example.co.uk:8443/img/a.png a%20b%2Fc true syntax error url: invalid character 'a' in port (line 1, column 28)

Functions

Hostname

func Hostname(u: URL): str
The host name without the port.

The example for this item is shown with the ones above: encoding-url-Parse.salam

IsIP

func IsIP(u: URL): bool
True when the host is four dot-separated groups of digits or contains : (an IPv6 address). The digits are not range checked.

The example for this item is shown with the ones above: encoding-url-Parse.salam

Labels

func Labels(u: URL): Vector<str>
The parts of the host name, separated by dots.

The example for this item is shown with the ones above: encoding-url-Parse.salam

PublicSuffix

func PublicSuffix(u: URL): str
The public suffix of the domain, such as co.uk or ir.

The example for this item is shown with the ones above: encoding-url-Parse.salam

Domain

func Domain(u: URL): str
The registered domain: one label before the public suffix, such as example.co.uk. An IP address or a one-label host such as localhost is returned unchanged. Case is kept.

The example for this item is shown with the ones above: encoding-url-Parse.salam

Subdomain

func Subdomain(u: URL): str
Whatever comes before the registered domain, such as shop.

The example for this item is shown with the ones above: encoding-url-Parse.salam

RemoveParam

func RemoveParam(u &: URL, key: str): Throws<bool>
Removes every value of a parameter and says whether it was there. Like the other parameter functions it re-encodes the rest of the query, and throws when the query has a broken % escape.

The example for this item is shown with the ones above: encoding-url-Parse.salam

SetParam

func SetParam(u &: URL, key: str, val: str): Throws<bool>
Sets a parameter to one value, replacing any values it had; a new name goes at the end. Throws when the query has a broken escape.

The example for this item is shown with the ones above: encoding-url-Parse.salam

KeepParams

func KeepParams(u &: URL, keep: Vector<str>): Throws<bool>
Keeps only the parameters you name, for stripping tracking parameters. Throws when the query has a broken escape.

The example for this item is shown with the ones above: encoding-url-Parse.salam

ClearQuery

func ClearQuery(u &: URL)
Removes the whole query.

The example for this item is shown with the ones above: encoding-url-Parse.salam

ParamNames

func ParamNames(u: URL): Throws<Vector<str>>
The names of the parameters, in order. Throws when the query has a broken escape.

The example for this item is shown with the ones above: encoding-url-Parse.salam

Parse

func Parse(raw: str): Throws<URL>
Takes an address apart into its pieces. Only text with :// (or a leading //) gets a scheme and host; anything else is a relative reference kept in path. Throws an Error with the column when the text is not a URL: empty, a space or control character, a bad scheme, a missing host, a bad host character, a port that is not a number or above 65535, a broken IPv6 address or a bad % escape.

The example for this item is shown with the ones above: encoding-url-Parse.salam

HostPort

func HostPort(u: URL): str
The host and port together, such as shop.example.co.uk:8443.

The example for this item is shown with the ones above: encoding-url-Parse.salam

String

func String(u: URL): str
Turns the address back into text.

The example for this item is shown with the ones above: encoding-url-Parse.salam

DecodedPath

func DecodedPath(u: URL): Throws<str>
The path with its percent encoding undone. Throws on a % not followed by two hex digits.

The example for this item is shown with the ones above: encoding-url-Parse.salam

DecodedUserinfo

func DecodedUserinfo(u: URL): Throws<str>
The user information, decoded. Throws on a broken % escape.

The example for this item is shown with the ones above: encoding-url-Parse.salam

EncodePathSegment

func EncodePathSegment(s: str): str
Encodes one path segment; / is encoded too.

The example for this item is shown with the ones above: encoding-url-Parse.salam

Query

func Query(u: URL): Throws<query.QueryMap>
Turns the address's query into a QueryMap. Throws on a broken % escape, like query.Decode.

The example for this item is shown with the ones above: encoding-url-Parse.salam

SetQuery

func SetQuery(u &: URL, q: query.QueryMap)
Replaces the address's query with the given map.

The example for this item is shown with the ones above: encoding-url-Parse.salam

Join

func Join(base: URL, ref: str): Throws<URL>
Resolves a relative address against a base one, the way a browser does; .. and . in the path are worked out, and a reference of only ?query or #fragment keeps the base path. Throws when the reference is not a valid URL.

The example for this item is shown with the ones above: encoding-url-Parse.salam