encoding.url
Taking a web address apart, building and editing its parameters, and resolving relative addresses.
import encoding.url
This package works with whole web addresses. Parse splits a URL such as https://user@shop.example.co.uk:8443/path?page=2#top into a URL struct with the fields scheme, userinfo, host, port, path, raw_query and fragment, and String puts it back together. Around that come helpers to edit query parameters, resolve relative links with Join, and find the registrable domain of a host.
Use it whenever you have a full address. For the a=1&b=2 part on its own, such as a form body, use encoding.query; for percent-encoding a single string, encoding.URLEncode in encoding; to fetch the address, net.http.
Raw and decoded parts
The fields hold the text exactly as it appeared, percent escapes included, so String(Parse(x)) normally gives back x. Read the human form with DecodedPath, DecodedUserinfo or Query. When you set a field yourself, put encoded text in it: escape path segments with EncodePathSegment and build queries with SetQuery.
What trips people up
Parsethrows on text that is not a URL. It is declaredthrows, so call it withtry. It refuses an empty string, a space or control character, a scheme that is not a letter followed by letters, digits,+,-or., a missing host after://(onlyfile:///may have none), a host with characters such as<, a port that is not a number or is above 65535, an IPv6 address without its], and a%not followed by two hex digits anywhere; the Error has the kind (Syntax,UnexpectedEnd,OutOfRange,InvalidArgument) and the column. A relative reference such as../a.htmlor?page=2is fine, and text without://(includingmailto:sara@example.com) still has no scheme or host and lands whole inpath, so checkschemeandhostbefore trusting an address from a user.- Nothing is lowercased.
HTTPS://Shop.Example.COMkeeps its case inschemeandhost, andDomainreturnsExample.COM. Lowercase withstr.Lowerbefore you compare hosts or domains. - Editing one parameter rewrites the whole query.
RemoveParam,SetParamandKeepParamsdecode the query and encode it again:%20becomes+, characters such as*get escaped, a bareflagbecomesflag=, and repeated keys are grouped together. The meaning stays the same; the text may not. - Everything that decodes throws too.
Joinparses its reference, andDecodedPath,DecodedUserinfo,Query,ParamNames,SetParam,RemoveParamandKeepParamsdecode a field. An address fromParsealways decodes, but a field you set by hand may not.
Guide: strip tracking parameters
Links copied from email and social media carry utm_ and click id parameters that only serve analytics. Before you store or compare links, parse them, drop those parameters with RemoveParam, clear the fragment and print the result with String. The parameters that stay keep their order.
import encoding.url
func clean(raw: str): str throws:
mut u := try url.Parse(raw)
names := try url.ParamNames(u)
each n in names:
if n.StartsWith("utm_") or n == "fbclid" or n == "gclid":
_ := try url.RemoveParam(u, n)
end
end
u.fragment = ""
ret url.String(u)
end
func main:
links := ["https://shop.example/item/42?color=red&utm_source=news&utm_medium=email#reviews", "https://blog.example/post?fbclid=IwAR0abc", "https://example.com/search?q=salam%20lang&utm_campaign=fall&page=2&debug", "https://example.com/sale?off=50%"]
each link in links:
try:
println clean(link)
catch e:
println "skipped:", e.ToStr()
end
end
endParamNames returns a copy of the names, so removing parameters inside the loop is safe. The last link has a bare %, which is not a valid URL, so Parse throws and the loop skips it with the reason. The fields of URL are plain strings you can set directly, as u.fragment = "" does. The third line shows the rewrite described above: salam%20lang comes out as salam+lang and the bare debug as debug=. Both mean the same to a server, but if you use the cleaned link as a cache or database key, clean every link the same way so equal links give equal text.
Guide: resolve the links of a page
A crawler or a link checker finds relative links like ../api/json.html in a page and needs the full address. Join resolves a reference against the page's own URL the way a browser does: ./ and ../ move through the folders, a leading / starts at the site root, //host keeps the scheme, a full URL replaces everything, and a reference of only ?query or #fragment keeps the page itself.
import encoding.url
func main throws:
page := try url.Parse("https://docs.example.com/guide/start/index.html?lang=en")
hrefs := ["install.html", "./setup/linux.html", "../api/json.html", "/blog/", "https://other.example/x", "//cdn.example.com/app.js", "?lang=fa", "#install"]
each h in hrefs:
println h, "->", url.String(try url.Join(page, h))
end
endThese are the rules of RFC 3986, section 5. A link that is not a valid reference, such as one with a broken % escape, makes Join throw, so a crawler should catch the error and skip that link.
Guide: is it the same site?
To allow redirects only within your own site, or to group links by owner, compare the registrable domain instead of the host name. Domain is the part a person registers, such as example.co.uk; PublicSuffix is the part under which anyone can register; Subdomain is what is left on the left. An address made of an IP has no domain, so compare the host name there. Host names are not case sensitive but the package keeps their case, so the program lowercases both addresses before parsing them.
import encoding.url
import str
func same_site(a: str, b: str): bool throws:
ua := try url.Parse(str.Lower(a))
ub := try url.Parse(str.Lower(b))
if url.IsIP(ua) or url.IsIP(ub):
ret url.Hostname(ua) == url.Hostname(ub)
end
ret url.Domain(ua) == url.Domain(ub)
end
func main throws:
u := try url.Parse("https://api.eu.shop.example.co.uk:8443/v1/orders")
println "host:", url.Hostname(u), "port:", u.port
println "domain:", url.Domain(u), "suffix:", url.PublicSuffix(u), "sub:", url.Subdomain(u)
println try same_site("https://www.example.com/a", "https://login.example.com/b")
println try same_site("https://WWW.Example.com/", "https://example.com/cart")
println try same_site("https://example.com/", "https://example.org/")
println try same_site("http://10.0.0.5:8080/", "http://10.0.0.5/")
endThe package knows the common public suffixes such as com, co.uk and com.au, not the complete Public Suffix List, so hosting platforms that give every customer a subdomain may count as one site. Check those hosts by name when it matters.
Guide: build a URL from user input
When parts of the address come from users, escape each path segment with EncodePathSegment, which also escapes / so a file name cannot add folders. Put the parameters in a QueryMap and attach it with SetQuery; Add allows the same key more than once.
import encoding.url
import encoding.query
func main throws:
folder := "My Files"
name := "report 2026/Q3.pdf"
mut u := try url.Parse("https://files.example.com")
u.path = "/users/sara/" + url.EncodePathSegment(folder) + "/" + url.EncodePathSegment(name)
mut q := query.NewQueryMap()
query.Set(q, "download", "1")
query.Add(q, "tag", "finance")
query.Add(q, "tag", "سهماهه")
url.SetQuery(u, q)
full := url.String(u)
println full
back := try url.Parse(full)
println "path:", try url.DecodedPath(back)
println "tags:", query.GetAll(back.raw_query, "tag")
endDecodedPath gives the readable path but turns %2F back into /, so it cannot tell a slash inside a name from a folder boundary. Split u.path at / first and decode each segment when you need the names exactly.
Types
URL
struct URL| Name | Type | Description |
|---|---|---|
scheme | str = "" | The scheme, such as https. |
userinfo | str = "" | The user name and password before the @, still encoded. |
host | str = "" | The host name or numeric address. |
port | int = 0 | The port, or zero when none was given. |
path | str = "" | The path, still encoded. |
raw_query | str = "" | The query after the ?, still encoded. |
fragment | str = "" | The part after the #. |
import encoding.url
import encoding.query
func main throws:
mut u := try url.Parse("https://user%20a:pw@shop.example.co.uk:8443/books/%D8%B3%D9%84%D8%A7%D9%85?page=2&sort=new#top")
println u.scheme, u.host, u.port, u.path, u.raw_query, u.fragment
println url.Hostname(u), url.HostPort(u), url.IsIP(u)
println url.Labels(u), url.PublicSuffix(u), url.Domain(u), url.Subdomain(u)
println try url.DecodedPath(u), try url.DecodedUserinfo(u)
println try url.ParamNames(u), query.Size(try url.Query(u))
try url.SetParam(u, "page", "3")
println try url.RemoveParam(u, "sort"), url.String(u)
mut keep := Vector {} as Vector<str>
keep.Push("page")
try url.KeepParams(u, keep)
mut q := query.NewQueryMap()
query.Add(q, "q", "hello world")
url.SetQuery(u, q)
println url.String(u)
url.ClearQuery(u)
println url.String(u)
println url.String(try url.Join(u, "../img/a.png")), url.EncodePathSegment("a b/c")
println url.IsIP(try url.Parse("http://127.0.0.1/"))
try:
_ := url.Parse("https://shop.example.com:80a/")
catch e:
println e.kind_name(), e.ToStr()
end
endFunctions
Hostname
func Hostname(u: URL): strThe example for this item is shown with the ones above: encoding-url-Parse.salam
IsIP
func IsIP(u: URL): bool: (an IPv6 address). The digits are not range checked.The example for this item is shown with the ones above: encoding-url-Parse.salam
Labels
func Labels(u: URL): Vector<str>The example for this item is shown with the ones above: encoding-url-Parse.salam
PublicSuffix
func PublicSuffix(u: URL): strco.uk or ir.The example for this item is shown with the ones above: encoding-url-Parse.salam
Domain
func Domain(u: URL): strexample.co.uk. An IP address or a one-label host such as localhost is returned unchanged. Case is kept.The example for this item is shown with the ones above: encoding-url-Parse.salam
Subdomain
func Subdomain(u: URL): strshop.The example for this item is shown with the ones above: encoding-url-Parse.salam
RemoveParam
func RemoveParam(u &: URL, key: str): Throws<bool>% escape.The example for this item is shown with the ones above: encoding-url-Parse.salam
SetParam
func SetParam(u &: URL, key: str, val: str): Throws<bool>The example for this item is shown with the ones above: encoding-url-Parse.salam
KeepParams
func KeepParams(u &: URL, keep: Vector<str>): Throws<bool>The example for this item is shown with the ones above: encoding-url-Parse.salam
ClearQuery
func ClearQuery(u &: URL)The example for this item is shown with the ones above: encoding-url-Parse.salam
ParamNames
func ParamNames(u: URL): Throws<Vector<str>>The example for this item is shown with the ones above: encoding-url-Parse.salam
Parse
func Parse(raw: str): Throws<URL>:// (or a leading //) gets a scheme and host; anything else is a relative reference kept in path. Throws an Error with the column when the text is not a URL: empty, a space or control character, a bad scheme, a missing host, a bad host character, a port that is not a number or above 65535, a broken IPv6 address or a bad % escape.The example for this item is shown with the ones above: encoding-url-Parse.salam
HostPort
func HostPort(u: URL): strshop.example.co.uk:8443.The example for this item is shown with the ones above: encoding-url-Parse.salam
String
func String(u: URL): strThe example for this item is shown with the ones above: encoding-url-Parse.salam
DecodedPath
func DecodedPath(u: URL): Throws<str>% not followed by two hex digits.The example for this item is shown with the ones above: encoding-url-Parse.salam
DecodedUserinfo
func DecodedUserinfo(u: URL): Throws<str>% escape.The example for this item is shown with the ones above: encoding-url-Parse.salam
EncodePathSegment
func EncodePathSegment(s: str): str/ is encoded too.The example for this item is shown with the ones above: encoding-url-Parse.salam
Query
func Query(u: URL): Throws<query.QueryMap>QueryMap. Throws on a broken % escape, like query.Decode.The example for this item is shown with the ones above: encoding-url-Parse.salam
SetQuery
func SetQuery(u &: URL, q: query.QueryMap)The example for this item is shown with the ones above: encoding-url-Parse.salam
Join
func Join(base: URL, ref: str): Throws<URL>.. and . in the path are worked out, and a reference of only ?query or #fragment keeps the base path. Throws when the reference is not a valid URL.The example for this item is shown with the ones above: encoding-url-Parse.salam