The other day at work, one of my colleagues was frustrated that he was unable to encode nested objects in a query string and still maintain a readable URL. I went home that night and coded up a simple solution to this problem, and I thought I’d share it here today. Here is a Github repo with specs and the solution code.
Today, in the Node.js ecosystem, numerous modules exist to encode query strings, but they generally have one of two flaws:
They do not permit the encoding of nested objects.
They can encode nested objects, but they delimit nesting using unsafe URL characters, yielding an operation and a result that look like this 1:
Node.js provides a
querystring module to encode objects to query strings. The only problem is that conforms to an official specification that doesn’t allow nested objects. Unfortunately, this specification does not allow for enough flexibility when creating a RESTful API.
For example, suppose the client wants to filter a collection of cars by make and model. The route might look like this:
This URI makes it reasonably clear that we want to filter cars by their make and model.
What if we wanted to do something more complicated. What if we wanted to filter cars and order them by price?
It’s no longer clear which query parameters describe the ordering and which describe the filter. Ideally, we want the url to look like this:
And then we quickly run into our problem. We need to encode the object above into
querystring can’t encode nested objects.
By default, the qs module creates ugly urls when it encodes nested query strings. If we encode our object above, we get
] characters are both considered unsafe in a URL and are required to be escaped. The URL becomes unreadable after this percent encoding operation.
. is not considered unsafe and does not need to be escaped, making it the perfect character to express object nesting.
The solution is broken down into two parts. The first is encoding a nested object into a query string. The second part is decoding a query string back into a nested object.
Let’s write some code to encode
into the query string
Notice that we use the escape function provided in Node.js core to percent encode specific characters.
If we want to add support to encode an object with array values, like the following:
then we only need to add another base case to our function
An encoding function is not very useful unless you can decode the encoded string back to it’s original form.
We want to write a function that will decode
filter.make=honda&filter.model=civic back into a nested object
If we want to add support to decode arrays like we did above, then we need to do a little additional work. Fortunately, two additional Lodash utilities, has and get, allow us to check for the existence of a nested key and to get the value associated with a nested key, respectively, greatly simplifying our problem.
And that’s it! The whole thing, encoding and decoding, only takes ~40 lines of code. Perhaps next time you encounter something that feels a little too fundamental to code yourself, you won’t hesitate to write some code if you can’t find a sufficient open source package.
2: It’s worth noting that you might not want to use this code in production. I’ve written the code in a functional style for clarity and conciseness. If you have a high read volume, given that this code might potentially run on a significant portion of GET requests, it should probably be written in an imperative style that doesn’t disregard performance. Even more importantly, this code does not protect against potential attackers who might try to create an arbitrarily deeply nested object or might include an unwieldy number of query parameters.