2

When using URLComponents's queryItems I've found that if you have a query item whose value contains some percent encoded characters, in my case a / being encoded as %2F, then if you construct a URLComponents object from a String URL that contains such a query item, then mutate the list of query items for the URLComponents object, then if you try to get a URL by calling .url on the URLComponents object, then the query items lose their percent encoding.

Here's the code I've been testing this with in a playground:

import UIKit

// --- Part 1 ---
print("--- Part 1 ---\n")

let startURL = "https://test.com/test.jpg?X-Test-Token=FQdzEPH%2F%2F%2F"
var components = URLComponents(string: startURL)!

if let compURL = components.url {
    print(URL(string: startURL)! == compURL) // True
    print(startURL)
    print(compURL)
}

// --- Part 2 ---
print("\n--- Part 2 ---\n")

let startURLTwo = "https://test.com/test.jpg?X-Test-Token=FQdzEPH%2F%2F%2F"
let finalURL = "https://test.com/test.jpg?X-Test-Token=FQdzEPH%2F%2F%2F&foo=bar"
var componentsTwo = URLComponents(string: startURLTwo)!

let extraQueryItem = URLQueryItem(name: "foo", value: "bar")
componentsTwo.queryItems!.append(extraQueryItem)

if let compURLTwo = componentsTwo.url {
    print(URL(string: finalURL)! == compURLTwo) // False
    print(finalURL)
    print(compURLTwo)
}

Here's an image if that makes it easier to understand what's going on:

enter image description here

hamchapman
  • 1,675
  • 2
  • 18
  • 36

2 Answers2

2

You should use percentEncodedQuery if you have a query that is already percent encoded:

let startURL = "https://test.com/test.jpg"
var components = URLComponents(string: startURL)!
components.percentEncodedQuery = "X-Test-Token=FQdzEPH%2F%2F%2F"

if let compURL = components.url {
    print(compURL)
}

Or you can specify it unescaped (and it leaves it unescaped as it's not necessary to escape / characters in a query):

let startURL = "https://test.com/test.jpg"
var components = URLComponents(string: startURL)!
components.queryItems = [URLQueryItem(name: "X-Test-Token", value: "FQdzEPH///")]

if let compURL = components.url {
    print(compURL)
}

And if you have to update queryItems, just make sure to set percentEncodedQuery at the very end:

let startURL = "https://test.com/test.jpg"
let encodedQuery = "X-Test-Token=FQdzEPH%2F%2F%2F"
var components = URLComponents(string: startURL)!
components.queryItems = [URLQueryItem(name: "foo", value: "bar, baz, & qux")]
if let query = components.percentEncodedQuery {
    components.percentEncodedQuery = query + "&" + encodedQuery
} else {
    components.percentEncodedQuery = encodedQuery
}

if let compURL = components.url {
    print(compURL)
}
Rob
  • 371,891
  • 67
  • 713
  • 902
  • I'm going with something like that last part of your answer. It works fine but it doesn't feel right. It feels wrong to me that the question of whether or not the query string will end up being left percent encoded for given characters is determined by whether or not the `queryItems` property is altered at all after instantiation. But hey, at least I know now. Thanks for the answer. – hamchapman Dec 22 '17 at 11:39
1

RFC 3986 specifically states that a URL query may contain the / character. It doesn't need to be percent encoded. URLComponents is simply following the standard and unencoding the %2F to / when you specifically modify any of the query parameters.

In the first case you don't modify anything at all so the URL stays unchanged. In the 2nd, you modify the query parameters property of the components. So the URLComponents builds a new query string from that updated array of query parameters. In the process, if normalizes all of them and the unnecessary percent encoding is removed.

rmaddy
  • 298,130
  • 40
  • 468
  • 517
  • I'm not modifying any of the query parameters. I'm adding an extra one to the `queryItems` array. Why should doing that lead to `URLComponents` decoding the `%2F` when it doesn't in the first case? – hamchapman Dec 21 '17 at 20:48
  • @hamchapman why do you need the percent encoding there? If you remove it prior to composing your url it will be consistent – Leo Dabus Dec 21 '17 at 20:50
  • I don't need it specifically, I just need the resulting `URL` to be consistent regardless of whether or not I've added any `URLQueryItem`s to the `queryItems` property of the `URLComponents` object – hamchapman Dec 21 '17 at 20:51
  • @hamchapman In the first case you don't modify anything at all so the URL stays unchanged. In the 2nd, you modify the query parameters property of the components. So the `URLComponents` builds a new query string from that updated array of query parameters. In the process, if normalizes all of them and the unnecessary percent encoding is removed. – rmaddy Dec 21 '17 at 20:52
  • Just do what I suggested. remove the percent encoding before composing it – Leo Dabus Dec 21 '17 at 20:52
  • I can't just remove the percent encoding. The request I'm making is currently failing because of this inconsistent behaviour. For context, I'm getting a presigned URL from AWS's S3 which provides the URL with the percent encoded `/`s (https://github.com/aws/aws-sdk-go/blob/43508f070a5e9ad20bbc81983d2cd17ff3c0a84a/aws/request/request.go#L269 which is calling this: https://golang.org/pkg/net/url/#URL.String). I'm not sure why this would escape the query string part though. Either way, I need to have the escaped version for my use case, so maybe I'll just escape it myself – hamchapman Dec 21 '17 at 21:11
  • If you need the `/` to be percent escaped then don't use `URLComponents`. Use `addingPercentEncoding` from `NSString`. – rmaddy Dec 21 '17 at 21:15
  • @hamchapman - "I can't just remove the percent encoding." ... I'm not sure about that. Just because it gave it to you percent encoding characters that didn't need it, doesn't mean you can't allow `URLComponents` to properly percent escape it. When you send a query with `%2F` in it, the server is going to just replace it with `/` anyway, so it invariably doesn't matter if you send `/` or `%2F` in the query. This feels like an XY problem. – Rob Dec 21 '17 at 23:59