REST API Design: GET requires sensitive parameters that should not be stored in web server logs

Question

I am designing an (as-RESTful-as-possible) API and would like to know how you would best solve the following:

• Assume we are designing a TLS endpoint to retrieve some resource: GET /objects/{id}
• We don't want object {id}s to be stored in our web server logs, so we want to avoid using querystring or URI params; which leaves us with params in the request body. (assume the data is such that the id is sensitive and we don't have access to another non-sensitive id)
• I understand that it is recommended against having parameters in a GET request body. HTTP GET with request body
• I understand that using POST to get data is also recommended against as it leads more towards an RPC design style and may generally be confusing.

How can (should) we design the API GET endpoint to avoid using query or URI params that could be logged?
Is it acceptable to use POST in this scenario or is there another creative way?
(Note: this API will NOT be exposed to third-parties)

Answer 1

I think there are many services that are facing this problem of wanting to protect sensitive identifiers. However even though this question is some years old now, I didn't found a proper solution either.

The offered solution to simply alter the logging of your webserver isn't perfect as mentioned already, but also leaves out the fact that every client should to do the same while consuming your API (among which possibly JavaScript clients, via proxies, in browsers... good luck with that)

Solutions I am aware of are:

1. Encrypting the parameters; but this makes your API more complex, and it requires encryption keys.

2. use a pseudo-ID, as mentioned by @jj-geewax; however this is possibly even more complex than encyption (1) since you have to exchange a pseudo-ID for every sensitive parameter instance:

   • Client initiates with a pseudo-ID in its request to the server, the server than does a request to the client to resolve the psuedo-ID! (So the client should have an endpoint as well!)
   • Client POSTS the sensitive ID to server, for which it receives a pseudoID, which it can use in requests for this ID
   • Client and server have exchanged psuedo ID via some other means in advance

3. POST the parameters in the body, while requesting data; this is not REST

   • optionally use X-HTTP-Method-Override to make more explicit to the application you are actually requesting data.

Solution 3 seems by far the most simple/easiest method to implement, although it breaks with the REST design rules. But I am interested to hear alternative approaches or other insights.

UPDATE: OWASP says the following regarding sensitive parameters in requests

Sensitive information in HTTP requests

RESTful web services should be careful to prevent leaking credentials. Passwords, security tokens, and API keys should not appear in the URL, as this can be captured in web server logs, which makes them intrinsically valuable.

• In POST/PUT requests sensitive data should be transferred in the request body or request headers.
• In GET requests sensitive data should be transferred in an HTTP Header.

https://cheatsheetseries.owasp.org/cheatsheets/REST_Security_Cheat_Sheet.html#sensitive-information-in-http-requests

This is maybe a bit harder than using the (POST) body, but also a lot nice when looking at how to implement REST.

Answer 2

If you are using Apache as your webserver you can use CustomLog to remove/replace the sensitive values, this answer provides an example script.

Answer 3

In general, if you're sticking to REST, you should keep the identifier in the URL path (e.g., /objects/{id}) and stick to the GET HTTP method, but I can appreciate the issue you're struggling with. If this ID is secret in some way, then it's definitely a good idea to turn off logging of this magical secret ID. That said, the problem you're having might be indicative of a bigger design issue with your API and preventing logging might not fix the bigger problem.

For example, is this "security by obscurity" (e.g., the ID is a secret that grants access to data for anyone who knows it)? Or simply protecting sensitive information (e.g., PII like an ID by US Social Security Number)? In either of these cases, using this value as an identifier is probably a bad idea.

If this ID itself is sensitive, then it might be worth while to generate a random identifier for each resource and then start passing that around instead. Google Cloud does this with projects (IDs versus Numbers), as described in aip.dev/2510. Note how they specifically state that third-parties are unable to use the project ID and will always operate on an opaque project "number" as the identifier.

If this is security by obscurity, then you could require some sort of authentication / authorization token or header in the GET request, which should be fine from a RESTful perspective. This would mean it's OK for someone to get a hold of the secret identifier because it's useless without some other set of credentials.

I'd certainly caution against switching your HTTP method to POST just to avoid logging something sensitive. In the long run, this will cause confusion for new hires working on the project as well as prevent you from using any tooling that makes RESTful assumptions about an API.