We have a REST API endpoint that returns a bunch of external URLs for a resource
The data is of the format:
{
“resource_data”:{
“name”:“Dummny Name”,
“project_name”:“Sinstein”,
“logs”:“http://example.com/cdd921b88114158fd34ae54c2b828bb094dd128c/logs”,
“public_url”:“http://example.com/cdd921b88114158fd34ae54c2b828bb094dd128c?auth_token=dummy_token”,
“video_url”:“http://example.com/cdd921b88114158fd34ae54c2b828bb094dd128c/video-1b6575be8867bfda659b25789c216c268194b2f0.mp4?AWSAccessKeyId=dummyid\u0026Signature=dummysignature\u0026response-content-type=video%2Fmp4”,
}
}
The URLs that are a part of the data are external URLs that point to where the files are stored.
We generate a hash of this data in rails and return a JSON from the hash. This causes the URLs to be encoded. I can prevent the encoding using JSON::dump instead of relying on to_json.
The question that brings up is what is the best practice here? Should the URLs be encoded when being returned in the body of the response? I understand that allowing unescaped JSON data to pass around opens up the possibility of a security breach but in this case, the data is generated by us and does not depend on the user input.
Should we return unescaped valid URLs or let them be encoded when sending a response?
Saurav Kothari
The question that brings up is what is the best practice here? Should the URLs be encoded when being returned in the body of the response? I understand that allowing unescaped JSON data to pass around opens up the
possibility of a security breach but in this case, the data is generated by us and does not depend on the user input.
Should we return unescaped valid URLs or let them be encoded when sending a response?
What Bad Thing happens if you allow the URLs to be encoded?
Personally I tend to follow the path of least resistance. If the URLS “want to be encoded”, and there is no downside to that, I’d just go with it. Experience has taught me that fiddling with this stuff unnecessarily tends to lead to problems later.
Click here to view Company Information and Confidentiality Notice.
What Bad Thing happens if you allow the URLs to be encoded?
Since our API is used by our users directly, they cannot open the URLs directly from a cUrl response.
Personally I tend to follow the path of least resistance. If the URLS “want to be encoded”, and there is no downside to that, I’d just go with it. Experience
has taught me that fiddling with this stuff unnecessarily tends to lead to problems later.
I would agree with you, no serious downside to having them encoded.
But I have issues around the usability of the API in that case. Do you, as a user, expect to get a URL as a reponse that cannot be opened in a browser?
Do you, as a user, expect to get a URL as a reponse that cannot be opened in a browser?
I’m not sure I have a strong expectation either way. It depends on the nature of the data. I note that RSS feeds – which are basically a JSON API – do not encode URLS.
I think in your circumstances you could make a good argument for un-encoding them.
Click here to view Company Information and Confidentiality Notice.
Should we return unescaped valid URLs or let them be encoded when sending a
response?
Saurav Kothari
In JSON "&" and "\u0026" are equivalent. If you control the thing
receiving this JSON data you can (and should) unescape strings before
doing anything with them (processing, writing into HTML DOM, etc.)
regardless; and so should anybody else.
When it comes to the hash of the JSON document, it doesn't matter
whether it's escaped or not, as long as the same rule is applied
consistently. I assume the hash is opaque to anything but your
system, so it really doesn't matter which way you decide to do it.
Note that this is also true of other presentational concerns, like
whitespace.
Since our API is used by our users directly, they cannot open the URLs directly from a cUrl response.
[...]
But I have issues around the usability of the API in that case. Do you, as a user, expect to get a URL as a reponse that cannot be opened in a browser?
If the resource representation I receive is encoded as
application/json then I expect to have to read it as such, which means
unescaping and sanity-checking strings (among other things). If you
sent it as XML I'd expect to have to decode entities like & as
well. That's a different level of concern than the fact that your API
documentation says that the string encodes a URL.
If your users really can't deal with JSON, why are you giving them
JSON? Send them HTML, which comes with actual, well-defined linking
mechanisms built right in.
(I'd also suggest looking at RFC 8288 - Web Linking for
providing links between resources using web standards that don't
depend on particular API conventions or documentation.)
Cheers
···
On 27 February 2018 at 17:22, Saurav Kothari <sauravkothari2@gmail.com> wrote:
--
Matthew Kerwin
https://matthew.kerwin.net.au/