Skip to content

Prevent duplicated/conflicting HTTP Headers for HurlStack#139

Closed
houtianze wants to merge 0 commit intogoogle:masterfrom
houtianze:master
Closed

Prevent duplicated/conflicting HTTP Headers for HurlStack#139
houtianze wants to merge 0 commit intogoogle:masterfrom
houtianze:master

Conversation

@houtianze
Copy link
Copy Markdown

Summary

It seems a subtle bug to me that HurlStack will allow duplicated/conflicting HTTP Headers (specifically, the "Content-Type" header) from being added, this PR eliminate this possibility.

Illustration

The following code will generate HTTP request that have conflicting "Content-Type" headers using the current version of Volley, which will result an HTTP 400 response from the server most of the time:

public class PostRequest extends Request<String> {

  @Override
  public String getBodyContentType() {
    return "text/plain";
  }

  @Override
  public Map<String, String> getHeaders() throws AuthFailureError {
    Map<String, String> headers = super.getHeaders();
    headers.put("Content-Type", "application/json");
    return headers;
  }

  // other codes ...
}

Assumptions

I assume that Volley intends to support unique HTTP Headers only in requests, because the return type of getHeaders() is Map<String, String> instead of (the more generic) Map<String, List<String>>.

Justifications

  • With the assumptions mentioned, the "normal scenarios" will still work as before, but this change prevents users from accidentally creating duplicated/conflicting HTTP Headers that will almost certainly fail.
  • If we check the code of (now deprecated) HttpClientStack, it does't have this problem (it in the end calls setHeaders() instead of addHeader() in performRequest()), which is the same behaviour as this change applied to HurlStack.
  • Adding "Content-Type" header is a common task for programmers, so this can potentially reduce some troubleshooting efforts for those who unknowingly tripped this error.

Background

I encountered this issue while using the swagger generated Android client (I believe this issue swagger-api/swagger-codegen#5575 describes exactly what I faced).

@jpd236 jpd236 added this to the 1.1.1 milestone Jan 9, 2018
Copy link
Copy Markdown
Collaborator

@jpd236 jpd236 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that this is a (subtle) issue - thanks for the detailed analysis. In fact I think it does actually affect HttpClientStack too which uses addHeader here:

https://github.com/google/volley/blob/master/src/main/java/com/android/volley/toolbox/HttpClientStack.java#L123

which makes me wish we had a stack-agnostic solution for this. I also want to fail more loudly if possible in this scenario rather than silently ignore the user's header so they actually are aware if something goes wrong.

Clients returning If-None-Match and If-Modified-Since headers in getHeaders will also be affected, as these are added in addition to provided headers:

https://github.com/google/volley/blob/master/src/main/java/com/android/volley/toolbox/BasicNetwork.java#L242

Given this, my current line of thinking is that there should be a method to get and validate the headers returned by Request#getHeaders() which detects "bad" headers and can log warnings about how they should be set instead. Then this can be called from HurlStack/HttpClientStack instead of getHeaders() directly. It won't fix other stacks in the wild, unfortunately, but I don't think we can easily do anything about that without changing APIs.

Beyond that, I'd be open to either fixing both stacks as you've done with HurlStack, or else having the get-and-validate method copy the returned map and filter out the bad keys so the stacks don't need to change except to call the validate method instead of getHeaders() directly.

HttpURLConnection connection = openConnection(parsedUrl, request);
for (String headerName : map.keySet()) {
connection.addRequestProperty(headerName, map.get(headerName));
connection.setRequestProperty(headerName, map.get(headerName));
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, technically this one seems like a no-op, because it's a brand new connection and so the connection will presumably not have any properties set. But it's fine to change. Only the one below where the content-type property is set after this is a problem. But it's fine to change for consistency.

@houtianze
Copy link
Copy Markdown
Author

houtianze commented Jan 9, 2018

Thanks for the detailed reply. Agree that failing/warning loud would be a better approach, but more code changes are needed for sufficient validations.

I actually thought about checking for "Content-Type" header before merging in getHeaders() results, but it a bit hacky.

For https://github.com/google/volley/blob/master/src/main/java/com/android/volley/toolbox/HttpClientStack.java#L123 , I guess it won't have this issue, as though this line calls addHeader(), it's right after a new HttpPost being created, so this addHeader() shouldn't give problem there, and subsequent header handling call addHeaders() which indeed call setHeader() instead, so it won't result in duplicated headers. Maybe we can also consider change that addHeader() call to setHeader(), similiarly for consistency.

@jpd236
Copy link
Copy Markdown
Collaborator

jpd236 commented Jan 10, 2018

Good point re: HttpClientStack. In fact what's (IMO) broken is that the header from getHeaders will take precedence over the content-type header, whereas in HurlStack it's the other way around (after your CL at least). There may not be duplicates but a different one will be used depending on which stack you're using which certainly isn't intended.

@anuj-sahai
Copy link
Copy Markdown

Hi @jpd236, is there some timeline for this being merged to Volley master or 1.1.1 milestone? Or would it be when the remaining 13 open issues in 1.1.1 get fixed?

@jpd236
Copy link
Copy Markdown
Collaborator

jpd236 commented May 23, 2018

Sorry, I botched my attempt to amend to this pull request and can't seem to recover it easily. Reopened as #193.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants