🐛 Bug Report

Current behavior

• Page link: https://amp.dev/page-experience/
• (Optional) Screenshots/videos:
  

Tested this URL. https://news.yahoo.co.jp/amp/pickup/6374389

Given URL is a valid AMP page, but the page audit doesn't run and shows "This URL is invalid." on AMP Page Experience Guide page. It's difficult to understand what's going on and how to fix it from this error message.

I looked into the network and noticed the lint API only returns {"status": "error"} without any other messages.
https://amp.dev/page-experience/api/lint?url=https%3A%2F%2Fnews.yahoo.co.jp%2Famp%2Fpickup%2F6374389

I checked the API code and realized this happens because the given AMP page doesn't have 'Content-Type' header in the response, and API doesn't care if the header exists or not, and makes type error. This is one problem I think.

[22:41:38] [Pixi API] › ✖  error     Unable to lint https://news.yahoo.co.jp/amp/pickup/6374389 TypeError: Cannot read property 'includes' of null
    at RateLimitedFetch.fetchHtmlResponse (/Users/sisidovski/Projects/amp.dev/platform/lib/utils/rateLimitedFetch.js:120:22)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async execChecks (/Users/sisidovski/Projects/amp.dev/pixi/backend/api.js:58:15)
    at async /Users/sisidovski/Projects/amp.dev/pixi/backend/api.js:100:25

Another problem that I think more important is, there is no way to let users know what the cause is. If above lint API is fixed, then the API response for the given URL would be like this.

{
  "status": "error",
  "errorId": "UNSUPPORTED_CONTENT_TYPE",
  "message": "https://news.yahoo.co.jp/amp/pickup/6374389 is no HTML document."
}

This is good if users can see those error messages as it is at the pixi frontend, they can understand the problem and fix their pages. However, the error messages from the lint API are thrown away and treated as very general "invalid-url" status id finally.

Expected behavior/code

1. The lint API should consider if the Content-Type header exists in a response
2. Show more detailed error message to users. In this case, "This URL is unsupported content type" or something.

Reproduction procedure

1. Go to https://amp.dev/page-experience/
2. Type https://news.yahoo.co.jp/amp/pickup/6374389 to the form
3. Submit

• What browser are you using?
  Chrome
• What O/S are you using?
  Mac OS