Implementation of Real Time Config#9857
Implementation of Real Time Config#9857alanorozco merged 59 commits intoampproject:masterfrom google:frizz-real-time-config
Conversation
| <title>Ad examples</title> | ||
| <link rel="canonical" href="http://nonblocking.io/" > | ||
| <meta name="viewport" content="width=device-width,minimum-scale=1,initial-scale=1"> | ||
| <style amp-custom> |
There was a problem hiding this comment.
Do you really need all of this style and extensions?
| return ''; | ||
| } | ||
|
|
||
| const rtcConfig = tryParseJson( |
There was a problem hiding this comment.
This should be within getPageLevelParameters_. Also, correct me if I'm wrong but this would cause the RTC request to be sent for every slot instead of once per page. We should refactor the code so that getPageLevelParameters_ returns a cached value if already calculated which will include ensuring RTC is only sent once.
| ['108809080'])); | ||
| .then(p => { | ||
| pageLevelParameters = p; | ||
| return googlePageParameters(this.win, this.getAmpDoc(), startTime); |
There was a problem hiding this comment.
I'm confused, getPageLevelParameters_ calls googlePageParameters so why do you need to do it?
| /** @private {?Promise} */ | ||
| let sraRequests = null; | ||
|
|
||
| /** @private {?Promise} */ |
There was a problem hiding this comment.
Type here can be more precise (e.g. ?Promise<!Object<string,string>>)
| return ''; | ||
| } | ||
|
|
||
| const ampRtcPageElement = document.getElementById('amp-rtc'); |
There was a problem hiding this comment.
Do not worry about other networks, assume only doubleclick
| rtcConfig = tryParseJson(ampRtcPageElement.innerHTML); | ||
| if (rtcConfig && typeof rtcConfig == 'object' | ||
| && !!rtcConfig['doubleclick']) { | ||
| rtcRequestPromise = this.sendRtcRequestPromise_(/** @type {!Object} */( |
There was a problem hiding this comment.
I think this can just be sendRtcRequest_ (no need to include promise)
| googleAdUrl(this, DOUBLECLICK_BASE_URL, startTime, | ||
| Object.assign(this.getBlockParameters_(), pageLevelParameters), | ||
| ['108809080'])); | ||
| const pageLevelParametersPromise = getPageLevelParameters_( |
There was a problem hiding this comment.
RTC modifies block level parameters so you don't actually have to wait on page level before merging. You can move the merge directly into sendRTCRequest. This has the added benefit of allowing you to add other parameters (e.g. time to get RTC response) without putting that logic in getAdUrl.
Also I don't think how you have this will work as this.jsonTargeting_ is unique per block but you only have one promise that merges. So the flow actually needs to be that the RTC promise just returns back the RTC response and then you call merge on every block. Does that make sense?
| // Default is to send request without RTC, so only reject if | ||
| // pub explicitly says not to. | ||
| if (rtcConfig['doubleclick']['sendAdRequestOnFailure'] === false) { | ||
| return Promise.reject(); |
There was a problem hiding this comment.
There are 3 failure conditions of which any failure should prevent ad request if indicated by pub:
- network failure
- timeout
- malformed response
This is addressing the last item but I would argue that only malformation we care about is failure to JSON parse which we can handle as part of sendRTCRequest flow
| ['108809080'])); | ||
| const pageLevelParametersPromise = getPageLevelParameters_( | ||
| this.win, this.getAmpDoc(), startTime); | ||
| const blockParameters = this.getBlockParameters_(); |
There was a problem hiding this comment.
You don't actually need blockParameters for RTC. You only need populateAdUrlState to have set this.jsonTargeting. I suggest we structure as follows:
- getPageParams
- populateAdUrlState
- sendRtcRequest which modifies this.jsonTargeting_ and return an object mapping to be merged into the end url. sendRtcRequest contains ALL RTC logic (perhaps rename to executeRTC?) from merge to timeout to reject if pub indicates no ad request on failure
- getBlockParams (AFTER RTC completion)
- Object.assign block, RTC, page (note that order indicates order of url parameters!)
| } | ||
| const errorUrl = rtcDblckConfig['errorReportingUrl']; | ||
| const endpoint = rtcDblckConfig['endpoint']; | ||
| const noCache = (rtcDblckConfig['noCache'] === true); |
There was a problem hiding this comment.
Consider looking for other fields in the object to point out typos to pubs.
| const DOUBLECLICK_BASE_URL = | ||
| 'https://securepubads.g.doubleclick.net/gampad/ads'; | ||
|
|
||
| /** @const {Object} */ |
| BAD_ENDPOINT: 'Bad RTC Endpoint', | ||
| }; | ||
| /** @const {number} */ | ||
| const RTC_TIMEOUT = 1000; |
There was a problem hiding this comment.
Add MS to indicate its milliseconds
| dev().warn(TAG, `Frame already exists, sra: ${this.useSra}`); | ||
| return ''; | ||
| } | ||
|
|
There was a problem hiding this comment.
nit: remove extra, empty lines
| // TODO(keithwrightbos): SRA blocks currently unnecessarily generate full | ||
| // ad url. This could be optimized however non-SRA ad url is required to | ||
| // fallback to non-SRA if single block. | ||
| this.populateAdUrlState(); |
There was a problem hiding this comment.
You need to call populateAdUrlState before sendRtcRequest to ensure that this.jsonTargeting_ has been populated
| } | ||
|
|
||
| const ampRtcPageElement = document.getElementById('amp-rtc'); | ||
| let rtcConfig; |
There was a problem hiding this comment.
if (!ampRtcPageElement) return Promise.resolve();
let rtcConfig;
let endPoint;
// NOTE usage of textContent instead of innerHTML
if (!isObject(rtcConfig = tryParseJson(ampRtcPageElement.textContent)) || !(endPoint = rtcConfig['endPoint']) || typeof endPoint != 'string' || !isSecureUrl(endPoint)) {
user().warn(TAG, 'invalid RTC config...');
return Promise.resolve();
}
| /** | ||
| * Sends RTC request as specified by rtcConfig. Returns promise which | ||
| * resolves to the targeting informtation from the request response. | ||
| * If the publisher has specified that noCache == true, then we will only |
There was a problem hiding this comment.
Whether or not the response was cached should be controlled by the end point via cache headers. I think it makes more sense to have an option that disables state while revalidate behavior (e.g. staleRevalidateDisabled: true). The behavior is that the first request has no special cache headers and if SWR is not disabled, we send a second request with no-cache option
| const endpoint = rtcConfig['endpoint']; | ||
| const noCache = (rtcConfig['noCache'] === true); | ||
|
|
||
| if (!endpoint) { |
There was a problem hiding this comment.
Note my previous comment does not cancel the ad request if invalid config. I think this makes more sense as the pub clearly misconfigured and we should default to ensuring they get ad revenue
|
|
||
| return rtcPromise = timerFor(window).timeoutPromise( | ||
| RTC_TIMEOUT, rtcRequest).then(res => { | ||
| // TODO :: Need to add the time on to the ad request. |
There was a problem hiding this comment.
I suggest we rename this function to executeRTC and have it return Promise<!Object<string,string>> allowing for custom parameters to be merged into the ad url (e.g. time delay). Note that this will be page level by definition.
|
|
||
| if (!noCache) { | ||
| // Repopulate the cache. | ||
| xhrFor(this.win).fetchJson(endpoint, { |
There was a problem hiding this comment.
Note that given the current flow, we will possibly send the SWR request before the first request has returned or possibly errored due to non-200 response. Let's change this flow so that sending the cache request is actually dependent on the first rtc request promise. Something like:
const rtcRequest = this.executeRTC_();
if (!swrDisabled) {
rtcRequest.then(() => {
// send rtc request with no-cache option
});
}
return rtcPromise = timerFor(window).timeoutPromise(
RTC_TIMEOUT, rtcRequest).then(res => {
// merge into this.jsonTargeting_ and generate ad url here
return customParams;
});
extensions/amp-ad/0.1/amp-ad.js
Outdated
| // TODO(tdrl): Check amp-ad registry to see if they have this already. | ||
| if (!a4aRegistry[type] || | ||
| this.win.document.querySelector('meta[name=amp-3p-iframe-src]') || | ||
| (this.win.document.querySelector('meta[name=amp-3p-iframe-src]') && |
There was a problem hiding this comment.
Not a fan of the fact that RTC (which is doubleclick specific) is cluttering up amp-ad. We should rethink how to do this later...
| }); | ||
| }); | ||
|
|
||
| it('should not send RTC if url invalid', () => {}); |
There was a problem hiding this comment.
note: still filling these cases in as I go. just pushed out to get the other changes out there.
| // Non-200 status codes are forbidden for RTC. | ||
| // TODO: Add to fetchResponse the ability to | ||
| // check for redirects as well. | ||
| if (res.status != 200) { |
There was a problem hiding this comment.
What about 204 no content? Other 200 codes?
| return null; | ||
| } | ||
| return res.text().then(text => { | ||
| if (text == '') { |
There was a problem hiding this comment.
Add a comment indicating that empty is assumed to indicate ok response with nothing to merge. Can also just do if (!text)
| if (!r) { | ||
| return this.shouldSendRequestWithoutRtc( | ||
| 'Bad response'); | ||
| } else if (typeof r == 'number') { |
There was a problem hiding this comment.
I prefer a consistent object type. How about instead you just check if !r.rtcResponse here?
| const rtcResponse = r.rtcResponse; | ||
| if (!!rtcResponse['targeting']) { | ||
| this.jsonTargeting_['targeting'] = deepMerge( | ||
| this.jsonTargeting_['targeting'] || {}, |
There was a problem hiding this comment.
No need to call deepMerge if this.jsonTargeting_['targeting'] exists, right? Can just assign it. Not sure how much that saves in terms of performance
There was a problem hiding this comment.
I assume you meant that we don't need to call deepMerge if this.jsonTargeting['targeting'] does not exist. I'll change it
| const rtcSuccess = values[1]['success']; | ||
| if (rtcTotalTime) { | ||
| parameters['artc'] = rtcTotalTime; | ||
| parameters['ard'] = parseUrl(rtcConfig['endpoint']).hostname; |
There was a problem hiding this comment.
I'm thinking a cleaner interface is to just have executeRtc return parameters to be added.
There was a problem hiding this comment.
done, switched to using Object.assign too. even cleaner
| } | ||
|
|
||
| const rtcResponse = r.rtcResponse; | ||
| if (!!rtcResponse['targeting']) { |
There was a problem hiding this comment.
Consider creating helper function that takes keyName (e.g. targeting) to reduce duplicate code
| shouldSendRequestWithoutRtc(errMessage) { | ||
| user().error(TAG, errMessage); | ||
| let rtcTotalTime; | ||
| // Have to use match instead of == because AMP |
There was a problem hiding this comment.
Does this comment still apply?
| * model. | ||
| * The targeting info from the RTC updates the targeting info on | ||
| * this object within mergeRtc. | ||
| * @return {?Promise<?number|?string>} The time the RTC callout took |
There was a problem hiding this comment.
Need to update return type here and mergeRtc
| const pageLevelParameters = values[0]; | ||
| let parameters = Object.assign( | ||
| this.getBlockParameters_(), pageLevelParameters); | ||
| if (values[1] && values[1]['artc']) { |
There was a problem hiding this comment.
Anyway you could not require looking for 'artc'?
There was a problem hiding this comment.
yea don't actually need it. Removed.
| const pageLevelParameters = values[0]; | ||
| let parameters = Object.assign( | ||
| this.getBlockParameters_(), pageLevelParameters); | ||
| if (!!values[1]) { |
There was a problem hiding this comment.
you don't actually need this check as assigning null/undefined has no effect. You can change this to:
const parameters = Object.assign(this.getBlockParameters_(), values[1], pageLevelParameters);
| if (!isObject(rtcConfig) || !(endpoint = rtcConfig['endpoint']) || | ||
| typeof endpoint != 'string' || !isSecureUrl(endpoint)) { | ||
| user().warn(TAG, | ||
| `invalid RTC config: ${ampRtcPageElement.textContent}`); |
There was a problem hiding this comment.
Let's add that we're sending the ad request anyway: 'invalid RTC config, sending ad request w/o callout: ${...}'
| } | ||
| let rtcTotalTime; | ||
| const disableSWR = ( | ||
| rtcConfig['disableStaleWhileRevalidate'] === true); |
There was a problem hiding this comment.
Did you consider edge case values such as 'true', 1, and '1'? Ideally there would be a helper function to ensure we are consistent across the codebase (perhaps one already exists?). See sendAdRequestOnFailure you have below.
Also let's move this directly into the if condition below, no need to create variable.
There was a problem hiding this comment.
I'll add 'true' as valid here too, but I'm not sure adding 1 and '1' is necessarily correct.
| Services.xhrFor(this.win).fetchJson( | ||
| endpoint, {credentials: 'include'}).then(res => { | ||
| rtcTotalTime = Date.now() - startTime; | ||
| if (!disableSWR) { |
There was a problem hiding this comment.
Add a comment that SWR should be disabled if the RTC is not cachable (no cache header). Can we read the cache header from the response? Also something about Cache-Control not being supported in IE?
| !!this.jsonTargeting_[key] ? | ||
| deepMerge(this.jsonTargeting_[key], | ||
| rtcResponse[key]) : | ||
| rtcResponse[key]; |
There was a problem hiding this comment.
This indentation looks off...
| rtcResponse[key]; | ||
| } | ||
| }; | ||
| mergeHelper('targeting'); |
There was a problem hiding this comment.
Can we make a whitelist of mergable properties? Then you can just iterate and makes adding others easier in the future
There was a problem hiding this comment.
not totally sure what a whitelist is supposed to look like. I changed it to use a list and that iterate over it. Is that good for now at least?
| // does not include the time to merge. | ||
| return Promise.resolve({ | ||
| artc: r.rtcTotalTime, | ||
| ati: 2, |
There was a problem hiding this comment.
Let's make an enum for Ati values to make the code clearer
| /** @private {?JsonObject|undefined} */ | ||
| let rtcConfig = null; | ||
|
|
||
| /** @private {Object} */ |
| rtcConfig['disableStaleWhileRevalidate'] === true); | ||
| if (rtcConfig['disableStaleWhileRevalidate'] != undefined && | ||
| typeof rtcConfig['disableStaleWhileRevalidate'] != 'boolean') { | ||
| user().warn('RTC disableStaleWhileRevalidate must be a boolean.'); |
There was a problem hiding this comment.
Include the value they set in the message. Also indentation of typeof looks off
| const init = { | ||
| credentials: 'include', | ||
| }; | ||
| // If stale-while-revalidate is disabled, our first RTC callout |
There was a problem hiding this comment.
Wait why? I think my last comment might have been confusing. The flow should be:
- always send RTC with ability to read from cache
- if response indicates not to use SWR OR response indicates it was not cached, then do NOT make SWR. The latter might be difficult depending on if you can read cache headers clientside
| } else if (!r.rtcResponse && r.success) { | ||
| return Promise.resolve({ | ||
| artc: r.rtcTotalTime, | ||
| ati: 2, |
There was a problem hiding this comment.
use your new enum here, can you also add a comment indicating that this is the case where you got an empty response so no need to merge
| if (errMessage.match(/^timeout/)) { | ||
| rtcTotalTime = -1; | ||
| } | ||
| return rtcConfig['sendAdRequestOnFailure'] !== false ? |
| rtcResponse[key]; | ||
| } | ||
| }; | ||
| ['targeting', 'categoryExclusions'].forEach(key => { |
There was a problem hiding this comment.
nit: can be forEach(key => mergeHelper) or move content of mergeHelper into forEach
| /** | ||
| * @param {string} member | ||
| * @param {string} expectedType | ||
| */ |
There was a problem hiding this comment.
Nit: Appears that you're only verifying boolean types so no need to actually pass it. Suggest rename to: logIfRtcConfigMemberNotBoolean(member)
There was a problem hiding this comment.
wanted to make it extensible so I can use for other fields in the config
| this.win.document.querySelector('meta[name=amp-3p-iframe-src]')) || | ||
| // Note that predicate execution may have side effects. | ||
| this.win.document.querySelector('meta[name=amp-3p-iframe-src]') && | ||
| !this.win.document.getElementById('amp-rtc')) || |
There was a problem hiding this comment.
Please bring back to the comment: // Note that predicate...
|
|
||
| const pageLevelParametersPromise = getPageLevelParameters_( | ||
| this.win, this.getAmpDoc(), startTime); | ||
| const rtcRequestPromise = this.executeRtc_(); |
There was a problem hiding this comment.
Actually one more thing: can you add an experiment so that we could turn this off easily if needed?
const rtcRequestPromise = isExperimentOn('disable-rtc') ? Promise.resolve({}) : this.executeRtc_();
Beginning implementation of RTC.