Skip to content

Commit bde3164

Browse files
gorhillgorhill
committed
Add support for 1P, 3P, header= filter options and other changes
New filter options ================== Strict partyness: `1P`, `3P` ---------------------------- The current options 1p/3p are meant to "weakly" match partyness, i.e. a network request is considered 1st-party to its context as long as both the context and the request share the same base domain. The new partyness options are meant to check for strict partyness, i.e. a network request will be considered 1st-party if and only if both the context and the request share the same hostname. For examples: - context: `www.example.org` - request: `www.example.org` - `1p`: yes, `1P`: yes - `3p`: no, `3P`: no - context: `www.example.org` - request: `subdomain.example.org` - `1p`: yes, `1P`: no - `3p`: no, `3P`: yes - context: `www.example.org` - request: `www.example.com` - `1p`: no, `1P`: no - `3p`: yes, `3P`: yes The strict partyness options will be visually emphasized in the editor so as to prevent mistakenly using `1P` or `3P` where weak partyness is meant to be used. Filter on response headers: `header=` ------------------------------------- Currently experimental and under evaluation. Disabled by default, enable by toggling `filterOnHeaders` to `true` in advanced settings. Ability to filter network requests according to whether a specific response header is present and whether it matches or does not match a specific value. For example: *$1p,3P,script,header=via:1\.1\s+google The above filter is meant to block network requests which fullfill all the following conditions: - is weakly 1st-party to the context - is not strictly 1st-party to the context - is of type `script` - has a response HTTP header named `via`, which value matches the regular expression `1\.1\s+google`. The matches are always performed in a case-insensitive manner. The header value is assumed to be a literal regular expression, except for the following special characters: - to anchor to start of string, use leading `|`, not `^` - to anchor to end of string, use trailing `|`, not `$` - to invert the test, use a leading `!` To block a network request if it merely contains a specific HTTP header is just a matter of specifying the header name without a header value: *$1p,3P,script,header=via Generic exception filters can be used to disable specific block `header=` filters, i.e. `@@*$1p,3P,script,header` will override the block `header=` filters given as example above. Dynamic filtering's `allow` rules override block `headers=` filters. Important: It is key that filter authors use as many narrowing filter options as possible when using the `header=` option, and the `header=` option should be used ONLY when other filter options are not sufficient. More documentation justifying the purpose of `header=` option will be provided eventually if ever it is decided to move it from experimental to stable status. To be decided: to restrict usage of this filter option to only uBO's own filter lists or "My filters". Changes ======= Fine tuning `queryprune=` ------------------------- The following changes have been implemented: The special value `*` (i.e. `queryprune=*`) means "remove all query parameters". If the `queryprune=` value is made only of alphanumeric characters (including `_`), the value will be internally converted to regex equivalent `^value=`. This ensures a better future compatibility with AdGuard's `removeparam=`. If the `queryprune=` value starts with `!`, the test will be inverted. This can be used to remove all query parameters EXCEPT those who match the specified value. Other ----- The legacy code to test for spurious CSP reports has been removed. This is no longer an issue ever since uBO redirects to local resources through web accessible resources. Notes ===== The following new and recently added filter options are not compatible with Chromium's manifest v3 changes: - `queryprune=` - `1P` - `3P` - `header=`
1 parent 50ad64d commit bde3164

File tree

8 files changed

+409
-208
lines changed

8 files changed

+409
-208
lines changed

‎src/css/codemirror.css‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -74,9 +74,9 @@
7474
.cm-s-default .cm-keyword {
7575
color: var(--sf-keyword-ink);
7676
}
77-
.cm-s-default .cm-regex {
77+
.cm-s-default .cm-notice {
7878
text-underline-position: under;
79-
text-decoration-color: var(--sf-regex-ink);
79+
text-decoration-color: var(--sf-notice-ink);
8080
text-decoration-style: solid;
8181
text-decoration-line: underline;
8282
}

‎src/css/themes/default.css‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -191,7 +191,7 @@
191191
--sf-error-ink: #ff0000;
192192
--sf-error-surface: #ff000016;
193193
--sf-keyword-ink: var(--purple-60);
194-
--sf-regex-ink: var(--light-gray-60);
194+
--sf-notice-ink: var(--light-gray-60);
195195
--sf-tag-ink: #117700;
196196
--sf-value-ink: var(--orange-80);
197197
--sf-variable-ink: var(--default-ink);

‎src/js/background.js‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,9 +61,10 @@ const µBlock = (( ) => { // jshint ignore:line
6161
debugScriptletInjector: false,
6262
disableWebAssembly: false,
6363
extensionUpdateForceReload: false,
64+
filterAuthorMode: false,
65+
filterOnHeaders: false,
6466
ignoreRedirectFilters: false,
6567
ignoreScriptInjectFilters: false,
66-
filterAuthorMode: false,
6768
loggerPopupType: 'popup',
6869
manualUpdateAssetFetchPeriod: 500,
6970
popupFontSize: 'unset',

‎src/js/codemirror/ubo-static-filtering.js‎

Lines changed: 32 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -212,20 +212,42 @@ CodeMirror.defineMode('ubo-static-filtering', function() {
212212

213213
const colorNetOptionSpan = function(stream) {
214214
const bits = parser.slices[parserSlot];
215-
let style;
216215
if ( (bits & parser.BITComma) !== 0 ) {
217-
style = 'def strong';
218216
netOptionValueMode = false;
219-
} else if ( netOptionValueMode ) {
217+
stream.pos += parser.slices[parserSlot+2];
218+
parserSlot += 3;
219+
return 'def strong';
220+
}
221+
if ( netOptionValueMode ) {
220222
return colorNetOptionValueSpan(stream, bits);
221-
} else if ( (bits & parser.BITTilde) !== 0 ) {
222-
style = 'keyword strong';
223-
} else if ( (bits & parser.BITEqual) !== 0 ) {
223+
}
224+
if ( (bits & parser.BITTilde) !== 0 ) {
225+
stream.pos += parser.slices[parserSlot+2];
226+
parserSlot += 3;
227+
return 'keyword strong';
228+
}
229+
if ( (bits & parser.BITEqual) !== 0 ) {
224230
netOptionValueMode = true;
231+
stream.pos += parser.slices[parserSlot+2];
232+
parserSlot += 3;
233+
return 'def';
225234
}
226-
stream.pos += parser.slices[parserSlot+2];
227-
parserSlot += 3;
228-
return style || 'def';
235+
const to = parser.skipUntil(
236+
parserSlot,
237+
parser.commentSpan.i,
238+
parser.BITComma | parser.BITEqual
239+
);
240+
if (
241+
to > parserSlot &&
242+
/^[13]P/.test(parser.strFromSlices(parserSlot, to - 3))
243+
) {
244+
parserSlot = to;
245+
stream.pos = parser.slices[to+1];
246+
return 'def notice';
247+
}
248+
parserSlot = to;
249+
stream.pos = parser.slices[to+1];
250+
return 'def';
229251
};
230252

231253
const colorNetSpan = function(stream) {
@@ -259,7 +281,7 @@ CodeMirror.defineMode('ubo-static-filtering', function() {
259281
if ( parser.patternIsRegex() ) {
260282
stream.pos = parser.slices[parser.optionsAnchorSpan.i+1];
261283
parserSlot = parser.optionsAnchorSpan.i;
262-
return 'variable regex';
284+
return 'variable notice';
263285
}
264286
if ( (parser.slices[parserSlot] & (parser.BITAsterisk | parser.BITCaret)) !== 0 ) {
265287
stream.pos += parser.slices[parserSlot+2];

‎src/js/pagestore.js‎

Lines changed: 47 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -272,7 +272,6 @@ const PageStore = class {
272272
this.popupBlockedCount = 0;
273273
this.largeMediaCount = 0;
274274
this.largeMediaTimer = null;
275-
this.internalRedirectionCount = 0;
276275
this.allowLargeMediaElementsRegex = undefined;
277276
this.extraData.clear();
278277

@@ -668,11 +667,57 @@ const PageStore = class {
668667
return result;
669668
}
670669

670+
filterOnHeaders(fctxt, headers) {
671+
fctxt.filter = undefined;
672+
673+
if ( this.getNetFilteringSwitch(fctxt) === false ) { return 0; }
674+
675+
let result = µb.staticNetFilteringEngine.matchHeaders(fctxt, headers);
676+
if ( result === 0 ) { return 0; }
677+
678+
const loggerEnabled = µb.logger.enabled;
679+
if ( loggerEnabled ) {
680+
fctxt.filter = µb.staticNetFilteringEngine.toLogData();
681+
}
682+
683+
// Dynamic filtering allow rules
684+
// URL filtering
685+
if (
686+
result === 1 &&
687+
µb.sessionURLFiltering.evaluateZ(
688+
fctxt.getTabHostname(),
689+
fctxt.url,
690+
fctxt.type
691+
) === 2
692+
) {
693+
result = 2;
694+
if ( loggerEnabled ) {
695+
fctxt.filter = µb.sessionURLFiltering.toLogData();
696+
}
697+
}
698+
// Hostname filtering
699+
if (
700+
result === 1 &&
701+
µb.userSettings.advancedUserEnabled &&
702+
µb.sessionFirewall.evaluateCellZY(
703+
fctxt.getTabHostname(),
704+
fctxt.getHostname(),
705+
fctxt.type
706+
) === 2
707+
) {
708+
result = 2;
709+
if ( loggerEnabled ) {
710+
fctxt.filter = µb.sessionFirewall.toLogData();
711+
}
712+
}
713+
714+
return result;
715+
}
716+
671717
redirectBlockedRequest(fctxt) {
672718
if ( µb.hiddenSettings.ignoreRedirectFilters === true ) { return; }
673719
const directive = µb.staticNetFilteringEngine.redirectRequest(fctxt);
674720
if ( directive === undefined ) { return; }
675-
this.internalRedirectionCount += 1;
676721
if ( µb.logger.enabled !== true ) { return; }
677722
fctxt.pushFilter(directive.logData());
678723
if ( fctxt.redirectURL === undefined ) { return; }

‎src/js/static-filtering-parser.js‎

Lines changed: 59 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -346,6 +346,7 @@ const Parser = class {
346346
// patternRightAnchorSpan: first slice to right-hand pattern anchor
347347
// optionsAnchorSpan: first slice to options anchor
348348
// optionsSpan: first slice to options
349+
// commentSpan: first slice to trailing comment
349350
analyzeNet() {
350351
let islice = this.leftSpaceSpan.len;
351352

@@ -369,7 +370,7 @@ const Parser = class {
369370
}
370371

371372
// Assume no options
372-
this.optionsAnchorSpan.i = this.optionsSpan.i = this.commentSpan.i;
373+
this.optionsAnchorSpan.i = this.optionsSpan.i = this.commentSpan.i;
373374

374375
// Assume all is part of pattern
375376
this.patternSpan.i = islice;
@@ -1900,41 +1901,44 @@ const BITFlavorNetAnchor = BITFlavorNetLeftAnchor | BITFlavorNetRightAnc
19001901
const OPTTokenMask = 0x000000ff;
19011902
const OPTTokenInvalid = 0;
19021903
const OPTToken1p = 1;
1903-
const OPTToken3p = 2;
1904-
const OPTTokenAll = 3;
1905-
const OPTTokenBadfilter = 4;
1906-
const OPTTokenCname = 5;
1907-
const OPTTokenCsp = 6;
1908-
const OPTTokenCss = 7;
1909-
const OPTTokenDenyAllow = 8;
1910-
const OPTTokenDoc = 9;
1911-
const OPTTokenDomain = 10;
1912-
const OPTTokenEhide = 11;
1913-
const OPTTokenEmpty = 12;
1914-
const OPTTokenFont = 13;
1915-
const OPTTokenFrame = 14;
1916-
const OPTTokenGenericblock = 15;
1917-
const OPTTokenGhide = 16;
1918-
const OPTTokenImage = 17;
1919-
const OPTTokenImportant = 18;
1920-
const OPTTokenInlineFont = 19;
1921-
const OPTTokenInlineScript = 20;
1922-
const OPTTokenMedia = 21;
1923-
const OPTTokenMp4 = 22;
1924-
const OPTTokenObject = 23;
1925-
const OPTTokenOther = 24;
1926-
const OPTTokenPing = 25;
1927-
const OPTTokenPopunder = 26;
1928-
const OPTTokenPopup = 27;
1929-
const OPTTokenRedirect = 28;
1930-
const OPTTokenRedirectRule = 29;
1931-
const OPTTokenQueryprune = 30;
1932-
const OPTTokenScript = 31;
1933-
const OPTTokenShide = 32;
1934-
const OPTTokenXhr = 33;
1935-
const OPTTokenWebrtc = 34;
1936-
const OPTTokenWebsocket = 35;
1937-
const OPTTokenCount = 36;
1904+
const OPTToken1pStrict = 2;
1905+
const OPTToken3p = 3;
1906+
const OPTToken3pStrict = 4;
1907+
const OPTTokenAll = 5;
1908+
const OPTTokenBadfilter = 6;
1909+
const OPTTokenCname = 7;
1910+
const OPTTokenCsp = 8;
1911+
const OPTTokenCss = 9;
1912+
const OPTTokenDenyAllow = 10;
1913+
const OPTTokenDoc = 11;
1914+
const OPTTokenDomain = 12;
1915+
const OPTTokenEhide = 13;
1916+
const OPTTokenEmpty = 14;
1917+
const OPTTokenFont = 15;
1918+
const OPTTokenFrame = 16;
1919+
const OPTTokenGenericblock = 17;
1920+
const OPTTokenGhide = 18;
1921+
const OPTTokenHeader = 19;
1922+
const OPTTokenImage = 20;
1923+
const OPTTokenImportant = 21;
1924+
const OPTTokenInlineFont = 22;
1925+
const OPTTokenInlineScript = 23;
1926+
const OPTTokenMedia = 24;
1927+
const OPTTokenMp4 = 25;
1928+
const OPTTokenObject = 26;
1929+
const OPTTokenOther = 27;
1930+
const OPTTokenPing = 28;
1931+
const OPTTokenPopunder = 29;
1932+
const OPTTokenPopup = 30;
1933+
const OPTTokenRedirect = 31;
1934+
const OPTTokenRedirectRule = 32;
1935+
const OPTTokenQueryprune = 33;
1936+
const OPTTokenScript = 34;
1937+
const OPTTokenShide = 35;
1938+
const OPTTokenXhr = 36;
1939+
const OPTTokenWebrtc = 37;
1940+
const OPTTokenWebsocket = 38;
1941+
const OPTTokenCount = 39;
19381942

19391943
//const OPTPerOptionMask = 0x0000ff00;
19401944
const OPTCanNegate = 1 << 8;
@@ -1974,9 +1978,11 @@ Parser.prototype.BITHostname = BITHostname;
19741978
Parser.prototype.BITPeriod = BITPeriod;
19751979
Parser.prototype.BITDash = BITDash;
19761980
Parser.prototype.BITHash = BITHash;
1981+
Parser.prototype.BITNum = BITNum;
19771982
Parser.prototype.BITEqual = BITEqual;
19781983
Parser.prototype.BITQuestion = BITQuestion;
19791984
Parser.prototype.BITPercent = BITPercent;
1985+
Parser.prototype.BITAlpha = BITAlpha;
19801986
Parser.prototype.BITTilde = BITTilde;
19811987
Parser.prototype.BITUnicode = BITUnicode;
19821988
Parser.prototype.BITIgnore = BITIgnore;
@@ -1993,7 +1999,10 @@ Parser.prototype.BITFlavorIgnore = BITFlavorIgnore;
19931999
Parser.prototype.BITFlavorUnsupported = BITFlavorUnsupported;
19942000
Parser.prototype.BITFlavorError = BITFlavorError;
19952001

1996-
Parser.prototype.OPTTokenInvalid = OPTTokenInvalid;
2002+
Parser.prototype.OPTToken1p = OPTToken1p;
2003+
Parser.prototype.OPTToken1pStrict = OPTToken1pStrict;
2004+
Parser.prototype.OPTToken3p = OPTToken3p;
2005+
Parser.prototype.OPTToken3pStrict = OPTToken3pStrict;
19972006
Parser.prototype.OPTTokenAll = OPTTokenAll;
19982007
Parser.prototype.OPTTokenBadfilter = OPTTokenBadfilter;
19992008
Parser.prototype.OPTTokenCname = OPTTokenCname;
@@ -2003,14 +2012,15 @@ Parser.prototype.OPTTokenDoc = OPTTokenDoc;
20032012
Parser.prototype.OPTTokenDomain = OPTTokenDomain;
20042013
Parser.prototype.OPTTokenEhide = OPTTokenEhide;
20052014
Parser.prototype.OPTTokenEmpty = OPTTokenEmpty;
2006-
Parser.prototype.OPTToken1p = OPTToken1p;
20072015
Parser.prototype.OPTTokenFont = OPTTokenFont;
20082016
Parser.prototype.OPTTokenGenericblock = OPTTokenGenericblock;
20092017
Parser.prototype.OPTTokenGhide = OPTTokenGhide;
2018+
Parser.prototype.OPTTokenHeader = OPTTokenHeader;
20102019
Parser.prototype.OPTTokenImage = OPTTokenImage;
20112020
Parser.prototype.OPTTokenImportant = OPTTokenImportant;
20122021
Parser.prototype.OPTTokenInlineFont = OPTTokenInlineFont;
20132022
Parser.prototype.OPTTokenInlineScript = OPTTokenInlineScript;
2023+
Parser.prototype.OPTTokenInvalid = OPTTokenInvalid;
20142024
Parser.prototype.OPTTokenMedia = OPTTokenMedia;
20152025
Parser.prototype.OPTTokenMp4 = OPTTokenMp4;
20162026
Parser.prototype.OPTTokenObject = OPTTokenObject;
@@ -2025,7 +2035,6 @@ Parser.prototype.OPTTokenScript = OPTTokenScript;
20252035
Parser.prototype.OPTTokenShide = OPTTokenShide;
20262036
Parser.prototype.OPTTokenCss = OPTTokenCss;
20272037
Parser.prototype.OPTTokenFrame = OPTTokenFrame;
2028-
Parser.prototype.OPTToken3p = OPTToken3p;
20292038
Parser.prototype.OPTTokenXhr = OPTTokenXhr;
20302039
Parser.prototype.OPTTokenWebrtc = OPTTokenWebrtc;
20312040
Parser.prototype.OPTTokenWebsocket = OPTTokenWebsocket;
@@ -2045,8 +2054,10 @@ Parser.prototype.OPTNotSupported = OPTNotSupported;
20452054
const netOptionTokenDescriptors = new Map([
20462055
[ '1p', OPTToken1p | OPTCanNegate ],
20472056
[ 'first-party', OPTToken1p | OPTCanNegate ],
2057+
[ '1P', OPTToken1pStrict ],
20482058
[ '3p', OPTToken3p | OPTCanNegate ],
20492059
[ 'third-party', OPTToken3p | OPTCanNegate ],
2060+
[ '3P', OPTToken3pStrict ],
20502061
[ 'all', OPTTokenAll | OPTNetworkType | OPTNonCspableType ],
20512062
[ 'badfilter', OPTTokenBadfilter ],
20522063
[ 'cname', OPTTokenCname | OPTAllowOnly | OPTModifierType ],
@@ -2066,6 +2077,7 @@ const netOptionTokenDescriptors = new Map([
20662077
[ 'genericblock', OPTTokenGenericblock | OPTNotSupported ],
20672078
[ 'ghide', OPTTokenGhide | OPTNonNetworkType | OPTNonCspableType | OPTNonRedirectableType ],
20682079
[ 'generichide', OPTTokenGhide | OPTNonNetworkType | OPTNonCspableType | OPTNonRedirectableType ],
2080+
[ 'header', OPTTokenHeader | OPTMustAssign | OPTAllowMayAssign | OPTNonCspableType | OPTNonRedirectableType ],
20692081
[ 'image', OPTTokenImage | OPTCanNegate | OPTNetworkType | OPTModifiableType | OPTRedirectableType | OPTNonCspableType ],
20702082
[ 'important', OPTTokenImportant | OPTBlockOnly ],
20712083
[ 'inline-font', OPTTokenInlineFont | OPTNonNetworkType | OPTCanNegate | OPTNonCspableType | OPTNonRedirectableType ],
@@ -2097,8 +2109,10 @@ Parser.prototype.netOptionTokenDescriptors =
20972109
Parser.netOptionTokenIds = new Map([
20982110
[ '1p', OPTToken1p ],
20992111
[ 'first-party', OPTToken1p ],
2112+
[ '1P', OPTToken1pStrict ],
21002113
[ '3p', OPTToken3p ],
21012114
[ 'third-party', OPTToken3p ],
2115+
[ '3P', OPTToken3pStrict ],
21022116
[ 'all', OPTTokenAll ],
21032117
[ 'badfilter', OPTTokenBadfilter ],
21042118
[ 'cname', OPTTokenCname ],
@@ -2118,6 +2132,7 @@ Parser.netOptionTokenIds = new Map([
21182132
[ 'genericblock', OPTTokenGenericblock ],
21192133
[ 'ghide', OPTTokenGhide ],
21202134
[ 'generichide', OPTTokenGhide ],
2135+
[ 'header', OPTTokenHeader ],
21212136
[ 'image', OPTTokenImage ],
21222137
[ 'important', OPTTokenImportant ],
21232138
[ 'inline-font', OPTTokenInlineFont ],
@@ -2145,7 +2160,9 @@ Parser.netOptionTokenIds = new Map([
21452160

21462161
Parser.netOptionTokenNames = new Map([
21472162
[ OPTToken1p, '1p' ],
2163+
[ OPTToken1pStrict, '1P' ],
21482164
[ OPTToken3p, '3p' ],
2165+
[ OPTToken3pStrict, '3P' ],
21492166
[ OPTTokenAll, 'all' ],
21502167
[ OPTTokenBadfilter, 'badfilter' ],
21512168
[ OPTTokenCname, 'cname' ],
@@ -2160,6 +2177,7 @@ Parser.netOptionTokenNames = new Map([
21602177
[ OPTTokenFont, 'font' ],
21612178
[ OPTTokenGenericblock, 'genericblock' ],
21622179
[ OPTTokenGhide, 'generichide' ],
2180+
[ OPTTokenHeader, 'header' ],
21632181
[ OPTTokenImage, 'image' ],
21642182
[ OPTTokenImportant, 'important' ],
21652183
[ OPTTokenInlineFont, 'inline-font' ],
@@ -2300,6 +2318,8 @@ const NetOptionsIterator = class {
23002318
}
23012319
// Keep track of which options are present: any given option can
23022320
// appear only once.
2321+
// TODO: might need to make an exception for `header=` option so as
2322+
// to allow filters which need to match more than one header.
23032323
const tokenId = descriptor & OPTTokenMask;
23042324
if ( tokenId !== OPTTokenInvalid ) {
23052325
if ( this.tokenPos[tokenId] !== -1 ) {

0 commit comments

Comments
 (0)