Issue with full Javascript blocking on Firefox 56/57

[ghost]

I have a specific use case and the extension is definitely not working as it should, there is a flaw somewhere - either in the extension or in Firefox:

Try this to see what I am talking about - I am developing an advert blocking solution:

• Deploy Nginx and set it to listen on port 80/443, and serve a no-content reply for all queries (return 204 flag).
• Add the following hosts to your hosts file and point them to 127.0.0.1

www.google-analytics.com
counter.yadro.ru
jsc.traffic-media.co
buhman.letysheeps.ru
trafgid.com
am15.net
stablemoney.ru
adbetnet.advertserve.com
piguiqproxy.com

• Open fastpic dot ru on the latest Firefox 57.0.3 with Umatrix 1.1.20 (the page displays like CSS is messed up or not loading or something is wrong with the page) - Umatrix is set-up to block both third and first party scripts on that domain and all domains that it loads. When I set the browser preference javascript.enabled to false the page loads correctly, I have tested this and definitely there is something wrong with javascript blocking.

I have tested this on Firefox 56 as well with the same parameters, and using an extension called Yesscript:

• I disable Umatrix
• I enable yesscript and block scripts on fastpic.ru, the page loads correctly
• Then I disable yesscript and enable Umatrix, and block all first and third party scripts on it, and the page does not load correctly, I am at a loss at what's going on, as the Umatrix logger is showing that the scripts are being blocked but the page / Firefox is detecting them as they are not. I am not using any other extensions or special modifications to the browser.

p.s. Maybe it has something to do with the way the addons are blocking Javascript - Yesscript is blocking it like this I think:

prefs.setCharPref("capability.policy.yesscript.javascript.enabled", "noAccess");

[gorhill]

I am going to transcribe your paragraph as steps for clarity -- tell me if I got something wrong:

1. uMatrix is set-up to block both third and first party scripts on that domain and all domains that it loads.
   • Rule * * script block added to uMatrix's default ruleset
2. Open fastpic.ru on the latest Firefox 57.0.3 with uMatrix 1.1.20 configured with above rule.
3. Result: the page displays like CSS is messed up or not loading or something is wrong with the page

When I set the browser preference javascript.enabled to false the page loads correctly.

Regarding the difference between uMatrix with javascript disabled and the browser with javascript disabled:

With javascript disabled through the browser's config:

With javascript disabled through uMatrix:

The difference disappears when you remove the block rule for static.fastpic.ru: there are images and stylesheets resources pulled from that subdomain, and that subdomain is blocked by one of the hosts file selected by default in uMatrix. The matrix UI shows clearly that some resources are blocked from that subdomain, so this should have been the first thing to look at. Once the block rule for the subdomain is removed, no difference with when javascript is blocked through browser's config:

Now to ensure that uMatrix works best, do not test it with YesScript2: this extension listens to onHeadersReceived event and potentially returns the response headers. There is an issue in Firefox which will make the browser discard the header modifications by one extension when another one does act as if it modified them. See arkenfox/user.js#265 (comment).

Aside this, it may happens that you need to reload and force the browser to bypass its cache: see #893.

Aside these known issues for which there are workaround (you will have to disable YesScript2), uMatrix is completely reliable. Use the browser console to see that the browser reports inline script tags as blocked.

[gorhill]

I corrected myself above, I misread the code in YesScript2. In any case, please provide exact, clearly laid out repro steps to be sure I understand what exact scenario you say you have an issue. As said in my response, scripts were blocked all fine on my side. Using uMatrix as the only extension.

[ghost]

Hi,

First, and foremost, thank you for responding so fast !

I am testing with YesScript (not YesScript2) on Firefox 56 (to exclude that the issue is purely with Firefox 57). Let's ignore that part for a sec.

I think the issue is this:

• I am blocking all of the third party advert resources on fastpic.ru via a local hosts file (static mapping to 127.0.0.1 for the domains described in my post)
• I am hosting an Nginx HTTP/HTTPS server on the same machine which responds to requests on 127.0.0.1, so that for any domains that are in the hosts file you get a simple "No content" reply - so called HTTP/204 response. In this case the page gets messed up as the browser/extension clearly does not like that "No Content" response. Again this works nicely with Firefox 56 and Yesscript, as well as Firefox 57 with javascript.enabled set to false.
• I am testing on fastpic.ru with static.fastpic.ru CSS/Images resources fully allowed.
• I have also disabled all hosts files used by uMatrix.

One thing I've noticed is that YesScript disables Javascript via internal calls to the Firefox 56 browser capability.policy, can uMatrix do something like this ?

[gorhill]

I don't know how YesScript plain works, a quick glance and I can't tell, I would need more time to find out. I can't try FF56 for the moment due to lack of time (need to download, new profile, etc.)

However see if the browser console reports that inline script tags aren't executed, this is a telltale sign that uMatrix is blocking 1st-party scripts:

By the way, mixing legacy extensions with WebExtensions ones is known to cause issues, it's something I've had to repeat often to people who were still using uBO with legacy extensions.

[ghost]

Ok, based on your screenshot I think I know what the issue really is:

I had the following Firefox preferences set to false for testing:

security.csp.enable
security.csp.enableStrictDynamic

Both rlslog dot net, and fastpic ru are now working as they should !

Based on this can you confirm that your extension is in fact relying on CSP, and not on Firefox Capability policy (if there is one at all in Firefox 57) ?

[gorhill]

With WebExtensions there is no other way to control javascript execution of inline script. All WebExtensions which purpose is to control javascript now relies on CSP (YesScript2, NoScript, etc.)

See https://bugzilla.mozilla.org/show_bug.cgi?id=1392090.

[ghost]

Damn, ok, sorry for taking up your time, in case someone runs into the same issue they will now know what's at fault.

I have corrected my Firefox Addons review, please modify yours as well so the users don't get bothered by it.

[WagnerGMD]

For the record :

uMatrix_Legacy won't receive any update but it still available (etc)...

I'm wonder it wouldn't be better to add one article on this matter ? Because on the moment, I haven't find these informations on the uMatrix Wiki.

• https://github.com/gorhill/uMatrix/releases
• https://github.com/gorhill/uMatrix/releases/tag/1.1.4
• https://github.com/gorhill/uMatrix/releases/download/1.1.4/uMatrix.firefox.xpi
• Missing uMatrix.firefox.xpi #824

* * * block
* 1st-party css allow
* 1st-party image allow
twitter.com twimg.com css allow
twitter.com twimg.com image allow

As today, I had noticed a very strange behavior : an infinite loop on Twitter (under Windows 10 x64 with Firefox_v59 or Firefox_v57). That's why I can't read it (despite these rules).
In fact, I had met this trouble right after the installation of uMatrix_WebExtension_v1.2.0. And I had check it, the trouble doesn't seem to exist with uMatrix_Legacy_v1.1.4 (and according to these links, it will be the last one).

## If you need one example just try to open the second link :
1) https://mobile.twitter.com/waterfoxproject
2) https://www.twitter.com/waterfoxproject
## To stop the loop, you will need to create this rule : "twitter.com 1st-party cookie allow"
## No it doesn't stop the redirection.
## But Twitter will be now accessible (readable without javascript) at this adress : https://mobile.twitter.com
## From my point of view, that's a bad design. Because any website should be able to load the CSS without them (cookies, javascript, etc).

Could you confirm it ?
We can't stop (neither extract (recopy, etc) the data from) the uMatrix Logger ? Where is the button (and which one ? Add one label to describe the button because it's missing (nothing as text appears to identify the buttons CSS:hover)) ? Unless it was broken by the loop ?