Skip to content
This repository was archived by the owner on Nov 6, 2023. It is now read-only.

Conversation

@cschanaj
Copy link
Collaborator

@cschanaj cschanaj commented Jun 7, 2019

This is a follow-up PR for #17855

There is around 500+ targets which are gone from DNS, however I am not going to handle the fetch errors in the PR.

Besides, it surprises me that the node URL object accepts hostname with leading dots. This make the script from #18068 a little buggy. Since it is not possible to whitelist rulesets from schematic errors, Travis failed. The problematic rulesets are fixed in eef2855

$ node
> new URL('https://.example.com/')
URL {
  href: 'https://.example.com/',
  origin: 'https://.example.com',
  protocol: 'https:',
  username: '',
  password: '',
  host: '.example.com',
  hostname: '.example.com',
  port: '',
  pathname: '/',
  search: '',
  searchParams: URLSearchParams {},
  hash: '' }
$ node --expose-gc utils/memusage.js

[Before]
Initial usage: 42 MB
Maximum usage: 77.8 MB

[After]
Initial usage: 42.3 MB
Maximum usage: 77.6 MB

@zoracon zoracon added the top-10k label Jun 7, 2019
@cschanaj cschanaj changed the title Run node utils/trivialize-rules/trivialize-targets.js WIP: Run node utils/trivialize-rules/trivialize-targets.js Jun 7, 2019
@cschanaj cschanaj changed the title WIP: Run node utils/trivialize-rules/trivialize-targets.js Run node utils/trivialize-rules/trivialize-targets.js Jun 7, 2019
@cschanaj
Copy link
Collaborator Author

cschanaj commented Jun 7, 2019

cc @Hainish @J0WI @Bisaloo @pipboy96 this is ready for review. note that this might cause a merge conflict with #18066 . thanks.

@Hainish
Copy link
Member

Hainish commented Jun 7, 2019

I know it's our normal policy to trivialize rules wherever possible, but have we done any memory analysis on this? I'm a bit worried about balooning the all_rules object...

@pipboy96
Copy link
Contributor

pipboy96 commented Jun 7, 2019

@Hainish I believe complex regexes themselves use up lots of memory.

@Hainish
Copy link
Member

Hainish commented Jun 7, 2019

I missed the memory analysis above! Thanks for this, merging

@Hainish Hainish merged commit 286dc23 into EFForg:master Jun 7, 2019
@cschanaj cschanaj deleted the patch-17855 branch June 7, 2019 22:26
@Bisaloo Bisaloo mentioned this pull request Jul 10, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants