CODEWITHSUNDEEP
privacyadblockthunderbird-addonmalware-detectionantimalware
atErik
atErik

Reputation: 1077

Add/Use EasyList type Of URLs Manually With Thunderbird Or Add AdBlock/uBlock-Origin/Alt Addon

How can I add EasyList type of URLs manually into Thunderbird? Or, how can I add AdBlock or uBlock-Origin or Alternative addon into Thunderbird?

DETAILS+EXTRA INFO:

Thunderbird is primarily an email client, but it can also do many other functions: instant-messaging/chat, it can do various type of interactions with various types of webservices to obtain user-preferred data, it also contains lightweight web-browser.

I'm using Thunderbird v68.11.0 on MacOS 10.15.0 (64bit-only) (Catalina).
I also use TB on Windows10.

I use "BrowseInTab" addon inside TB to open a browser tab in TB, to access email websites directly inside TB, for those email-service providers who do not provide free IMAPS/POPS/SMTPS service.

As of July-2020, there is NO AdBlock or AdBlockPlus or any alternative addon (uBlock-origin,etc) available directly from Thunderbird-addons site. AdBlock addon stopped appearing in search results inside TB.

ISSUES: Some ADs play/show not only images, but videos too (or very fast changing (aka, animated) GIFs), & show too many annoying/flashing or adult contents, etc , Multiple ADs are shown even inside multiple popups inside a single browsing tab, and began to consume too much computing resources, and those activities reach to a point where TB consumes much more computing-resource than even a video player! I can tolerate up to: simple & small-size TEXT based ADs, or tiny-image based non-annoying & small ADs , I can also tolerate if+when they use very-very less-amount of computing resources ONLY-WHEN computer-hardwares go into IDLE mode or when AD uses resources in lowest-priority mode, ( but even then those activities are still using/consuming/occupying memory/files/etc 😞 ).
Big Problems are: most ADs & their supporters & their implementors are invading Privacy or intrusive or profiling or harvesting/mining/STEALING user's data, etc thus breaking Privacy-Rights, and did not obtain my/user's specific fully-aware consent/permission before showing each specific AD, and did not obtain my/user's permission to consume my/user's computing-resources & deprive my/user's actual works/programs, etc.

EasyList contain list of websites that has list of AD-providers, privacy invaders, etc . There are also few other lists (EasyPrivacy, Peter Lowe’s ad/tracking/malware servers, Malicious URL Blocklist, uBlock-Origin, etc) for listing harmful actors, exploiters, Spammers, adult-AD-sites, malware/virus-distributors, user's-data-miners, etc.
Here i will use just "EasyList",etc to indicate all of those various lists.

Please suggest solutions+instructions for these:

I would prefer solutions for above cases/requests that utilize or compatible-with these : scripts,parsers,etc that is using or compatible-with Bash [v3.2.57(1)], Perl [perl 5, version 18, subversion 4 (v5.18.4)], etc, Or source-code of C/objC based programs compile-able on [Xcode command-line-tools(CLT) 11.5, Apple clang version 11.0.3 (clang-1103.0.32.62), Target: x86_64-apple-darwin19.6.0, Thread model: posix, MacOSX/Catalina/64bit, LLVM 9.0.0, Swift 5.2.4] and gcc/mingw64 on Windows], or manual instructions, etc.

IDEAS/PARTIAL-SOLUTIONS:
I'm adding helpful & more links/ideas related to above sections, so that an answer provider/giver can get more ideas on how to create helpful answer more quickly & easily, to answer my question:

Upvotes: -2

Views: 1267

Answers (1)

atErik
atErik

Reputation: 1077

SOLUTION # 1 :
Use uBlock addon in Thunderbird(TB) :
Load uBlock in this way (summary: download addon XPI file from Firefox addon site, then load XPI into Thunderbird):

  • Goto Mozilla Firefox Addons site :
        https://addons.mozilla.org/en-US/firefox/
  • search for "uBlock" or "uBlock Origin" addon, search result should show it, go into it, make sure it is from Developer "Raymond Hill".
  • right-click on [ + Add to Firefox ] button, & choose the option "Save link as" or something similar (if you use different than Firefox).
    save the XPI file ublock_origin-1.28.4-an+fx.xpi in your Downloads ( ~/Downloads ) folder.
  • goto/start Thunderbird > Tools > Addons >
    find the line that shows "Manage Your Extensions", there is a âš™ "GEAR" button in its right-side, that gear also has a tiny â–¼ down-arrow symbol next to it, to indicate it has a drop-down menu/options, Click on that dropdown option,
    choose "Install Add-on From File" option > browse into Downloads folder & choose the ublock_origin-1.28.4-an+fx.xpi file > "Open" >
    select "Accept" to permit/allow it to access what it needs to >
    select "Done"/"Ok".
  • now its available for use in Thunderbird.
    Goto its "Preferences" button to configure further, if you need to.
    i have tried it, tested it, works great. Options can be set as well.

Note: this is a good temporary solution for now until uBlock devs release correct replacement. So we do need to request uBlock-Origin devs in Github (here i think) to release a correct one correctly for the Thunderbird.

SOLUTION # 2 :
Create a combined block-list, & apply on where applicable, i.e /etc/hosts file:
Here(Reddit:/r/pihole) is a bash & perl script combination, that can obtain various latest list file, then combine them, Thank to the dev who posted it there 4yrs ago. Links are not right there anymore, etc, but script commands are valid, and needs lots of improvements . A simple adaptation is shown below, you must understand coding pattern & reason, & modify further yourself:

  • MacOS (upto Catalina/10.15 or BugSur/11.0) pre-includes various Unix & Linux tools, so below commands, examples, etc are valid for it directly without any further package-manager. And you may try MacPorts for more+latest Unix+Linux tools, (Homebrew package-manager does-not respect Privacy-Rights 1, 2, 3, 4, 5, 6, ..., so choice is yours).
    In Windows, please get Cygwin package manager to get Unix, Linux tools, in Cygwin based (bash)-shell you can use/run below commands, examples, etc . Instead of Cygwin you can also obtain MSYS2 shell for the mingw64 toolchain, as you indicated you want to use those Open-Source tools to develop/compile programs . Windows10 also allows to get Debian-GNU/Linux from AppStore, its not complete but should be sufficient for development related purpose, so you may use that also if you need GNU/Linux . And another option is Homebrew package-manager but it does-not respect Privacy-Rights, so choice is yours.
  • content of easylist.pl file : this script will go thru a file and filter out anything that doesn't match a domain name, which will allow you to use lists in the easylist format. 
    #!/usr/bin/perl -w
# Source: https://newspaint.wordpress.com/2014/08/18/filtering-easylist-for-hosts-file-style-adblock/

use strict;

my %hosts = ();
while ( <> ) {
  if ( $_ =~ m/^\|\|([a-z][a-z0-9-_.]+\.([a-z]{2,3}))\^\s*$/ ) {
      $hosts{$1} = 1;
  }
}

foreach my $host ( sort keys %hosts ) {
  print( "$host\n" );
}
  • MacOS upto BigSur(11.0) pre-includes Perl. Get Strawberry Perl for Windows.
  • here is a sample lists.lst file w/only 6 lists, please modify them to updated & correct list(s), which you want to apply for your implementation case: 
    https://easylist-downloads.adblockplus.org/easylist.txt
https://easylist-downloads.adblockplus.org/abpindo+easylist.txt
https://raw.githubusercontent.com/ABPindo/indonesianadblockrules/master/subscriptions/abpindo.txt
https://easylist-downloads.adblockplus.org/bulgarian_list+easylist.txt
http://stanev.org/abp/adblock_bg.txt
https://easylist-downloads.adblockplus.org/easylistchina+easylist.txt
  • content of manual-easylist.sh file (developed by reddit user) : it downloads the sources above, then parses them using easylist.pl, then appends them to the blacklist (and cleans any duplicates), and finally runs "gravity.sh" to update the HOSTS file in Raspberry-Pi, so change the line with "gravity" to suit with your OS/distro. 
    #!/bin/bash
for source in `cat lists.lst` ; do
  echo "$source" ;
  curl --silent $source >> ads.txt
  echo -e "\t`wc -l ads.txt | cut -d " " -f 1` lines downloaded"
done

echo -e "\nFiltering non-url content..."
perl easylist.pl ads.txt > ads_parsed.txt
rm ads.txt
echo -e "\t`wc -l ads_parsed.txt | cut -d " " -f 1` lines after parsing"

echo -e "\nRemoving duplicates..."
sort -u ads_parsed.txt > ads_unique.txt
rm ads_parsed.txt
echo -e "\t`wc -l ads_unique.txt | cut -d " " -f 1` lines after deduping"

cat ads_unique.txt >> /etc/pihole/blacklist.txt
sort -u /etc/pihole/blacklist.txt > /etc/pihole/blacklist.txt2
rm /etc/pihole/blacklist.txt
mv /etc/pihole/blacklist.txt2 /etc/pihole/blacklist.txt
rm ads_unique.txt

# Below is for Raspberry-Pi. Change it suit with your OS/distro.
sudo gravity.sh
  • To download lists again & recreate another combined blocklist file,
    either run cmd: bash "manual-easylist.sh" Or schedule it in cron.
    ( Do not use the word cmd: as part of a command, use the actual command shown after it. )
  • in MacOS, to add the list in /etc/hosts file you could do something like this to change HOSTS file:
    cmd: sudo nano /private/etc/hosts
    copy-paste from "blacklist.txt" into nano editor in Terminal.
    when changes+editing completes, press [Control]+[X] to exit nano editor, press [Y] to save, and press [Return] to overwrite old content with new changes.
    ( i may later add complete script(s) to do this on MacOS & Debian-GNU/Linux & Windows )
  • Flush DNS on MacOS (more info here) :
    for MacOSX 10.11, 10.12 & later : cmd: sudo killall -HUP mDNSResponder && { say DNS cache has been flushed } || { say Some error occurred, could not flush DNS cache }
    for MacOSX Snow-Leopard : cmd: sudo dscacheutil -flushcache
    for MacOS X Mavericks : cmd: sudo dscacheutil -flushcache ; sudo killall -HUP mDNSResponder
    for MacOS X Mountain Lion, and Lion : cmd: sudo killall -HUP mDNSResponder ; sudo discoveryutil mdnsflushcache
    for MacOSX Yosemite : cmd: sudo discoveryutil mdnsflushcache && { say mdnsflushcache flushed } ; sudo discoveryutil udnsflushcaches && { say udnsflushcaches flushed }
  • Flush DNS (on Windows) : run Command-Prompt with Administrative privilege, & then run cmd: ipconfig /flushdns.
  • Test it:
    MacOS/Unix/Linux usually pre-includes "dig" tool. To use "dig" tool in Windows, get ISC BIND installer for Windows & run it, choose "Tools-only" option.
      cmd: dig www.google.com. +short
      172.217.4.164
    google website was not blocked by /etc/hosts so you see above ip-adrs 172.217.4.164.
    if we add below 2-lines in HOSTS file to test google domain name:
      0.0.0.0 www.google.com
      0.0.0.0 google.com
    then, running dig again will show result like this:
      cmd: dig www.google.com. +short
      Host not found
    Our above test with google domain name is done, So now remove those two google lines from /etc/hosts HOSTS file, or else you'll not be able to visit the site.
  • here is an old discussion page on /etc/host & AdBlocklist.

Upvotes: -2

Related Questions