Removing URL Parameters

Here are more efficient methods to filter URL parameters, but they may break website functionality. A more detailed explanation of these can be read from AdGuard.

If you want to block all URL tracking parameters for one specific website, such as TELUS, add this static filter rule:

||telus.com^$removeparam

If you want to whitelist one parameter on eBay but remove everything else, use this instead:

||ebay.com^$removeparam=~_nkw

If you just want to block all URL parameters, period:

$removeparam
3 Likes

Just throwing an idea out there but maybe one could remove all URL parameters except for whitelisted domains with legitimate parameters i.e. where the domain does actually support some kind of lookup or query via GET parameters (query string) and you trust the domain and you know what the legitimate parameters of that query are.

4 Likes

See my post above yours for a static filter rule with eBay showcasing that exact situation. Also, if you are interested in the nitty gritty details of how most of eBay’s tracking URL parameters work, see these two articles below.

https://developer.ebay.com/api-docs/buy/static/ref-epn-link.html

https://partnerhelp.ebay.com/helpcenter/s/article/What-are-the-parameters-of-an-EPN-link

2 Likes

Indeed, much like the recommended method for input sanitation is to use an allowlist rather than a blocklist, I think the same should apply here if there is concern from any person.

It is a common mistake to use block list validation in order to try to detect possibly dangerous characters and patterns like the apostrophe ' character, the string 1=1, or the <script> tag, but this is a massively flawed approach as it is trivial for an attacker to bypass such filters.
Allow list validation is appropriate for all input fields provided by the user. Allow list validation involves defining exactly what IS authorized, and by definition, everything else is not authorized.

1 Like

Here is a further manual refinement of eBay’s static filter rules, which now uses regular expression:

||ebay.com^$removeparam=~/(_nkw|_pgn|_sop|blrs|ipg)/

As for an explanation of the whitelisted URL parameters listed so far:

  • _nkw - Search
  • _pgn - Page Number
  • _sop - Sort Order Page
  • blrs - Best Listings Results
  • ipg - Items Per Page
1 Like

When will people learn that security through obscurity is meaningless? :joy:

Awesome find. Mind sharing the filters as an export from the addon when you feel you have a fairly robust list?

1 Like

Well my strict security practices already presanitize my web browsing experience: I only have Matomo Campaign Tracking; Urchin Tracking Module; and eBay as justified entries for URL parameters. Even then, it is clear that the way I use eBay is pretty limited already, and that I do not use all of the available search filters; what I use right now is enough for my needs, but may be subject to change in the future.

1 Like

Here is another manual refinement of eBay’s static filter rules.

||ebay.com^$removeparam=~/(_nkw|_pgn|_sop|action|blrs|cartid|guestCheckoutEligible|ipg|item|srt)/

New URL parameters and their functions:

  • action - Action, used with the value “create” during the “Buy It Now → Check out as guest” process; bypasses the other URL parameters and processes mentioned below
  • cartid - Cart ID, used during the “Go to checkout → Continue as guest” process
  • guestCheckoutEligible - The value “true” is used to authorize the “Go to checkout → Continue as guest” process
  • item - Item ID, used during the “Add to cart” process
  • srt - ? (maybe something like “Seller Reference Tracker”), required along with the item URL parameter during the “Add to cart” process

This will allow you to buy the item(s) immediately or to add items to your cart for checkout. I have not had any justification to purchase anything from eBay yet to confirm and verify that the entire checkout process works, but I will do so when an opportunity occurs later in the future.

1 Like

The eBay checkout process works with these static filter rules, although the URL states the transaction has succeeded while the HTML content itself states there is an error. I have received a confirmation email nonetheless, but I may continue to add more whitelisted URL parameters in the future for a more pleasant shopping experience as well as to reassure confidence with the entire process.

Here is my updated eBay static filter rules:

||ebay.com^$removeparam=~/(_nkw|_pgn|_sop|action|blrs|cartid|_dmd|guestCheckoutEligible|ipg|item|itemId|itemid|LH_All|LH_Auction|LH_BIN|LH_ItemCondition|LH_PrefLoc|srt|transId|transid)/

Pre-transaction:

  • _dmd - Allows listings to change between List and Grid View.
  • LH_ALL - Left Hand All, presents all listings.
  • LH_Auction - Left Hand Auction, presents all auction listings.
  • LH_BIN - Left Hand Buy It Now, presents all Buy It Now listings.
  • LH_ItemCondition - Left Hand Item Condition, presents specified item conditions (New, Used, Not Specified, etc.)
  • LH_PrefLoc - Left Hand Preferred Location, presents listings specified by country (Canada Only, North America, Worldwide, etc.)

Post-transaction:

  • itemId - Item ID, used when viewing your order details.
  • transId - Transaction ID, used when viewing your order details.
  • itemid - Item ID, used when viewing your order’s tracking information.
  • transid - Transaction ID, used when viewing your order’s tracking information.

This is good enough for now until I order another product in the future. I am aware that hash is somehow ignoring uBlock Origin, so I will eventually get around to addressing it.

Worth a mention:
Open-source browser add-on ClearURLs: ClearURLs – Get this Extension for 🦊 Firefox (en-US)

1 Like

I took a look at their documentation and their rule catalog files.

To briefly compare and contrast between the methodologies:

  • I use an allowlist, whereas they use a blocklist with exceptions.
  • My rules are for services outside of Big Tech, whereas their rules are designed for users who directly continue to use Big Tech services (Amazon, Bing, Facebook, Google, Reddit, X, etc).
  • My eBay allowlist is well defined and fairly comprehensive for guest checkout, whereas their eBay blocklist only blocks 4 URL parameters.
    "ebay": {
      "urlPattern": "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?ebay(?:\\.[a-z]{2,}){1,}",
      "rules": [
        "_trkparms",
        "_trksid",
        "_from",
        "hash"
      ],
      "redirections": [
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?rover\\.ebay(?:\\.[a-z]{2,}){1,}\\/rover.*mpre=([^&]*)"
      ]
    },
  • My wildcard filter rules have no exceptions, but their global filter rules allow tracking exceptions:
"exceptions": [
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?matrix\\.org\\/_matrix\\/",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?(?:cloudflare\\.com|prismic\\.io|tangerine\\.ca|gitlab\\.com)",
        "^https?:\\/\\/myaccount.google(?:\\.[a-z]{2,}){1,}",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?gcsip\\.(?:com|nl)[^?]*\\?.*?&?ref_?=.",
        "^https?:\\/\\/[^/]+/[^/]+/[^/]+\\/-\\/refs\\/switch[^?]*\\?.*?&?ref_?=.",
        "^https?:\\/\\/bugtracker\\.[^/]*\\/[^?]+\\?.*?&?ref_?=[^/?&]*",
        "^https?:\\/\\/comment-cdn\\.9gag\\.com\\/.*?comment-list.json\\?",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?battle\\.net\\/login",
        "^https?:\\/\\/blizzard\\.com\\/oauth2",
        "^https?:\\/\\/kreditkarten-banking\\.lbb\\.de",
        "^https?:\\/\\/www\\.tinkoff\\.ru",
        "^https?:\\/\\/www\\.cyberport\\.de\\/adscript\\.php",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?tweakers\\.net\\/ext\\/lt\\.dsp\\?.*?(?:%3F)?&?ref_?=.",
        "^https?:\\/\\/git(lab)?\\.[^/]*\\/[^?]+\\?.*?&?ref_?=[^/?&]*",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?amazon(?:\\.[a-z]{2,}){1,}\\/message-us\\?",
        "^https?:\\/\\/authorization\\.td\\.com",
        "^https?:\\/\\/support\\.steampowered\\.com",
        "^https?:\\/\\/privacy\\.vakmedianet\\.nl\\/.*?ref=",
        "^https?:\\/\\/sso\\.serverplan\\.com\\/manage2fa\\/check\\?ref=",
        "^https?:\\/\\/login\\.meijer\\.com\\/.*?\\?ref=",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?facebook\\.com\\/(?:login_alerts|ajax|should_add_browser)/",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?facebook\\.com\\/groups\\/member_bio\\/bio_dialog\\/",
        "^https?:\\/\\/api\\.taiga\\.io",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?gog\\.com\\/click\\.html",
        "^https?:\\/\\/login\\.progressive\\.com",
        "^https?:\\/\\/www\\.sephora\\.com\\/api\\/",
        "^https?:\\/\\/www\\.contestgirl\\.com",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?agenciatributaria\\.gob\\.es",
        "^https?:\\/\\/login\\.ingbank\\.pl",
        "^wss?:\\/\\/(?:[a-z0-9-]+\\.)*?zoom\\.us",
        "^https?:\\/\\/api\\.bilibili\\.com",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?onet\\.pl\\/[^?]*\\?.*?utm_campaign=.",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?stripe\\.com\\/[^?]+.*?&?referrer=[^/?&]*",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?lichess\\.org\\/login.*?&?referrer=.*?",
        "^https?:\\/\\/like.co\\/api\\/like\\/likebutton\\/[^?]+.*?&?referrer=[^/?&]*",
        "^https?:\\/\\/button.like.co\\/in\\/.*?&?referrer=[^/?&]*",
        "^https?:\\/\\/www\\.mma\\.go\\.kr",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?github\\.com",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?billiger\\.de\\/.*?mc=",
        "^https?:\\/\\/(?:[a-z0-9-]+\\.)*?\\.youtrack\\.cloud"
      ]
    },
2 Likes

Another update:

||ebay.com^$removeparam=~/(_nkw|_pgn|_sop|_ssn|action|blrs|cartid|_dmd|guestCheckoutEligible|ipg|item|itemId|itemid|LH_All|LH_Auction|LH_BIN|LH_ItemCondition|LH_PrefLoc|sessionid|srt|token|transId|transid)/

Definitions:

  • _ssn - Seller Number, used when looking at other products by the same seller.
  • token - Token, used as a unique identifier when updating email preferences (such as unsubscribing).

Here is an untested value:

  • sessionid - Session ID, very likely used after completing the checkout process to show your order.

I watched someone else make an order on eBay and noticed the sessionid URL parameter after completing the checkout process, so I am assuming that it is required to display your order. Actual testing is needed to confirm this.

Small update:

||ebay.com^$removeparam=~/(_nkw|_pgn|_sop|_ssn|action|blrs|cartid|_dmd|guestCheckoutEligible|ipg|item|itemId|itemid|LH_|sessionid|srt|token|transId|transid)/

This consolidates and whitelists all LH URL parameters, so now you can use the remaining search filters; I assume that this is the only purpose for them. I could optimize _, item and trans, but I need to carefully examine the URL to determine it is safe to do so.

I may order something with guest checkout sometime soon to confirm that these static filter rules continue to work as intended.

I figured out the issue with the hash URL parameter being ignored by uBlock Origin: since hash includes item as its value, it gets excluded. Here is a temporary solution for that by using two static filters:

||ebay.*^$removeparam=~/(_nkw|_pgn|_sop|_ssn|action|blrs|cartid|_dmd|guestCheckoutEligible|ipg|item|itemId|itemid|LH_|sessionid|srt|token|transId|transid)/
||ebay.*^$removeparam=hash

Also, they include a wildcard, so all potential eBay TLDs (.com, .ca, .co.uk, com.au, etc.) are now affected by these rules.

I have fully confirmed that these uBlock Origin static filter rules work with eBay’s guest checkout. For reference, here are all of the static filter rules from this thread I currently use:

*$removeparam=/^mtm_/
*$removeparam=/^utm_/
||ebay.*^$removeparam=~/(_dmd|_nkw|_pgn|_sop|_ssn|action|blrs|cartid|guestCheckoutEligible|ipg|item|itemId|itemid|LH_|sessionid|srt|token|transId|transid)/
||ebay.*^$removeparam=hash

I have been working on removing URL parameters from TELUS:

||telus.com*^$removeparam=~/(category|client_id|next|redirect_uri|response_type|slugs)/

Definitions:

  • category - Category, used as a value after eli-modal.js (“Eligibility Modal”) with the Offers API to display services.
  • client_id - Client ID, used for authentication with My TELUS.
  • locale - Locale, used with the Legals API to display legal terms and conditions.
  • next - Next URL, used for post-URL redirection for eli-modal.js with TELUS Business Internet plans.
  • redirect_uri - Redirect URI, used for post-URL redirection for logging in to My TELUS.
  • response_type - Response Type, value is code, used for authentication with My TELUS.
  • slugs - Used with the Legals API to display legal terms and conditions.

I have not authenticated with My TELUS just yet using these static filter rules, so I will continue updating it as time and space permits.

OK, I’ve looked into your method of cleaning URLs just a little bit. For me, it is outside of my current ability to zero in on the perfect level of blocking the URL tracking data the way you do it. I definitely need a curated (open source) list of URL tracking to be cleaned for me. At least for now, that is best for me.

I do use youtube and sometimes i use google search, ebay, amazon, and even twitter just a very small amount of the time. I actually mostly only use FreeTube over RiseupVPN for my youtube consumption. I also use Odysee and Bitchute also over VPN for other video content.

My thinking about these sites is that I am already logged in, and if ClearURLs blocks some, or most of their URL tracking, then it works good enough for me. I don’t use social media at all, unless you count the once a week I click on twitter. I don’t post on twitter, and I hardly ever comment on twitter. I only singned up after Elon took over twitter. I’m no fanboy of Elon, and I don’t trust him, or twitter, it’s just the only social media that I even use at all.

I found another URL cleaner…

It is also open source so I added the extension next to CleanURLs as a second level of cleaning.

Until I learn more, these two cleaners will be better than me trying to clean by myself.

One thing I considered was to copy their definitions, modify them and post on my own domain and link to them instead. Maybe I could remove their opinions for my own, but at least have a starting point that at least works.

1 Like

I came across this: The “Copy Link Without Site Tracking” is smarter in Firefox 126. It’s now able to strip tracking parameters from nested URLs as well as a further 300 smaller elements within tracking URLs, including ones from major shopping websites.
(Enhanced Tracking Protection in Firefox for desktop | Firefox Help). Is that more or less useful or private? Maybe their filters are available somewhere…

1 Like

@arkenfox makes arguments against ClearURLs and Neat URL in their user.js wiki:

Instead, they suggest enabling AdGuard URL Tracking Protection, which can be found in uBlock Origin’s dashboard → Filter lists → Privacy. The rules themselves can be found here:

https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_17_TrackParam/filter.txt

Here is more general information about AdGuard’s filters:

Reasonably easy to parse:

{
    "categories": [
        { "name": "Action Map", "params": ["action_object_map", "action_ref_map", "action_type_map"]},
        { "name": "AliExpress.com", "params": ["aff_platform", "aff_trace_key", "algo_expid@*.aliexpress.*", "algo_pvid@*.aliexpress.com", "btsid@*.aliexpress.com", "expid@*.aliexpress.com", "initiative_id@*.aliexpress.com", "scm_id@*.aliexpress.com", "spm@*.aliexpress.com", "ws_ab_test*.aliexpress.com"]},
        { "name": "Amazon", "params": ["_encoding@amazon.*", "ascsubtag@amazon.*", "pd_rd_*@amazon.*", "pf@amazon.*", "pf_rd_*@amazon.*", "psc@amazon.*", "ref_@amazon.*", "tag@amazon.*"]},
        { "name": "Bilibili.com", "params": ["callback@bilibili.com"]},
        { "name": "Bing", "params": ["cvid@bing.com", "form@bing.com", "pq@bing.com", "qs@bing.com", "sc@bing.com", "sk@bing.com", "sp@bing.com"]},
        { "name": "Campaign tracking (Adobe Analytics)", "params": ["sc_cid"]},
        { "name": "Campaign tracking (Adobe Marketo)", "params": ["mkt_tok"]},
        { "name": "Campaign tracking (Amazon Kendra)", "params": ["trk", "trkCampaign"]},
        { "name": "Campaign tracking (at)", "params": ["at_campaign", "at_custom*", "at_medium"]},
        { "name": "Campaign tracking (Change.org)", "params": ["guest@change.org", "recruited_by_id@change.org", "recruiter@change.org", "short_display_name@change.org", "source_location@change.org"]},
        { "name": "Campaign tracking (DPG Media)", "params": ["dpg_*"]},
        { "name": "Campaign tracking (Google Analytics, ga)", "params": ["ga_*", "gclid", "gclsrc"]},
        { "name": "Campaign tracking (Humble Bundle)", "params": ["hmb_campaign", "hmb_medium", "hmb_source"]},
        { "name": "Campaign tracking (IBM Acoustic Campaign)", "params": ["spJobID", "spMailingID", "spReportId", "spUserID"]},
        { "name": "Campaign tracking (itm)", "params": ["itm_*"], "docs": "https://www.parse.ly/help/post/4843/campaign-data-tracking/"},
        { "name": "Campaign tracking (Omniture)", "params": ["s_cid"], "docs": "https://moz.com/community/q/omniture-tracking-code-urls-creating-duplicate-content"},
        { "name": "Campaign tracking (Oracle Eloqua)", "params": ["assetId", "assetType", "campaignId", "elqTrack", "elqTrackId", "recipientId", "siteId"]},
        { "name": "Campaign tracking (MailChimp)", "params": ["mc_cid", "mc_eid"], "docs": "https://www.learndigitaladvertising.com/solved-why-how-to-remove-mc_cid-and-mc_eid-from-google-analytics/"},
        { "name": "Campaign tracking (Matomo/Piwik)", "params": ["mtm_*", "pk_*"]},
        { "name": "Campaign tracking (ns)", "params": ["ns_*"]},
        { "name": "Campaign tracking (sc)", "params": ["sc_campaign", "sc_channel", "sc_content", "sc_country", "sc_geo", "sc_medium", "sc_outcome"]},
        { "name": "Campaign tracking (stm)", "params": ["stm_*"]},
        { "name": "Campaign tracking (utm)", "params": ["nr_email_referer", "utm_*"]},
        { "name": "Campaign tracking (Vero)", "params": ["vero_conv", "vero_id"], "docs": "https://help.getvero.com/articles/conversion-tracking.html"},
        { "name": "Campaign tracking (Yandex)", "params": ["_openstat", "yclid"], "docs": "https://yandex.com/support/direct/statistics/url-tags.html"},
        { "name": "Campaign tracking (others)", "params": ["c_id", "campaign_id", "Campaign", "cmpid", "mbid", "ncid"], "docs": "https://www.parse.ly/help/post/4843/campaign-data-tracking/"},
        { "name": "Caseking.de", "params": ["campaign@caseking.de", "sPartner@caseking.de"]},
        { "name": "Ebay", "params": ["hash@ebay.*", "_trkparms@ebay.*", "_trksid@ebay.*", "amdata@ebay.*", "epid@ebay.*", "hash@ebay.*", "var@ebay.*"]},
        { "name": "Etsy", "params": ["click_key@etsy.com", "click_sum@etsy.com", "organic_search_click@etsy.com", "ref@etsy.com"]},
        { "name": "Facebook", "params": ["fb_action_ids", "fb_action_types", "fb_ref", "fb_source", "fbclid", "hrc@facebook.com", "refsrc@facebook.com"]},
        { "name": "Google", "params": ["ei@google.*", "gs_gbg@google.*", "gs_l", "gs_lcp@google.*", "gs_mss@google.*", "gs_rn@google.*", "gws_rd@google.*", "sei@google.*", "ved@google.*"]},
        { "name": "Hubspot", "params": ["_hsenc", "_hsmi", "__hssc", "__hstc", "hsCtaTracking"]},
        { "name": "IMDb", "params": ["pf_rd_*@imdb.com", "ref_@imdb.com"]},
        { "name": "LinkedIn", "params": ["eBP@linkedin.com", "lgCta@linkedin.com", "lgTemp@linkedin.com", "lipi@linkedin.com", "midSig@linkedin.com", "midToken@linkedin.com", "recommendedFlavor@linkedin.com", "refId@linkedin.com", "trackingId@linkedin.com", "trk@linkedin.com", "trkEmail@linkedin.com"]},
        { "name": "Medium", "params": ["_branch_match_id@medium.com", "source@medium.com"]},
        { "name": "SourceForge.net", "params": ["position@sourceforge.net", "source@sourceforge.net"]},
        { "name": "Spotify", "params": ["context@open.spotify.com", "si@open.spotify.com"]},
        { "name": "TikTok", "params": ["_d@tiktok.com", "checksum@tiktok.com", "is_copy_url@tiktok.com", "is_from_webapp@tiktok.com", "language@tiktok.com", "preview_pb@tiktok.com", "sec_user_id@tiktok.com", "sender_device@tiktok.com", "sender_web_id@tiktok.com", "share_app_id@tiktok.com", "share_link_id@tiktok.com", "share_item_id@tiktok.com", "source@tiktok.com", "timestamp@tiktok.com", "tt_from@tiktok.com", "u_code@tiktok.com", "user_id@tiktok.com"]},
        { "name": "Twitch.tv", "params": ["tt_content", "tt_medium"]},
        { "name": "Twitter", "params": ["cxt@*.twitter.com", "ref_*@*.twitter.com", "s@*.twitter.com", "t@*.twitter.com", "twclid"]},
        { "name": "Yahoo", "params": ["guccounter@*.yahoo.com", "soc_src", "soc_trk"]},
        { "name": "Yandex", "params": ["lr@yandex.*", "redircnt@yandex.*"]},
        { "name": "YouTube.com", "params": ["feature@youtube.com", "kw@youtube.com"]},
        { "name": "Zeit.de", "params": ["wt_mc", "wt_zmc"]}
    ]
}

A total of 44 entries, not updated since August 14th, 2022:

Here is a simple comparison between Neat URL and AdGuard for eBay static filter rules:

Neat URL:

        { "name": "Ebay", "params": ["hash@ebay.*", "_trkparms@ebay.*", "_trksid@ebay.*", "amdata@ebay.*", "epid@ebay.*", "hash@ebay.*", "var@ebay.*"]},

AdGuard:

! eBay tracking parameters
||www.ebay.$removeparam=ssspo
||www.ebay.$removeparam=sssrc
||www.ebay.$removeparam=ssuid
||www.ebay.$removeparam=mkevt
||www.ebay.$removeparam=mkcid
||www.ebay.$removeparam=_trkparms
||www.ebay.$removeparam=_trksid
||www.ebay.$removeparam=amdata
||www.ebay.$removeparam=mkrid
||www.ebay.$removeparam=campid

Conclusion:

  • Neat URL redundantly repeats hash twice, but has epid and var as unique rules.
  • AdGuard covers the rest of Neat URL’s rules, whle also adding sssrc, ssuid, mkevt, mkcid, mkrid, and campid.
  • Both use blocklists.

For reference, compare it to my original post when I was just starting to learn about static filter rule syntax:

Then compare that to the present, where I use an allowlist:

Here you go:

ETP is unrelated to removing URL parameters, but it is very relevant for blocking trackers.

https://searchfox.org/mozilla-release/source/toolkit/components/antitracking/StripOnShareLists/LGPL/StripOnShareLGPL.json

Their eBay static filter rules:

  "ebay": {
    "queryParams": [
      "_trkparms",
      "mkcid",
      "_trksid",
      "mkevt",
      "amdata",
      "ssuid",
      "mkrid",
      "campid",
      "sssrc",
      "ssspo",
      "_from",
      "hash"
    ],
1 Like