Today, I’m going to talk about a little tool that’s super useful and will change your life (I know, I never exaggerate): trurl!
So, what is trurl? Well, it’s just a command-line program, inspired by our good old friend tr, that allows you to parse and manipulate URLs. It might sound simple, but let me tell you, it’s super handy and incredibly powerful.
Because URLs aren’t always easy to handle. Between special characters, encodings, query parameters, fragments, there’s enough to make you pull your hair out. And if it’s poorly implemented, it can bring its share of security vulnerabilities if you’re not careful. But with trurl, there’s no hassle! It uses the URL parser from the libcurl library, which is super robust and proven, so you can manipulate your URLs with complete peace of mind.
With just a simple little command, you can, for example:
- Change the domain name of a URL
- Modify the port
- Extract the path
- Add query parameters
- Sort parameters alphabetically
- Remove certain parameters
- Encode/decode special characters
- Reformat the URL in JSON
- And a bunch of other things!
For instance, if I want to change the domain name of a URL, I just do:
trurl --url https://tech2geek.net --set host=example.com
https://example.com/And bam, it’s done! Easy, right? No more dealing with 3km-long Regex or reinventing the wheel every time.
What I also like is that you can use trurl interactively by passing URLs to it one by one, or process an entire file of URLs at once. It even handles spaces and other special characters without a hitch.
For example, to add a parameter to an existing URL:
trurl --url "https://example.com?param1=value1" --add-query "param2=value2"
https://example.com?param1=value1¶m2=value2Or, to remove a specific parameter:
trurl --url "https://example.com?param1=value1¶m2=value2" --remove-query "param1"
https://example.com?param2=value2Installation is super easy too. On most Linux distros, you should be able to install it directly from your preferred package manager. Otherwise, the source code is available on GitHub.
It has allowed me to get rid of tons of messy hand-crafted code in scripts and replace it with clean trurl commands. For example, I had a script that needed to retrieve the RSS feed of a site, filter certain URL parameters, and add a tracker for stats. Before, I did that with ugly Regex and concatenations; now, with a few trurl commands, it’s done!
curl "https://example.com/feed.xml" | trurl --query-sort --query-exclude "utm_*" --add-query "tracker_id=tech2geek"However, be aware that trurl has some limitations to know about. For example, it doesn’t handle redirects by itself; you’ll need to use curl if necessary. And in some tricky cases with exotic characters, there might be surprises. So be careful, always test before pushing to production!
As a bonus tip, trurl has a --accept-space option that’s very handy when your URLs contain spaces. And --sort-query is great for putting your parameters in order.
See you soon on Tech To Geek.
"Because of the Google update, I, like many other blogs, lost a lot of traffic."
Join the Newsletter
Please, subscribe to get our latest content by email.
