Fq: Extracting Structured Data from Binary Files

Do you remember Jq, which allows you to extract data from any software output and format it in JSON?

Here is the same concept: having data formatted in JSON, but this time extracted from a binary file. Fq can thus display the data in its hex form and “transform” it into JSON, which is practical for outputting data from media such as MP3, MP4, FLAC, JPEG, etc., or even listing the values and functions included in a program.

For example, to output the header of an MP3, you can use the command:

fq '.frames[1].header | tovalue' fichier.mp3

We can also why not, extract the first JPEG image encountered in the binary:

fq 'first(.. | select(format=="jpeg")) | tobytes' file > file.jpeg

This makes it possible to feed databases or websites by extracting data from binary files, network packet captures, etc.

For example, for network frames, we can recover TCP frames that have HTTP GET headers like this (from a PCAP file):

fq'.tcp_connections | grep("GET/.* HTTP/1.?")'file.pcap

…etc., etc.

In short, a tool halfway between Jq and GDB (the debugger).

If you’re interested, the entire doc can be found here.

Mohamed SAKHRI
Mohamed SAKHRI

I'm the creator and editor-in-chief of Tech To Geek. Through this little blog, I share with you my passion for technology. I specialize in various operating systems such as Windows, Linux, macOS, and Android, focusing on providing practical and valuable guides.

Articles: 1378

Newsletter Updates

Enter your email address below and subscribe to our newsletter

Leave a Reply

Your email address will not be published. Required fields are marked *