Uncategorized

Using ‘jq’ to query JSON

Handy little tip, stashing it here so i can remember.

You’re probably familiar with the fact that you can use the command line tool jq to do pretty printing of JSON on the command line, like such:

$ cat file.json  | jq "." | head -n 30
[
  {
    "collection": "state_of_oklahoma",
    "ip_address": "156.110.191.1",
    "port": 3389,
    "protocol": "tcp",
    "service": "RDP",
    "first_seen": null,
    "last_seen": null,
    "timestamp": "2019-05-12T02:02:29+00:00"
  },
  {
    "collection": "state_of_oklahoma",
    "ip_address": "156.110.192.1",
    "port": 3389,
    "protocol": "tcp",
    "service": "RDP",
    "first_seen": null,
    "last_seen": null,
    "timestamp": "2019-05-12T02:02:41+00:00"
  },

But did you know you can use it to query attributes? For instance, if you want to get the ip_address of all items running RDP, you can query like such:

$ cat file.json | jq -c '.[]| select(.service=="RDP") | .ip_address'

This essentially tells jq to iterate through the outer array, selecting any hash with the “service” field set to “RDP, and then print the ip_address field. And the output?

$ cat file.json | jq -c '.[]| select(.service=="RDP") | .ip_address'
"156.110.191.1"
"156.110.192.1"
...

Also, if you want those quotes removed, add a ‘-r’ switch:

$ cat file.json | jq -r -c '.[]| select(.service=="RDP") | .ip_address'

Hope this is helpful, check out the jq tutorial for more info !

You and Your Research

I was recently pointed to a great speech entitled “You and Your Research” given by Richard Hamming of Bell Labs (and Hamming codes!) fame. It’s essentially Hamming giving his insight on how to do great work as a scientist. I think it’s relevant for anyone doing infosec research today.

Here are my key takeaways:

  • Work on important problems.
  • Luck favors the prepared mind.
  • Courage is a characteristic of the successful.
  • Plant acorns to grow oak trees.
  • Follow the greats in your field.
  • Every defect can be looked at as an asset.
  • Knowledge and productivity are like compound interest.
  • Keep track of flaws in your theories.
  • Work on problems you’re committed to.
  • Get emotionally involved, otherwise your subconscious goofs off.
  • Reach out to people outside of your field.
  • Pursue opportunity when its presented.
  • Find and know the important problems in your field.
  • Practice makes perfect.
  • Schedule some dedicated time to make “great thoughts time”
  • Open doors -> more input -> finding the right problems.
  • Zoom out to see the larger problem.
  • You want others to stand on the shoulders of your work.
  • It’s not sufficient to do a job, you have to sell it.
  • Write clearly and well so that people will read it.
  • Learn to give formal talks.
  • Learn to give informal talks.
  • When giving a talk, start slowly and paint a general picture of why its imporant, and give a sketch of what was done.
  • Educate your boss, get other people to ask for what you need.
  • Take advantage of the systems around you to scale yourself.
  • Know thyself & watch thy ego.
  • The appearance of conforming gets you a long way.
  • Don’t spend effort needlessly fighting the system and don’t fool yourself by creating alibis for disappointment.
  • A little extra effort goes a long way with people.

Powerful words to guide your career.