Just a note that the older / dead blog at https://blog.0x0e.org has been merged into this stream.
Category Archives: Uncategorized
Using ‘jq’ to query JSON
jq is great for CLI manipulation of JSON data! Handy little bit of knowledge here, stashing so i can remember.
You’re probably familiar with the fact that you can use the command line tool jq to do pretty printing of JSON on the command line, like this:
$ cat file.json | jq "." | head -n 30 [ { "collection": "state_of_oklahoma", "ip_address": "156.110.191.1", "port": 3389, "protocol": "tcp", "service": "RDP", "first_seen": null, "last_seen": null, "timestamp": "2019-05-12T02:02:29+00:00" }, { "collection": "state_of_oklahoma", "ip_address": "156.110.192.1", "port": 3389, "protocol": "tcp", "service": "RDP", "first_seen": null, "last_seen": null, "timestamp": "2019-05-12T02:02:41+00:00" },
But did you know you can use it to query attributes? For instance, if you want to get the ip_address of all items running RDP, you can query like such:
$ cat file.json | jq -c '.[]| select(.service=="RDP") | .ip_address'
This essentially tells jq to iterate through the outer array, selecting any hash with the “service” field set to “RDP, and then print the ip_address field. And the output?
$ cat file.json | jq -c '.[]| select(.service=="RDP") | .ip_address' "156.110.191.1" "156.110.192.1" ...
Also, if you want those quotes removed, add a ‘-r’ switch:
$ cat file.json | jq -r -c '.[]| select(.service=="RDP") | .ip_address'
Hope this is helpful, check out the jq tutorial for more info !
You and Your Research
I was recently pointed to a great speech entitled “You and Your Research” given by Richard Hamming of Bell Labs (and Hamming codes!) fame. It’s essentially Hamming giving his insight on how to do great work as a scientist. I think it’s relevant for anyone doing infosec research today.
Here are my key takeaways:
- Work on important problems.
- Luck favors the prepared mind.
- Courage is a characteristic of the successful.
- Plant acorns to grow oak trees.
- Follow the greats in your field.
- Every defect can be looked at as an asset.
- Knowledge and productivity are like compound interest.
- Keep track of flaws in your theories.
- Work on problems you’re committed to.
- Get emotionally involved, otherwise your subconscious goofs off.
- Reach out to people outside of your field.
- Pursue opportunity when its presented.
- Find and know the important problems in your field.
- Practice makes perfect.
- Schedule some dedicated time to make “great thoughts time”
- Open doors -> more input -> finding the right problems.
- Zoom out to see the larger problem.
- You want others to stand on the shoulders of your work.
- It’s not sufficient to do a job, you have to sell it.
- Write clearly and well so that people will read it.
- Learn to give formal talks.
- Learn to give informal talks.
- When giving a talk, start slowly and paint a general picture of why its imporant, and give a sketch of what was done.
- Educate your boss, get other people to ask for what you need.
- Take advantage of the systems around you to scale yourself.
- Know thyself & watch thy ego.
- The appearance of conforming gets you a long way.
- Don’t spend effort needlessly fighting the system and don’t fool yourself by creating alibis for disappointment.
- A little extra effort goes a long way with people.
Powerful words to guide your career.
Pentestify.
i’m over here now.
password hangover
Just saw the Hangover 2. – funny (and true) bit on passwords…
as an international drug dealer tranfers money between accounts:
“your password is bologna1?”
“it used to be bologna, but they make you include a stupid number now”
*facepalm*
sadly (… or happily, depending on your perspective :] ) , weak passwords are still common…. metasploit has some awesome modules to test passwords:
jcran@disko:~/framework/modules$ find . |grep _login | grep -v svn
- ./auxiliary/scanner/snmp/snmp_login.rb
- ./auxiliary/scanner/mssql/mssql_login.rb
- ./auxiliary/scanner/postgres/postgres_login.rb
- ./auxiliary/scanner/http/wordpress_login_enum.rb
- ./auxiliary/scanner/http/axis_login.rb
- ./auxiliary/scanner/http/tomcat_mgr_login.rb
- ./auxiliary/scanner/http/http_login.rb
- ./auxiliary/scanner/http/frontpage_login.rb
- ./auxiliary/scanner/ftp/ftp_login.rb
- ./auxiliary/scanner/vnc/vnc_login.rb
- ./auxiliary/scanner/ssh/ssh_login_pubkey.rb
- ./auxiliary/scanner/ssh/ssh_login.rb
- ./auxiliary/scanner/telnet/telnet_login.rb
- ./auxiliary/scanner/sap/sap_mgmt_con_brute_login.rb
- ./auxiliary/scanner/lotus/lotus_domino_login.rb
- ./auxiliary/scanner/mysql/mysql_login.rb
- ./auxiliary/scanner/rservices/rsh_login.rb
- ./auxiliary/scanner/rservices/rlogin_login.rb
- ./auxiliary/scanner/rservices/rexec_login.rb
- ./auxiliary/scanner/smb/smb_login.rb
- ./auxiliary/scanner/oracle/isqlplus_login.rb
- ./auxiliary/scanner/oracle/oracle_login.rb
- ./auxiliary/fuzzers/tds/tds_login_username.rb
- ./auxiliary/fuzzers/tds/tds_login_corrupt.rb
- ./auxiliary/fuzzers/smb/smb_ntlm1_login_corrupt.rb
- ./auxiliary/admin/oracle/oracle_login.rb
- ./exploits/windows/imap/mailenable_login.rb
- ./exploits/windows/imap/mercury_login.rb
- ./exploits/windows/http/hp_power_manager_login.rb
pianobar is epic!
@mrbarrett just pointed me at pianobar, a command-line client for pandora. It gets rid of the need for flash player, and allows you to bypass some of the limitations of the web client (you can skip more than 5 songs!).
To install (on Ubuntu):
sudo apt-get install git-core libao-dev libfaad-dev libmad0-dev git clone git://github.com/PromyLOPh/pianobar.git cd pianobar make sudo make install
enjoy!
Loading only the Metasploit modules you use
The framework is taking quite a while to load on my machine these days, so i decided to stop loading all modules by default, and load only those modules i need. Here’s the process:
Simply comment the module-loading lines in the framework file lib/msf/base/simple/framework.rb, so :
if (Msf::Config.module_directory) framework.modules.add_module_path(Msf::Config.module_directory) end
becomes
#if (Msf::Config.module_directory) #framework.modules.add_module_path(Msf::Config.module_directory) #end
Or you can apply this patch: http://www.0x0e.org/x/framework-no-default-mods.patch
Alos make sure to remove the ~/.msf3/modcache directory.
Then, mirroring the framework modules directory structure, copy the modules you’d like to load into your .msf3/modules directory. For example, if you wanted to load only the psexec module and the reverse_tcp payload, copy
- modules/exploits/windows/smb/psexec.rb into ~/.msf3/modules/exploits/windows/smb/psexec.rb
- modules/payloads/stagers/windows/reverse_tcp.rb into ~/.msf3/modules/payloads/stagers/windows/reverse_tcp.rb
You should now see a load-time speed improvement on the order of:
Before:
<pre>jcran@disko:~/framework$ time ./msfconsole -r exit.rc | | _) | __ `__ \ _ \ __| _` | __| __ \ | _ \ | __| | | | __/ | ( |\__ \ | | | ( | | | _| _| _|\___|\__|\__,_|____/ .__/ _|\___/ _|\__| _| =[ metasploit v3.5.1-dev [core:3.5 api:1.0] + -- --=[ 630 exploits - 310 auxiliary + -- --=[ 215 payloads - 27 encoders - 8 nops =[ svn r10985 updated today (2010.11.11) resource (exit.rc)> exit resource (exit.rc)> exit real 0m42.750s user 0m40.710s sys 0m0.820s</pre>
After:
<pre>jcran@disko:~/framework$ time ./msfconsole -r exit.rc o 8 o o 8 8 8 ooYoYo. .oPYo. o8P .oPYo. .oPYo. .oPYo. 8 .oPYo. o8 o8P 8' 8 8 8oooo8 8 .oooo8 Yb.. 8 8 8 8 8 8 8 8 8 8 8. 8 8 8 'Yb. 8 8 8 8 8 8 8 8 8 8 `Yooo' 8 `YooP8 `YooP' 8YooP' 8 `YooP' 8 8 ..:..:..:.....:::..::.....::.....:8.....:..:.....::..::..: ::::::::::::::::::::::::::::::::::8::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::: =[ metasploit v3.5.1-dev [core:3.5 api:1.0] + -- --=[ 1 exploits - 0 auxiliary + -- --=[ 1 payloads - 0 encoders - 0 nops =[ svn r10985 updated today (2010.11.11) resource (exit.rc)> exit resource (exit.rc)> exit real 0m12.232s user 0m11.340s sys 0m0.510s</pre>
Not huge, but definitely an improvement.
IP List to KML generator (Create a google map from a list of IPs)
Pretty simple, it takes a file with a list of ips, one/line and generates a kml file. Very handy if you’re working on a large pentest and want to track down (and visualize) where a particular host is located. It uses the Yahoo GeoIP API to grab location data.
#!/usr/bin/ruby require 'net/http' require 'rexml/document' include REXML def getAddress(ip) #takes an ip and returns an xml blob with city/state # example: http://ipinfodb.com/ip_query.php?ip=65.23.23.33 url = "http://ipinfodb.com/ip_query.php?ip=" + ip #puts "DEBUG: URL: #{url.to_s}" resp = Net::HTTP.get(URI.parse(url)) #print "DEBUG: got " + resp return resp end def getCoordinates(address) #takes a hash with city, state address and returns a hash w/ coords url = "http://local.yahooapis.com/MapsService/V1/geocode" params = { "appid" => "GwLDY.bV34HH7gkBDs97p_5U5P_tBfXBnfDyYFwpTRLwZDEvgj8BOQqws1JOCFPyhTQR", "street" => "", "city" => address["city"], "state" => address["state"] } #puts "DEBUG: URL: #{url.to_s}" resp = Net::HTTP.post_form(URI.parse(url), params) resp_text = resp.body #print "DEBUG: got " + resp_text return resp_text end def parseAddress(xml) #takes an xml blob with city / state & returns a hash with address,city,state doc = Document.new xml root = doc.root city = root.elements["City"].get_text.to_s state = root.elements["RegionName"].get_text.to_s country = root.elements["CountryCode"].get_text.to_s #puts "DEBUG: city: " + city #puts "DEBUG: state: " + state #puts "DEBUG: country: " + country toReturn = Hash["city" => city, "state" => state, "country" => country] return toReturn end def parseCoordinates(xml) #takes an xml blob with coordinates & returns a hash with long/lat doc = REXML::Document.new xml root = doc.root long = REXML::XPath.first( doc, "//Longitude" ).get_text.to_s lat = REXML::XPath.first( doc, "//Latitude" ).get_text.to_s toReturn = Hash["long" => long, "lat" => lat] #puts "DEBUG: long: " + long #puts "DEBUG: lat: " + lat return toReturn end def genKML(ips) kml = "" kml = kml + "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" kml = kml + "<kml xmlns=\"http://www.opengis.net/kml/2.2\">\n" kml = kml + "<Document>\n" ips.each do |ip| ip = ip.to_s.chomp kmlplacemark = mip(ip,"error.log") if kmlplacemark.to_s != "" then # puts "DEBUG: adding non-blank placemark" + kmlplacemark kml = kml + kmlplacemark else # puts "DEBUG: unable to map ip: " + ip + "\n" end end kml = kml + "</Document>\n" kml = kml + "</kml>\n" end def genPlacemark(ip,address,coordinates) xml = "" xml = xml + " <Placemark>\n" xml = xml + " <name>" + ip + "</name>\n" xml = xml + " <description>" xml = xml + address["city"] + ", " xml = xml + address["state"] + ", " xml = xml + address["country"] xml = xml + "</description>\n" xml = xml + " <Point>\n" xml = xml + " <coordinates>" + coordinates["long"] + "," + coordinates["lat"] + ",0</coordinates>\n" xml = xml + " </Point>\n" xml = xml + " </Placemark>\n" end def mip(ip,errorfile) begin if (ip != "") then xmlAddress = getAddress(ip) objAddress = parseAddress(xmlAddress) if (objAddress["state"] != "") then xmlCoordinates = getCoordinates(objAddress) objCoordinates = parseCoordinates(xmlCoordinates) kmlplacemark = genPlacemark(ip,objAddress,objCoordinates) else File.open(errorfile, 'w') {|f| f.write(ip) } end end rescue kmlplacemark = "" end return kmlplacemark end def mips(file) counter = 0 ips = Array.new File.open(file, "r") do |infile| while (line = infile.gets) #puts "mapping #{counter}: #{line}" ips[counter] = line counter = counter + 1 end end kml = genKML(ips) return kml end kml = mips(ARGV[0]) out = File.new(ARGV[0]+".kml", "w") out.puts kml