-
Notifications
You must be signed in to change notification settings - Fork 14k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
elasticsearch enum module enhancements #18310
Conversation
@jmartin-tech I believe this is more of a 'gather' module than a 'scanner' at this point, and think it should be moved, but wanted to get other's thoughts. |
I would agree, this makes sense as |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I left a bunch of comments mostly related to newish patterns in MSF. The three themes were:
When making a request with send_request_cgi
(or raw really):
#normalize_uri
should be used with theTARGETURI
datastore option so the module can be fully targeted towards a system with a non-default root path- There should not be any exception handling, it's only necessary to check that
res
is nil and the status code - JSON data should be parsed with
res.get_json_document
elastic_table = Rex::Text::Table.new( | ||
'Header' => "#{index} Data", | ||
'Indent' => 2, | ||
# we know at least 1 row since we wouldn't query an index w/o a row | ||
'Columns' => json_body.dig('hits', 'hits')[0]['_source'].keys | ||
) | ||
|
||
json_body.dig('hits', 'hits').each do |hash| | ||
elastic_table << hash['_source'].values | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While testing this module I ran into this stack trace here where hash['_source'].values.length
was less than json_body.dig('hits', 'hits')[0]['_source'].keys
.
Here's the stack trace:
[+] .kibana-event-log-7.9.1-000030 Data
===================================
event message @timestamp ecs kibana
----- ------- ---------- --- ------
{"provider"=>"eventLog", "action"=>"starting"} eventLog starting 2023-06-09T15:12:52.818Z {"version"=>"1.5.0"} {"server_uuid"=>"480b1a88-e0c9-43f3-87c9-8734570c0acf"}
[-] Auxiliary failed: Rex::RuntimeError Invalid number of columns!
[-] Call stack:
[-] /home/smcintyre/.rvm/gems/ruby-3.0.4@metasploit-framework/gems/rex-text-0.2.52/lib/rex/text/wrapped_table.rb:194:in `add_row'
[-] /home/smcintyre/.rvm/gems/ruby-3.0.4@metasploit-framework/gems/rex-text-0.2.52/lib/rex/text/wrapped_table.rb:186:in `<<'
[-] /home/smcintyre/Repositories/metasploit-framework.pr/modules/auxiliary/gather/elasticsearch_enum.rb:88:in `block in get_results'
[-] /home/smcintyre/Repositories/metasploit-framework.pr/modules/auxiliary/gather/elasticsearch_enum.rb:87:in `each'
[-] /home/smcintyre/Repositories/metasploit-framework.pr/modules/auxiliary/gather/elasticsearch_enum.rb:87:in `get_results'
[-] /home/smcintyre/Repositories/metasploit-framework.pr/modules/auxiliary/gather/elasticsearch_enum.rb:165:in `block in get_indices'
[-] /home/smcintyre/Repositories/metasploit-framework.pr/modules/auxiliary/gather/elasticsearch_enum.rb:164:in `each'
[-] /home/smcintyre/Repositories/metasploit-framework.pr/modules/auxiliary/gather/elasticsearch_enum.rb:164:in `get_indices'
[-] /home/smcintyre/Repositories/metasploit-framework.pr/modules/auxiliary/gather/elasticsearch_enum.rb:397:in `run'
[*] Auxiliary module execution completed
msf6 auxiliary(gather/elasticsearch_enum) >
And here's what I did to fix it:
elastic_table = Rex::Text::Table.new( | |
'Header' => "#{index} Data", | |
'Indent' => 2, | |
# we know at least 1 row since we wouldn't query an index w/o a row | |
'Columns' => json_body.dig('hits', 'hits')[0]['_source'].keys | |
) | |
json_body.dig('hits', 'hits').each do |hash| | |
elastic_table << hash['_source'].values | |
end | |
columns = json_body.dig('hits', 'hits')[0]['_source'].keys | |
elastic_table = Rex::Text::Table.new( | |
'Header' => "#{index} Data", | |
'Indent' => 2, | |
# we know at least 1 row since we wouldn't query an index w/o a row | |
'Columns' => columns | |
) | |
json_body.dig('hits', 'hits').each do |hash| | |
elastic_table << columns.map { |column| hash['_source'][column] } | |
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So when I target an elasticsearch instance I use for things that has some data in it, the modules output is effectively unustable. The columns span so wide and the output scrolls way off the screen. I'd recommend simply printing less and not going into pulling the individual records themselves.
json_body.each do |username, attributes| | ||
elastic_table << [ | ||
username, | ||
attributes['roles'], | ||
attributes['email'], | ||
attributes['metadata'], | ||
attributes['enabled'], | ||
] | ||
end | ||
print_good(elastic_table.to_s) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm getting another exception here. I don't think I have any users setup.
[-] Auxiliary failed: NoMethodError undefined method `each' for nil:NilClass
[-] Call stack:
[-] /home/smcintyre/Repositories/metasploit-framework.pr/modules/auxiliary/gather/elasticsearch_enum.rb:375:in `get_users'
[-] /home/smcintyre/Repositories/metasploit-framework.pr/modules/auxiliary/gather/elasticsearch_enum.rb:399:in `run'
[*] Auxiliary module execution completed
msf6 auxiliary(gather/elasticsearch_enum) >
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
more likely the user you gave didn't have permission to view that data, which is a good case to test
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for those updates, it's working much better now so I'll get this landed.
msf6 auxiliary(gather/elasticsearch_enum) > run
[*] Running module against 192.168.249.3
[+] Elastic Information
===================
Name Cluster Name Version Build Type Lucene Version
---- ------------ ------- ---------- --------------
b6b69aad1328 docker-cluster 7.9.1 docker 8.6.2
[+] Node Information
================
IP Transport Port HTTP Port Version Name Uptime Ram Usage Node Role Master CPU Load Disk Usage
-- -------------- --------- ------- ---- ------ --------- --------- ------ -------- ----------
10.88.0.3 9300 10.88.0.3:9200 7.9.1 b6b69aad1328 25.1d 3.3gb/3.8gb dilmrt * 1% 29.6gb/49.1gb
[+] Cluster Information
===================
Cluster Name Status Number of Nodes
------------ ------ ---------------
docker-cluster yellow 1
[+] Indicies Information
====================
Name Health Status UUID Documents Storage Usage (MB)
---- ------ ------ ---- --------- ------------------
.apm-agent-configuration green open HWuN0mCoRrS9p7dF6GmGgQ 0 0MB
.apm-custom-link green open QgNjk6s4Tpm8nu5bI84tUQ 0 0MB
.async-search green open NLQRQdm1SqmSUPbWbQVBRg 0 0MB
.kibana-event-log-7.9.1-000030 green open UsoR3jPrROifPpbBg77Yiw 1 0MB
.kibana-event-log-7.9.1-000031 green open IGbdcS29SUiUt5byJ4sHUg 1 0MB
.kibana-event-log-7.9.1-000032 green open tR_JqMG4Tv68ctolHynA_Q 0 0MB
.kibana-event-log-7.9.1-000033 green open pZYjn3Z3REmlllDcKx35Vw 0 0MB
.kibana_1 green open YGu3riAEQqeqy322DEP4Qg 243 10MB
.kibana_task_manager_1 green open KDPL7nunRQ6rQYgj_Qc9PQ 6 11MB
.reporting-2020-09-13 green open Thglhp-8TmODvy22KBUC-g 1 0MB
.reporting-2021-06-20 green open eEP_GjxnQ_mEEee9SfBUbQ 1 0MB
.reporting-2022-04-03 green open Vbi_VAl2TeiC3tqUWUWNqg 2 0MB
.reporting-2022-11-06 green open dO0O8FPbTl2v28FpRrJDog 3 0MB
.reporting-2022-11-13 green open PO-9xMNHQFuGjX0whtkR-A 1 0MB
.reporting-2022-11-27 green open y9yZlMyFTK-u1xRJfd79hQ 1 0MB
comicsdb yellow open 2oEl-DaARy-Gq9jTaLiBFA 2460 0MB
github-api-v3.issues yellow open mU0kc2N1TWekqIZaHFFaUw 21469 68MB
github-api-v3.pull-requests yellow open -aX4LXhyTZKnTCoMUxM95Q 14482 151MB
github-api-v3.repositories yellow open sX0RvG1rReOeLiNB1A5psQ 73 0MB
metasploit.modules yellow open sGRIQZD5TS-XVkW9CX3-Gg 4452 6MB
[+] .kibana-event-log-7.9.1-000031 data stored to /home/smcintyre/.msf4/loot/20230824183003_default_192.168.249.3_elasticserch.ind_274242.csv
[+] .kibana-event-log-7.9.1-000030 data stored to /home/smcintyre/.msf4/loot/20230824183003_default_192.168.249.3_elasticserch.ind_984103.csv
[+] metasploit.modules data stored to /home/smcintyre/.msf4/loot/20230824183003_default_192.168.249.3_elasticserch.ind_701298.csv
[+] .reporting-2022-11-13 data stored to /home/smcintyre/.msf4/loot/20230824183003_default_192.168.249.3_elasticserch.ind_577512.csv
[+] comicsdb data stored to /home/smcintyre/.msf4/loot/20230824183003_default_192.168.249.3_elasticserch.ind_265916.csv
[+] github-api-v3.pull-requests data stored to /home/smcintyre/.msf4/loot/20230824183004_default_192.168.249.3_elasticserch.ind_230802.csv
[+] .reporting-2020-09-13 data stored to /home/smcintyre/.msf4/loot/20230824183004_default_192.168.249.3_elasticserch.ind_800440.csv
[+] .kibana_1 data stored to /home/smcintyre/.msf4/loot/20230824183014_default_192.168.249.3_elasticserch.ind_704807.csv
[+] .reporting-2021-06-20 data stored to /home/smcintyre/.msf4/loot/20230824183016_default_192.168.249.3_elasticserch.ind_102443.csv
[+] .reporting-2022-04-03 data stored to /home/smcintyre/.msf4/loot/20230824183016_default_192.168.249.3_elasticserch.ind_838023.csv
[+] .reporting-2022-11-06 data stored to /home/smcintyre/.msf4/loot/20230824183017_default_192.168.249.3_elasticserch.ind_740292.csv
[+] .kibana_task_manager_1 data stored to /home/smcintyre/.msf4/loot/20230824183017_default_192.168.249.3_elasticserch.ind_244783.csv
[+] .reporting-2022-11-27 data stored to /home/smcintyre/.msf4/loot/20230824183017_default_192.168.249.3_elasticserch.ind_408942.csv
[+] github-api-v3.repositories data stored to /home/smcintyre/.msf4/loot/20230824183017_default_192.168.249.3_elasticserch.ind_716946.csv
[+] github-api-v3.issues data stored to /home/smcintyre/.msf4/loot/20230824183017_default_192.168.249.3_elasticserch.ind_354774.csv
[-] Unable to pull user data
[*] Auxiliary module execution completed
Release NotesThis updates the Elasticsearch auxiliary module. It has been renamed to |
The original elasticsearch module had a few issues:
This PR enhances the module to:
Verification
msfconsole
use auxiliary/scanner/elasticsearch/enum
set ssl false
set username elastic
set password esbackup-password
set rhosts 127.0.0.1
run