Add NText column parsing to MSSQL #19054

zgoldman-r7 · 2024-04-04T13:37:10Z

Addresses Unsupported column type: 99. error for mssql

Verification:

In an mssql shell, create a table with an ntext type, ie CREATE TABLE xml_table(Col1 int primary key, Col2 xml);
INSERT INTO xml_table values (3, '<ProductDescription ProductID="1"/>');
in an mssql session or with the mssql_sql module, run select * from xml_table
Verify you get proper output with no errors

adfoster-r7 · 2024-04-04T23:43:35Z

Smaller replication steps 🟢

msf6 auxiliary(admin/mssql/mssql_sql) > run rhost=127.0.0.1 username=sa password=P4$$w0rd123 database='' sql="select cast('foo' as ntext);" createsession=true
[*] Running module against 127.0.0.1

[*] 127.0.0.1:1433 - SQL Query: select cast('foo' as ntext);
[*] 127.0.0.1:1433 - Row Count: 1 (Status: 16 Command: 193)
Response
========

 NULL
 ----
 foo

[*] Auxiliary module execution completed

Looks like it crashes with null though 🔴

msf6 auxiliary(admin/mssql/mssql_sql) > run rhost=127.0.0.1 username=sa password=P4$$w0rd123 database='' sql="select cast(null as ntext);" createsession=true
[*] Running module against 127.0.0.1

[-] 127.0.0.1:1433 - Auxiliary failed: TypeError no implicit conversion from nil to integer
[-] 127.0.0.1:1433 - Call stack:
[-] 127.0.0.1:1433 -   /Users/user/Documents/code/metasploit-framework/lib/rex/proto/mssql/client_mixin.rb:346:in `slice!'

Other examples of failing types, not a blocker for this PR

msf6 auxiliary(admin/mssql/mssql_sql) > run rhost=127.0.0.1 username=sa password=P4$$w0rd123 database='' sql="select cast(null as datetime);" createsession=true
[-] 127.0.0.1:1433 - Unsupported column type: 111. 


msf6 auxiliary(admin/mssql/mssql_sql) > run rhost=127.0.0.1 username=sa password=P4$$w0rd123 database='' sql="select cast(null as float);" createsession=true
[-] 127.0.0.1:1433 - Unsupported column type: 109. 

etc

List from: https://learn.microsoft.com/en-us/sql/t-sql/data-types/data-types-transact-sql?view=sql-server-ver16

adfoster-r7 · 2024-04-09T21:33:45Z

lib/rex/proto/mssql/client_mixin.rb

+        col[:cflags] = data.slice!(0, 2).unpack('v')[0]
+        col[:charset_id] =  data.slice!(0, 1).unpack('C')[0]
+        col[:namelen] = data.slice!(0, 1).unpack('C')[0]
+        col[:table_name] = data.slice!(0, (col[:namelen] * 2) + 1).gsub("\x00", '')


I think there's a directive for null terminated strings as an alternative to this gsub approach 👀

https://apidock.com/ruby/String/unpack
Z | String | null-terminated string

unless this is needed for utf16 support?

I think it is, but I'm not positive. Do you have any suggestions for verifying this?

Circling back - this had some unexpected behavior. ntext doesn't just end in null, but separates characters with it, and unpack with Z was removing characters to the right of null values

smcintyre-r7 · 2024-04-11T13:46:13Z

lib/rex/proto/mssql/client_mixin.rb

@@ -328,6 +337,18 @@ def mssql_parse_tds_row(data, info)
        end
        row << str.gsub("\x00", '')

+      when :ntext
+        str = ""


It looks here like str will always be a string, either empty or with data. I'm wondering though how it will handle a column from the target SQL server with nullable values where there's a mix of some set, some empty and some are null. On the ruby side I would expect nil for a null value, and strings for the rest.

So from what I can tell, it looks like when we're outputting the data, our format outputs both empty string and nil as blank. Do you think this is worth changing? @adfoster-r7 thoughts? I believe we've had a similar discussion before, but I can't remember.

the client should return real ruby values, regardless of our presentation layer

it's the same feedback as this: #18872 (comment) 📈

So you'll want to implement @smcintyre-r7's suggestion 💯

@zgoldman-r7 If in your testing the output data looks correct for NULL and empty string values without crashing, then it's probably just a matter of tweaking the Ruby code to identify when it should be nil and change the path from when it's an empty string.

I'm totally shooting in the dark here because I can't find the T-SQL docs you shared with me at one point, but I'd guess it might be something like this:

str = nil ptrlen = data.slice!(0, 1).unpack("C")[0] ptr = data.slice!(0, ptrlen) unless ptrlen == 0 str = "" # at this point, ptrlen being non-zero implies there's data timestamp = data.slice!(0, 8) datalen = data.slice!(0, 4).unpack("V")[0] # this is probably zero if the string is empty? str = data.slice!(0, datalen) end

I could be very very wrong though. Could you also add a link to the docs you've been reading into this file? That would help future travelers and myself.

@smcintyre-r7 As far as that pattern goes, I implemented something similar locally and it seems to work, so that looks good - I'll tweak the presentation layer too 👍

As far as the docs link, I have a link to https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-tds/ce3183a6-9d89-47e8-a02f-de5a1a1303de from a recent PR on line 234 - is that what you're referring to?

It might have been a different page we were looking at. Based on those docs though

Null is represented by a length of 65535 (0xFFFF). A non-nullable char or binary can still have a length of zero (for example, an empty value).

It looks like something should be compared to 0xffff which is different than what I'd suggested.

@smcintyre-r7 interesting catch - especially considering that doesn't seem to be what I get on wireshark running run rhost=127.0.0.1 username=sa password=P4$$w0rd123 database='' sql="select cast(null as ntext);" - I wonder if/how that's different.

Looking at our :string parse though, it seems to have a similar check for both (line 326):

if len > 0 && len < 65535 str << data.slice!(0, len) end

adfoster-r7 · 2024-04-18T15:27:44Z

  4) MSSQL sessions and MSSQL modules linux/mssql2019-latest auxiliary/scanner/mssql/mssql_hashdump when targeting a session linux/mssql2019-latest session opens and passes the "auxiliary/scanner/mssql/mssql_hashdump" tests
     Failure/Error: expect(test_line).to_not include('FAILED', '[-] FAILED', '[-] Exception', '[-] '), "Unexpected error: #{test_line}"
       Unexpected error: [-] Unsupported SQL Version: 2019

Looks like test failures 👀

adfoster-r7 · 2024-04-18T16:10:05Z

modules/auxiliary/scanner/mssql/mssql_hashdump.rb

For the docs:

hashcat --force -m 1731 ./hashes.txt ./passwords.txt

131 MSSQL (2000) 0x01002702560500000000000000000000000000000000000000008db43dd9b1972a636ad0c7d4b8c515cb8ce46578 132 MSSQL (2005) 0x010018102152f8f28c8499d8ef263c53f8be369d799f931b2fbe 1731 MSSQL (2012, 2014, and above) 0x02000102030434ea1b17802fd95ea6316bd61d2c94622ca3812793e8fb1672487b5c904a45a31b2ab4a78890d563d2fcf5663e46fe797d71550494be50cf4915d3f4d55ec375

adfoster-r7 · 2024-04-18T16:39:58Z

Fixing that mssql hashdump module:

diff --git a/modules/auxiliary/scanner/mssql/mssql_hashdump.rb b/modules/auxiliary/scanner/mssql/mssql_hashdump.rb
index cfac867a4f..a1fd781fbc 100644
--- a/modules/auxiliary/scanner/mssql/mssql_hashdump.rb
+++ b/modules/auxiliary/scanner/mssql/mssql_hashdump.rb
@@ -28,6 +28,12 @@ class MetasploitModule < Msf::Auxiliary
       set_session(session.client)
     elsif !mssql_login(datastore['USERNAME'], datastore['PASSWORD'])
       print_error('Invalid SQL Server credentials')
+      info = self.mssql_client.initial_connection_info
+      if info[:errors] && !info[:errors].empty?
+        info[:errors].each do |err|
+          print_error(err)
+        end
+      end
       return
     end
 
@@ -98,6 +104,8 @@ class MetasploitModule < Msf::Auxiliary
       hashtype = "mssql05"
     when "2012", "2014"
       hashtype = "mssql12"
+    else
+      hashtype = "mssql12"
     end
 
     this_service = report_service(
@@ -125,12 +133,15 @@ class MetasploitModule < Msf::Auxiliary
       next if row[0].nil? or row[1].nil?
       next if row[0].empty? or row[1].empty?
 
+      username = row[0]
+      upcase_hash = "0x#{row[1].upcase}"
+
       credential_data = {
           module_fullname: self.fullname,
           origin_type: :service,
           private_type: :nonreplayable_hash,
-          private_data: "0x#{row[1]}",
-          username: row[0],
+          private_data: upcase_hash,
+          username: username,
           jtr_format: hashtype
       }
 
@@ -146,8 +157,8 @@ class MetasploitModule < Msf::Auxiliary
       login_data.merge!(service_data)
       login = create_credential_login(login_data)
 
-      tbl << [row[0], row[1]]
-      print_good("Saving #{hashtype} = #{row[0]}:#{row[1]}")
+      tbl << [username, upcase_hash]
+      print_good("Saving #{hashtype} = #{username}:#{upcase_hash}")
     end
   end
 
@@ -167,6 +178,8 @@ class MetasploitModule < Msf::Auxiliary
 
     when "2005", "2008", "2012", "2014"
       results = mssql_query(mssql_2k5_password_hashes())[:rows]
+    else
+      results = mssql_query(mssql_2k5_password_hashes())[:rows]
     end
 
     return results

Will also need to update the user docs if that hasn't been added 👀

adfoster-r7 · 2024-04-18T16:52:00Z

diff --git a/modules/auxiliary/scanner/mssql/mssql_hashdump.rb b/modules/auxiliary/scanner/mssql/mssql_hashdump.rb
index cfac867a4f..6656c83744 100644
--- a/modules/auxiliary/scanner/mssql/mssql_hashdump.rb
+++ b/modules/auxiliary/scanner/mssql/mssql_hashdump.rb
@@ -107,12 +107,6 @@ class MetasploitModule < Msf::Auxiliary
           :proto => 'tcp'
           )
 
-    tbl = Rex::Text::Table.new(
-      'Header'  => 'MS SQL Server Hashes',
-      'Indent'   => 1,
-      'Columns' => ['Username', 'Hash']
-    )
-
     service_data = {
         address: ::Rex::Socket.getaddress(mssql_client.peerhost,true),
         port: mssql_client.peerport,
@@ -146,7 +140,6 @@ class MetasploitModule < Msf::Auxiliary
       login_data.merge!(service_data)
       login = create_credential_login(login_data)
 
-      tbl << [row[0], row[1]]
       print_good("Saving #{hashtype} = #{row[0]}:#{row[1]}")
     end
   end

This can be deleted 👍

adfoster-r7 · 2024-04-18T20:31:47Z

Looks like there's still some test failures 👀

github-actions · 2024-04-18T20:32:19Z

Thanks for your pull request! Before this can be merged, we need the following documentation for your module:

zgoldman-r7 · 2024-04-18T20:33:41Z

Looks like there's still some test failures 👀

yep - schemadump adjustments I made need tweaking for the tests, looking shortly

adfoster-r7 · 2024-04-19T13:13:05Z

documentation/modules/auxiliary/scanner/mssql/mssql_hashdump.md

+
+The `mssql_hashdump` module queries an MSSQL instance or session and returns hashed user:pass pairs. These pairs can be decripted via or `hashcat`.
+
+## Available Options


Not a blocker; I don't think this follows the format that's expected for module documentation: #19054 (comment)

adfoster-r7 · 2024-04-19T13:15:54Z

test/modules/post/test/mssql.rb

@@ -41,6 +41,15 @@ def test_console_query
    end
  end

+  def test_datatypes
+    it "should support ntext TDS datatype" do
+      stdout = with_mocked_console(session) {|console| console.run_single(%{ query "select cast('foo' as ntext);"})}


We'll probably want this to use the convention of the other PR, but can fix separately

adfoster-r7 · 2024-04-19T13:17:42Z

I think there's some follow up work here, but this seems like a good first step

adfoster-r7 · 2024-04-19T13:18:10Z

Release Notes

Adds NText column parsing to MSSQL modules

zgoldman-r7 force-pushed the mssql-ntext-support branch 2 times, most recently from 83f3cee to f00cc0d Compare April 4, 2024 17:47

zgoldman-r7 marked this pull request as ready for review April 4, 2024 17:50

adfoster-r7 mentioned this pull request Apr 4, 2024

Error handling response - admin/mssql/mssql_sql - unsupported token and invalid number of columns #18970

Closed

zgoldman-r7 force-pushed the mssql-ntext-support branch from f00cc0d to 72bdc7e Compare April 9, 2024 19:44

adfoster-r7 reviewed Apr 9, 2024

View reviewed changes

smcintyre-r7 reviewed Apr 11, 2024

View reviewed changes

zgoldman-r7 force-pushed the mssql-ntext-support branch 3 times, most recently from 995d609 to ac54e92 Compare April 17, 2024 22:16

adfoster-r7 reviewed Apr 18, 2024

View reviewed changes

zgoldman-r7 force-pushed the mssql-ntext-support branch from ac54e92 to f3cb214 Compare April 18, 2024 17:11

adfoster-r7 added the needs-docs label Apr 18, 2024

zgoldman-r7 force-pushed the mssql-ntext-support branch from f3cb214 to 3e0b5eb Compare April 18, 2024 20:49

adds ntext parsing to mssql

d357484

zgoldman-r7 force-pushed the mssql-ntext-support branch from 3e0b5eb to d357484 Compare April 18, 2024 21:41

adfoster-r7 reviewed Apr 19, 2024

View reviewed changes

adfoster-r7 merged commit 376bdef into rapid7:master Apr 19, 2024
59 checks passed

smcintyre-r7 added the rn-enhancement release notes enhancement label Apr 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add NText column parsing to MSSQL #19054

Add NText column parsing to MSSQL #19054

zgoldman-r7 commented Apr 4, 2024 •

edited

Loading

adfoster-r7 commented Apr 4, 2024 •

edited

Loading

adfoster-r7 Apr 9, 2024 •

edited

Loading

adfoster-r7 Apr 9, 2024

zgoldman-r7 Apr 11, 2024

zgoldman-r7 Apr 16, 2024

smcintyre-r7 Apr 11, 2024

zgoldman-r7 Apr 11, 2024

adfoster-r7 Apr 11, 2024

smcintyre-r7 Apr 11, 2024

zgoldman-r7 Apr 11, 2024

smcintyre-r7 Apr 11, 2024

zgoldman-r7 Apr 11, 2024 •

edited

Loading

adfoster-r7 commented Apr 18, 2024

adfoster-r7 Apr 18, 2024 •

edited

Loading

adfoster-r7 commented Apr 18, 2024 •

edited

Loading

adfoster-r7 commented Apr 18, 2024

adfoster-r7 commented Apr 18, 2024

github-actions bot commented Apr 18, 2024

zgoldman-r7 commented Apr 18, 2024

adfoster-r7 Apr 19, 2024

adfoster-r7 Apr 19, 2024

adfoster-r7 commented Apr 19, 2024

adfoster-r7 commented Apr 19, 2024


		The `mssql_hashdump` module queries an MSSQL instance or session and returns hashed user:pass pairs. These pairs can be decripted via or `hashcat`.

		## Available Options

Add NText column parsing to MSSQL #19054

Add NText column parsing to MSSQL #19054

Conversation

zgoldman-r7 commented Apr 4, 2024 • edited Loading

adfoster-r7 commented Apr 4, 2024 • edited Loading

adfoster-r7 Apr 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zgoldman-r7 Apr 11, 2024 • edited Loading

Choose a reason for hiding this comment

adfoster-r7 commented Apr 18, 2024

adfoster-r7 Apr 18, 2024 • edited Loading

Choose a reason for hiding this comment

adfoster-r7 commented Apr 18, 2024 • edited Loading

adfoster-r7 commented Apr 18, 2024

adfoster-r7 commented Apr 18, 2024

github-actions bot commented Apr 18, 2024

zgoldman-r7 commented Apr 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adfoster-r7 commented Apr 19, 2024

adfoster-r7 commented Apr 19, 2024

Release Notes

zgoldman-r7 commented Apr 4, 2024 •

edited

Loading

adfoster-r7 commented Apr 4, 2024 •

edited

Loading

adfoster-r7 Apr 9, 2024 •

edited

Loading

zgoldman-r7 Apr 11, 2024 •

edited

Loading

adfoster-r7 Apr 18, 2024 •

edited

Loading

adfoster-r7 commented Apr 18, 2024 •

edited

Loading