Fixing multiple bugs in credential generation + refactoring #19653

Mathiou04 · 2024-11-15T17:12:58Z

Fixes #19652

Summary

In this PR, I added various tests on the CredentialCollection class, all of them currently failing and illustrating what I think are bugs.

Issues can be grouped in two categories:

those linked to the additional_publics array
those linked to the password_spray option

I added some comments on each test explaining what it is currently yielding (instead of the result I think should be expected - let me know if I misunderstood anything).

Refactoring opportunity

You will see that there are many bugs linked to the password_spray option.

It seems that this branch of the code has been copy/pasted from the original branch (that didn't handle password spraying) and adapted.
There was not a lot of tests on this part of the code + the structure is difficult to maintain which probably explains those issues.

The original code branch (without spraying: #each_unfiltered_username_first) is also a bit complex with a lot of duplicated code.

I think there are some opportunities to simplify and clarify the code a lot with some refactoring.

Non-regression tests

For this, I created 2 additional "non-regression" tests that activate all options and show how the credentials should be yielded.
There is one test with password spraying, one without.

I don't know if we want to keep them after the refactoring, but they will surely help.

Also, even if a part of the order in which the credentials are yielded should be "fixed", there are questions around others.
For example:

when do we want to yield the credentials generated by the user_as_pass option in case of password_spraying?
when do we want to yield credentials generated by the "userpass" file?

Additional question

As a side note, the nil_passwords, blank_passwords and user_as_pass options do not apply to the userpass file.
Is it expected behaviour or should we extend this behaviour to users yielded in the userpass file?

Next steps

1. I first plan to fix the easiest bugs (wrong var names for example) to make more tests pass
2. Then I would tackle the cleaning of the password spraying code branch, so as to make all tests pass.
3. Then in a third commit, I would refactor both branches to simplify the code and reduce the probability to have such bugs in the future (also facilitating maintenance)

Mathiou04 · 2024-11-15T17:14:02Z

spec/lib/metasploit/framework/credential_collection_spec.rb

+          let(:password) { nil }
+          specify  do
+            expect { |b| collection.each(&b) }.to yield_successive_args(
+              Metasploit::Framework::Credential.new(public: username, private: nil),


Fails because the collection yields nothing in case of "password spraying".

Mathiou04 · 2024-11-15T17:14:43Z

spec/lib/metasploit/framework/credential_collection_spec.rb

+        # REMOVE BEFORE COMMIT currently failing
+        specify  do
+          expect { |b| collection.each(&b) }.to yield_successive_args(
+            Metasploit::Framework::Credential.new(public: "test_public", private: "test_public"),


Crashes because of a non-existent variable.

Mathiou04 · 2024-11-15T17:14:51Z

spec/lib/metasploit/framework/credential_collection_spec.rb

+          # REMOVE BEFORE COMMIT currently failing
+          specify  do
+            expect { |b| collection.each(&b) }.to yield_successive_args(
+              Metasploit::Framework::Credential.new(public: "test_public", private: "test_public"),


Crashes because of a non-existent variable.

Mathiou04 · 2024-11-15T17:16:02Z

spec/lib/metasploit/framework/credential_collection_spec.rb

+        # REMOVE BEFORE COMMIT: this option is ignored currently for additional_publics
+        specify do
+          expect { |b| collection.each(&b) }.to yield_successive_args(
+            Metasploit::Framework::Credential.new(public: "test_public", private: nil),


This case yields nothing: nil passwords are ignored in additional_public credential generation. (is there a reason for this?)

Mathiou04 · 2024-11-15T17:17:02Z

spec/lib/metasploit/framework/credential_collection_spec.rb

+                Metasploit::Framework::Credential.new(public: "test_public1", private: ""),
+                Metasploit::Framework::Credential.new(public: "test_public2", private: ""),
+                Metasploit::Framework::Credential.new(public: "test_public1", private: nil),
+                Metasploit::Framework::Credential.new(public: "test_public2", private: nil),


This fails because the password spraying option does not work with additional_publics.
It generates the correct list of credentials, but not in the correct order.

Mathiou04 · 2024-11-15T17:18:24Z

spec/lib/metasploit/framework/credential_collection_spec.rb

+              expect { |b| collection.each(&b) }.to yield_successive_args(
+                Metasploit::Framework::Credential.new(public: "adsf", private: "password"),
+                Metasploit::Framework::Credential.new(public: "jkl", private: "password"),
+                Metasploit::Framework::Credential.new(public: "test_public", private: "password"),


This fails because for some reason, in addition to those 3 credentials, it also uses the additional_publics ("test_public") as a password.

Mathiou04 · 2024-11-15T17:19:09Z

spec/lib/metasploit/framework/credential_collection_spec.rb

+
+          # REMOVE BEFORE COMMIT: this yields empty creds (no username, no pass)
+          specify  do
+            expect { |b| collection.each(&b) }.to yield_successive_args()


This fails because in this case (no username defined!) it generates a set of empty credentials.

Mathiou04 · 2024-11-15T17:20:05Z

spec/lib/metasploit/framework/credential_collection_spec.rb

+
+          specify  do
+            expect { |b| collection.each(&b) }.to yield_successive_args(
+              Metasploit::Framework::Credential.new(public: username, private: ''),


This fails because it generates no credential.
But in this case we have a username and are asking for a blank password so it should generate this. (this is the case I chose to describe in the issue)

…monstrate multiple bugs in the each method

Mathiou04 · 2024-11-15T17:27:11Z

spec/lib/metasploit/framework/credential_collection_spec.rb

+            expect { |b| collection.each(&b) }.to yield_successive_args(
+              Metasploit::Framework::Credential.new(public: "asdf", private: ''),
+              Metasploit::Framework::Credential.new(public: "jkl", private: ''),
+            )


Fails also because it yields nothing. The given file is completely ignored even if we would like to have the usernames with the blank passwords generated.

Mathiou04 · 2024-11-15T17:28:15Z

spec/lib/metasploit/framework/credential_collection_spec.rb

+          Metasploit::Framework::Credential.new(public: "test_public", private: ""),
+          Metasploit::Framework::Credential.new(public: "test_public", private: "passfile"),
+          Metasploit::Framework::Credential.new(public: "test_public", private: "test_private")
+        )


Currently failing because of the various bugs that make the other tests fail.

Mathiou04 · 2024-11-15T17:28:21Z

spec/lib/metasploit/framework/credential_collection_spec.rb

+            Metasploit::Framework::Credential.new(public: "userpass_user", private: "userpass_pass"),
+            Metasploit::Framework::Credential.new(public: "user", private: "test_private"),
+            Metasploit::Framework::Credential.new(public: "userfile", private: "test_private"),
+            Metasploit::Framework::Credential.new(public: "test_public", private: "test_private"),


Currently failing because of the various bugs that make the other tests fail.

Mathiou04 · 2024-11-15T17:29:57Z

spec/lib/metasploit/framework/credential_collection_spec.rb

+      end
+    end
+
+    context "when every possible option is used" do


I tried to create some generic non-regression tests to demonstrate how the credentials should be yielded in case of various user/password data source (in fact in case of all them being active).

The idea is mainly to check that all credentials are present, but also to have the proper order.
I am not sure they should be kept after the fixes/refacto have been made (as there may be some question regarding the order, many are probably valid?) but here they will help demonstrate what we expect.

…lics

…of password spraying) to make all tests pass

Mathiou04 · 2024-11-18T16:52:10Z

lib/metasploit/framework/credential_collection.rb

+
+        additional_publics.each do |add_public|
+          yield Metasploit::Framework::Credential.new(public: add_public, private: add_public, realm: realm, private_type: :password)
+        end


We can easily see the potential for a refactoring here as many blocks are almost complete c/p of previous blocks.

cgranleese-r7

I know it's still in draft but I thought I'd give it a once over anyway. Looks great and thanks for following up on the last pull request 🚀

cgranleese-r7 · 2024-12-06T17:38:50Z

spec/lib/metasploit/framework/credential_collection_spec.rb

+      context "when using password spraying" do
+        let(:password_spray) { true }
+
+        # REMOVE BEFORE COMMIT: yields nothings, fails because of bug in method
+        context "without password" do


What would your thoughts be on not nesting the contexts here. My thinking is it would make searching for failing tests much harder, as the sting would be broken across multiple lines whereas if it was a single context it would be much easier to find.

Here is where I usually check "best practices" when writing tests, just in case it's useful:
https://www.betterspecs.org/#contexts

Ok with me 👍
I struggled a bit to find a proper organization of the cases, given the already existing specs.

Do you want to merge only those two, or do you think I should also do the same for the other cases?
(meaning having only 1-level deep contexts)

Do you want to merge only those two, or do you think I should also do the same for the other cases? (meaning having only 1-level deep contexts)

I think having only one context for each test would be better when combined with this comment.

Ok, fine with me. I'll change that!

This was a bit painful to do but I agree, once it is done, it is much clearer to read and to debug 🎉

cgranleese-r7 · 2024-12-06T17:47:05Z

spec/lib/metasploit/framework/credential_collection_spec.rb

+          end
+        end
+
+        context "when given a user_file" do


I'm assuming from the comment below you want to have blank_passwords set here.

It is actually set up on line 434, but with the nested contexts it is not easy to follow along...
I am wondering if it is better to duplicate setup code so as to have 1-level context where you can properly see all the setup, or be a bit more dry and nest.

Happy to follow what you feel is best.

better to duplicate setup code so as to have 1-level context where you can properly see all the setup

I would personally be in favor of that as it is much easier to see the configuration for each test.

Mathiou04 commented Nov 15, 2024

View reviewed changes

Mathiou04 force-pushed the fix_bugs_in_credentials_collection_enumerator_and_refacto branch from 64aeccc to 4e9d771 Compare November 15, 2024 17:21

CredentialCollection: adding various tests that currently fails to de…

7cab903

…monstrate multiple bugs in the each method

Mathiou04 force-pushed the fix_bugs_in_credentials_collection_enumerator_and_refacto branch from 4e9d771 to 7cab903 Compare November 15, 2024 17:26

Mathiou04 commented Nov 15, 2024

View reviewed changes

Mathiou04 mentioned this pull request Nov 18, 2024

Various bugs when using the PASSWORD_SPRAY option #19652

Open

Mathiou04 added 4 commits November 18, 2024 15:38

Fixes trivial bugs when combining additional_publics with user_as_pass

5d7d8cf

Fixes the fact that no nil credential is generated for additional_pub…

397f466

…lics

Fix incorrect expectations of currently failing tests

5fce8d2

Re-implement the each_unfiltered_password_first method (used in case …

383dc0d

…of password spraying) to make all tests pass

Mathiou04 commented Nov 18, 2024

View reviewed changes

adfoster-r7 requested a review from cgranleese-r7 December 2, 2024 11:38

cgranleese-r7 self-assigned this Dec 4, 2024

cgranleese-r7 reviewed Dec 6, 2024

View reviewed changes

cgranleese-r7 added enhancement rn-enhancement release notes enhancement labels Dec 13, 2024

Inline all specs to avoid nesting context

e03f2e2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixing multiple bugs in credential generation + refactoring #19653

Fixing multiple bugs in credential generation + refactoring #19653

Mathiou04 commented Nov 15, 2024 •

edited by cgranleese-r7

Loading

Mathiou04 Nov 15, 2024

Mathiou04 Nov 15, 2024 •

edited

Loading

Mathiou04 Nov 15, 2024 •

edited

Loading

Mathiou04 Nov 15, 2024 •

edited

Loading

Mathiou04 Nov 15, 2024

Mathiou04 Nov 15, 2024

Mathiou04 Nov 15, 2024 •

edited

Loading

Mathiou04 Nov 15, 2024

Mathiou04 Nov 15, 2024

Mathiou04 Nov 15, 2024

Mathiou04 Nov 15, 2024

Mathiou04 Nov 15, 2024 •

edited

Loading

Mathiou04 Nov 18, 2024 •

edited

Loading

cgranleese-r7 left a comment

cgranleese-r7 Dec 6, 2024

Mathiou04 Dec 10, 2024

cgranleese-r7 Dec 12, 2024 •

edited

Loading

Mathiou04 Dec 13, 2024

Mathiou04 Dec 13, 2024

cgranleese-r7 Dec 6, 2024

Mathiou04 Dec 10, 2024

cgranleese-r7 Dec 12, 2024

Fixing multiple bugs in credential generation + refactoring #19653

Are you sure you want to change the base?

Fixing multiple bugs in credential generation + refactoring #19653

Conversation

Mathiou04 commented Nov 15, 2024 • edited by cgranleese-r7 Loading

Summary

Refactoring opportunity

Non-regression tests

Additional question

Next steps

Choose a reason for hiding this comment

Mathiou04 Nov 15, 2024 • edited Loading

Choose a reason for hiding this comment

Mathiou04 Nov 15, 2024 • edited Loading

Choose a reason for hiding this comment

Mathiou04 Nov 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mathiou04 Nov 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mathiou04 Nov 15, 2024 • edited Loading

Choose a reason for hiding this comment

Mathiou04 Nov 18, 2024 • edited Loading

Choose a reason for hiding this comment

cgranleese-r7 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cgranleese-r7 Dec 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mathiou04 commented Nov 15, 2024 •

edited by cgranleese-r7

Loading

Mathiou04 Nov 15, 2024 •

edited

Loading

Mathiou04 Nov 15, 2024 •

edited

Loading

Mathiou04 Nov 15, 2024 •

edited

Loading

Mathiou04 Nov 15, 2024 •

edited

Loading

Mathiou04 Nov 15, 2024 •

edited

Loading

Mathiou04 Nov 18, 2024 •

edited

Loading

cgranleese-r7 Dec 12, 2024 •

edited

Loading