TDP has 2 sys admins, and one has turned evil 😈 and decided to compromise the other's account such that they can no longer access TDP DAC. Below is a screenshot of the compromised account:
Sys admin account (i.e. [email protected]
) has been modified with a different username/email address, first name, and last name.
When the sys admin tries to sign back in, she is taken to the request access page. She will be unable to access TDP DAC until this request is approved by a user with OFA System Admin
permissions
- Follow communication protocol
- Sys admin with compromised account should work with System owner to confirm appropriate admin roles are assigned to their cloud.gov account. This account will be necessary to change TDP DAC access remotely.
- Sys admin with compromised account should use CF CLI to remotely restore permissions:
CF CLI commands
1. Login via CF CLI
$ cf login -a api.fr.cloud.gov --sso
API endpoint: api.fr.cloud.gov
Temporary Authentication Code ( Get one at https://login.fr.cloud.gov/passcode ): <redacted passcode>
Authenticating...
OK
Select an org:
1. hhs-acf-prototyping
2. sandbox-hhs
Org (enter to skip): hhs-acf-prototyping
hhs-acf-prototyping
Targeted org hhs-acf-prototyping.
Select a space:
1. tanf-dev
2. tanf-prod
3. tanf-staging
Space (enter to skip): tanf-staging
tanf-staging
Targeted space tanf-staging.
2. Target the appropriate backend application and initiate django shell
2a. The following commands work on non-GFE (press after each command)
cf ssh tdp-backend-staging
/tmp/lifecycle/shell
python manage.py shell_plus
2b. If 2a does not work on GFE, the following commands should work (reference) (make sure you are NOT connected to VPN):
- Retrieve the PROCESS_GUID for the web process for your app. You will be prompted for this value below.
cf curl v3/apps/$(cf app tdp-backend-staging --guid)/processes | jq --raw-output '.resources | .[]? | select(.type == "web").guid'
- Retrieve a one-time ssh passcode. You will be prompted for this code below.
cf ssh-code
- Use ssh to connect to your app container. Insert the value for your PROCESS_GUID obtained above.
ssh -p 2222 cf:<PROCESS_GUID>/[email protected]
- You will be prompted to enter a password. Use the one-time ssh passcode generated from
cf ssh-code
above (Note: pasting the value will be invisible). If this is successful, the command line will appear like this:
vcap@SomeHashValue...
- Initiate interactive SSH session using the following 2 commands (press
enter
after each command):
/tmp/lifecycle/shell
python manage.py shell_plus
You will know you are in an interactive session if the command line appears like this:
In [1]:
3. In the Django shell, give your user account (the one associated with PIV/CAC) the appropriate permissions:
user = User.objects.get(username='[email protected]')
user.groups.set(Group.objects.filter(name='OFA System Admin'))
user.account_approval_status = 'Approved'
user.is_staff = True
user.is_superuser = True
user.is_deactivated = False
user.save()
4. Login to TDP frontend via ACF AMS and confirm access to TDP DAC restored.
Note: be sure to request access again, if prompted. after submitting the request, all appropriate access should be restored.
-
Conduct post-mortem:
- How did the system admin's account get compromised? What facilitated this?
- What was the impact of this compromise?
- What lessons were learned?
- Next steps?
-
Contact system owner via Mattermost OFA TDP Product Channel. If Mattermost isn't available, please send an email and cc: ACF Tech Lead, Product Manager, vendor Tech Lead.
-
Include details of the compromise, which should include:
- where compromise occurred (e.g. TDP? cloud.gov?)
- how the compromise occurred (if known)
- preliminary assessment of the scope and impact of the compromise (e.g. were spaces and/or services deleted? was PII compromised? exposed?)
- Next steps (e.g. schedule mtg to discuss incident response plan, rotate keys, etc.)
-
If system admin's compromised account is not restored by following the abovementioned steps, this could be because the evil user also changed the admin's cloud.gov account permissions and/or compromised other components of TDP. If this is the case, the
system owner
should reach out to cloud.gov (https://cloud.gov/docs/help/) support to have any necessary user permissions and/or services (e.g. database) restored.
- Tools needed for this incident response strategy:
- cloud.gov account with
user
andmanager
roles assigned at and levels. - git bash - this usually comes with the download package for Git for Windows. Can alternatively use Command Prompt.
- Curl for Windows 64 bit - this usually comes with the download package for Git for Windows
- cloud foundry binary for Windows 64 bit
- JQuery version 1.6 for Windows 64 bit
- cloud.gov account with
- Mitigation steps outlined above target
hhs-acf-prototyping
org,tdp-staging
space andtdp-backend-staging
app.org
,space
, andapp
targets will need to be modified in a production context. - Review the latest TDP Incidence Response Plan here, and see incidence response tips from cloud.gov.
- Should Sys owner and sys admins all have the same roles in cloudgov? Is this an acceptable risk?
- Yes. System owner should reach out to cloud.gov support if a privileged user's cloud.gov account roles are compromised.
- Compromised account will still appear as a user in TDP DAC. Should we delete this user account?
- No. This account will remained in a deactivated state.
- The evil user's account should-- at a minimum-- be deactivated from TDP, but should we delete it? (presume this person will no longer be working for ACF)
- No. This account will remain in a deactivated state. Deleting the account would result in losing the history of activity associated with the user which should be kept for auditing purposes. Additionally, this prevents the evil user from attempting to re-request access to TDP.