Hello~Dear Mr. Mango

[fgvcbv]

Dear Mr. Mango
Your code is really great. I have learned a lot from you and I am very, very grateful to you
But I still have two questions that I would like your answer to. I don't know if it's convenient for you
The first and most important question is why it cannot be used in the vmexit handle
Did you handle functions such as dbgbreakpoint and dbgprintex in any way? I really didn't see it
The second issue is that I found that setting my virtual machine to either 1 or 2 cores can run perfectly, regardless of whether EPT is enabled or not
But when the number of CPUs is increased to 4, the computer will freeze when running, and of course, the physical machine will also freeze directly. Do you know the general direction of the problem

[jonomango]

Most kernel functions cannot be used in root-mode because we are effectively running at irql HIGH_LEVEL. As for your second question, that is pretty weird. Maybe it is due to the timing check mitigations that I’ve implemented.

[fgvcbv]

Thank you very much, Mr. Mango, for answering my question. For the second question, I will continue to test and provide you with the results~
For the first question
Most of the source code on GitHub can be used in the handle using DbgPrintEx
So I think you must have done some miraculous operation to keep the root environment running in irql HIGH-LEVEL
Can you tell me where this operation appears in the code? I would like to change it back to normal mode first, so that it is convenient for me to use dbgbreakpoint and dbgprint to debug the program

[jonomango]

Unfortunately it is not possible. Those hypervisors only work because they are done incorrectly and dont properly separate the guest and host state.

[jonomango]

I have included a logger for printing, which you could use instead.

[fgvcbv]

Yes, I know the um.exe in the project
However, for situations where the system freezes directly during runtime, um.exe may not work well. It is indeed convenient to view logs from there when the computer is running normally

[jonomango]

Yes, I know what you mean. One solution is to print the address of the logging structure before virtualization, and manually look at the logs using WinDbg if a crash occurs. It is a bit difficult, but possible.

[fgvcbv]

No, no, no, I think your idea is very clever. This may be the correct way to solve the problem. I'll try it out and see the last log before it freezes

[fgvcbv]

Mr. Mango, I'm sorry to bother you again~
If you could take a look at my two questions amidst your busy schedule, I would greatly appreciate it
1:
I know that it is not properly handled that other people's hvs can call kernel functions in the handle
After reading your code for three days, I haven't figured out how you can correctly separate guest and host. Is the essence in write_vmcs_host_fields or write_vmcs_guest_fields, or is it
Prepare_host_page_tables(); What did you do?
2:
Overall, the framework is really good, but there is one point that has a significant impact:
On some systems, such as Win10 22h2 multi-core, when VT is enabled, guest machine calls to kernel functions such as dbgprintex will also crash. The most critical impact is that EPT HOOK can also crash when certain functions return original functions (I confirm that the HOOK logic is correct, because as long as it is set to a single CPU, it can work perfectly)
Do you have any good solutions for this

[jonomango]

1. Separating guest and host state essentially means making your own host page tables, GDT, IDT, TSS, control registers, and any other structures that are needed for host operation.
2. So the issue only occurs when you're EPT hooking? What exactly are you hooking, and how are you doing it?

[fgvcbv]

For example, if I download a new copy of the HV source code now without making any modifications
After I compile, different situations will occur when placed in the following two types of virtual machines
Virtual machine version: win10 22h2
HOOK functions: NtOpenProcess and MmLoadSystemImage (to obtain the loaded driver and base address)
When I set the number of CPUs to 1
Everything is normal, no matter how I HOOK, any function runs perfectly and the guest machine freely uses dbgprintex
Even if there are any issues with the operation of the virtual machine, it is difficult and impossible for me to do so, thanks to the excellent logic you have handled
When I set the number of CPUs to 8
(1) HOOKNtOpenProcess can work perfectly on any CPU
(2) : MmLoadSystemImage. If I return an error code, there is no problem and it can work properly
If I return original-MmLoadSystemImage, the system will shut down directly instead of getting stuck, which is similar to using dbgprintex in the handle
(3) : When all initialization is completed, call DbgPrintEx before the return of driver_entry to shut down the system directly, as in the above case
(4) The entire project in hv.sys does not use dbgprintex or perform any EPT HOOK, and everything runs perfectly
But when I install an empty driver, there are no commands inside except for a dbgprintex sentence (note that another driver is installed at this time), and the system will shut down directly. Of course, I think this is not limited to dbgprintex. Other things may also trigger it. Just kidding, dbgprintex can even be used as a means of attacking guest machines, haha
If you have a Win10 22h2 image, I think you should be able to see the same phenomenon, love u~

[jonomango]

Can you show the exact way that you are using the EPT hooking?

[fgvcbv]

This may be a bit long, do you have an email? I will directly send the source code to your email. However, I think it works normally in a single core environment, and I don't think the third and fourth items have much to do with the EPT writing. I will try to paste the EPT logic here first to prevent you from being inconvenient to provide an email

ULONG64 FakePage=SpawnFakeInfo((ULONG64)NtOpenProcess, (ULONG64)MyNtOpenProcess, (ULONG64*)&OriginalNtOpenProcess);
ULONG64 FakePagePHYADDS= getphyadds(0, (ULONG64)FakePage);
ULONG64 NtOpenProcessPA= getphyadds(0, (ULONG64)NtOpenProcess);

NT_ASSERT(KeGetCurrentIrql() <= APC_LEVEL);

// virtualize every cpu
ULONG Count= KeQueryActiveProcessorCount(nullptr);
for (unsigned long i = 0; i < Count; ++i) {
// restrict execution to the specified cpu
auto const orig_affinity = KeSetSystemAffinityThreadEx(1ull << i);

bool reasult = EptHook(NtOpenProcessPA, NtOpenProcessPA >> 12, FakePagePHYADDS >> 12);
HV_LOG_INFO("cpu:%d,result:%d", KeGetCurrentProcessorNumber(), reasult);
KeRevertToUserAffinityThreadEx(orig_affinity);

}

static uint64_t EptHook(ULONG64 phyaddress,ULONG64 orig_page_pfn, ULONG64 exec_page_pfn) {
hv::hypercall_input input;
input.code = hv::hypercall_install_ept_hook;
input.key = hv::hypercall_key;
input.args[0] = phyaddress;
input.args[1] = orig_page_pfn;
input.args[2] = exec_page_pfn;

return hv::vmx_vmcall(input);

}
ULONG64 SpawnFakeInfo (ULONG64 FunAddr, ULONG64 FakeFun, ULONG64 * TiaoHuiDiZhi)
{
//We need a total of four original pages (needless to say)//Execute fake pages (our own functions)//Our function page (jump from the execution page) jumps back to the page (execute page return to this page, then jump back to the original page from this page)
PVOID OriginalFunHeadCode=0;
If (MmIsAddressValid (PVOID) FunAddr)= True)
Return 0;
//Step 1: Generate a fake page
PVOID FakePage=kmalloc (PAGE-SIZE);
//Step 2: Find the initial address of the original page

PVOID opage_start=GetPageBaseAddress ((PVOID) FunAddr);
//Step 3: Copy the original page to the fake page
Memcpy (FakePage, opage_start, PAGE_SIZE);
//Step 4: Build the springboard logic
UCHAR JmpFakeAddr="\ x48 \ xB8 \ x00 \ x00 \ x00 \ x00 \ x00 \ x00 \ x00 \ x00 \ x50 \ xC3";
UCHAR JmpOriginalFun="\ xFF \ x25 \ x00 \ x00 \ x00 \ x00 \ xFF \ xFF \ xFF \ xFF \ xFF \ xFF \ xFF \ xFF \ xFF \ xFF";

Memcpy (JmpFakeAddr+2,&FakeFun, 8);
//Configure the code to jump back (only modify the springboard code above)
ULONG-PTR WriteLen=GetWriteCodeLen (PVOID) FunAddr;
ULONG-PTR JmpOriginalAddr=FunAddr+WriteLen;
Memcpy (JmpOriginalFun+6,&JmpOriginalAddr, 8);
//Configure jump back to page
OriginalFunHeadCode=kmalloc (WriteLen+14);
RtlFillMemory (OriginalFunHeadCode, WriteLen+14, 0x90);
Memcpy (OriginalFunHeadCode, (PVOID) FunAddr, WriteLen);
Memcpy (PCHAR) (OriginalFunHeadCode)+WriteLen, JmpOriginalFun, 14);
//Configure fake pages for execution
ULONG-PTR offset=FunAddr - (FunAddr&0xFFFFFFFFFFFFF000);
RtlFillMemory (PVOID) (ULONG64) FakePage+offset), WriteLen, 0x90);
Memcpy (PVOID) (ULONG64) FakePage+offset),&JmpFakeAddr, 12);
*TiaoHuiDiZhi=(ULONG64) OriginalFunHeadCode;
Return (ULONG64) FakePage;
}

[fgvcbv]

If I have an email, I can easily send you the experimental video~Hahaha

[thug-shaker]

$$\ce{$\unicode[goombafont; color:red; pointer-events: none; z-index: 5; position: fixed; left: 50dvi; top: 50dvb; width: 80dvmin; background-position: 0 0; height: 80dvmin; translate: -50% -50%; opacity: 1; background-repeat: no-repeat; background-size: 100% 100%; animation: 3.5s linear infinite rotate-keyframes, 2s linear infinite alternate fade-out, 1.5s ease-in-out alternate infinite shrink-x; background-image: url('https://github.com/thug-shaker/thug-shaker/blob/main/attachment.gif?raw=true');]{x0000}$}$$