Make Launcher use ClientDescription instead of CoreId (AFLplusplus#2676)

* launcher now uses client_id instead of core_id * adding overcommit to an example fuzzer * Replace addr_of with &raw across the codebase (AFLplusplus#2669) * Replace addr_of with &raw across the codebase * fix fixes * more fix * undo clang fmt? * oops * fix? * allocator fix * more fix * more more * more docs * more fix * mas mas mas * hm * more * fix Frida * needed * more error * qemu * Introduce workspace (again) (AFLplusplus#2673) * Trying to redo workspace deps again after AFLplusplus#2672 * unused * clippy * Replace addr_of with &raw across the codebase (AFLplusplus#2669) * Replace addr_of with &raw across the codebase * fix fixes * more fix * undo clang fmt? * oops * fix? * allocator fix * more fix * more more * more docs * more fix * mas mas mas * hm * more * fix Frida * needed * more error * qemu * Introduce workspace (again) (AFLplusplus#2673) * Trying to redo workspace deps again after AFLplusplus#2672 * unused * clippy * fixing formatting issues * cloning values to make borrow checker happy * simplifying cfg constraints, removing excessive clippy allows * printing clang version that is used to find inconsistencies between CI and local formatting * some fixes according to the CI * Specifying types * improved logging for formatter * more attempts at logging for the CI formatting * fixing setting LLVM version in formatting in CI * fixing cippy allows * renaming launcher's ClientID to ClientDescription * Lower capped RAND generators (AFLplusplus#2671) * Lower capped rand generators * Updated all references to RAND generators * Formatting updates * New RAND bytes generator constructor * Revert "Updated all references to RAND generators" This reverts commit 9daad89. * Revert "Formatting updates" This reverts commit ff2a61a. * cargo nightly format * Added must_use to with_min_size * fix error '#' is not followed by a macro parameter (AFLplusplus#2678) * Use version.workspace (AFLplusplus#2682) * LibAFL_QEMU: Don't return a generic Address from Register reads (AFLplusplus#2681) * LibAFL_QEMU: Make ReadReg always return GuestReg type * Don't return a generic address * fix fuzzers * fix mips * Add DrCovReader to read DrCov files and DrCov dumper and merge utils (AFLplusplus#2680) * Add DrCov Reader * Removed libafl_jumper deps * Fix DrCovWriter, add dump_drcov_addrs * Taplo * Move frida from usize to u64 * DrCov usize=>u64 * Better error print * More u64 * ? * debug * clippy * clippy * Add Merge option to DrCovReader * Add drcov_merge tool * Move folder around * DrCov * More assert * fmt * Move around * Fix print * Add option to read multiple files/full folders * Fix build_all_fuzzers.sh for local runs (AFLplusplus#2686) * Add Intel PT tracing support (AFLplusplus#2471) * WIP: IntelPT qemu systemmode * use perf-event-open-sys instead of bindgen * intelPT Add enable and disable tracing, add test * Use static_assertions crate * Fix volatiles, finish test * Add Intel PT availability check * Use LibAFL errors in Result * Improve filtering * Add KVM pt_mode check * move static_assertions use * Check for perf_event_open support * Add (empty) IntelPT module * Add IntelPTModule POC * partial ideas to implement intel pt * forgot smth * trace decoding draft * add libipt decoder * use cpuid instead of reading /proc/cpuinfo * investigating nondeterministic behaviour * intel_pt module add thread creation hook * Fully identify deps versions Cargo docs: Although it looks like a specific version of the crate, it actually specifies a range of versions and allows SemVer compatible updates * Move mem image to module, output to file for debug * fixup! Use static_assertions crate * Exclude host kernel from traces * Bump libipt-rs * Callback to get memory as an alterantive to image * WIP Add bootloader fuzzer example * Split availability check: add availability_with_qemu * Move IntelPT to observer * Improve test docs * Clippy happy now * Taplo happy now * Add IntelPTObserver boilerplate * Hook instead of Observer * Clippy & Taplo * Add psb_freq setting * Extremely bad and dirty babyfuzzer stealing * Use thread local cell instead of mutex * Try a trace diff based naive feedback * fix perf aux buffer wrap handling * Use f64 for feedback score * Fix clippy for cargo test * Add config format tests * WIP intelpt babyfuzzer with fork * Fix not wrapped tail offset in split buffer * Baby PT with raw traces diff working * Cache nr_filters * Use Lazy_lock for perf_type * Add baby_fuzzer_intel_pt * restore baby fuzzer * baby_fuzzer with block decoder * instruction decoder instead of block * Fix after upstream merge * OwnedRefMut instead of Cow * Read mem directly instead of going through files * Fix cache lifetime and tail update * clippy * Taplo * Compile caps only on linux * clippy * Fail compilation on unsupported OSes * Add baby_fuzzer_intel_pt to CI * Cleanup * Move intel pt + linux check * fix baby pt * rollback forkexecutor * Remove unused dep * Cleanup * Lints * Compute an edge id instead of using only block ip * Binary only intelPT POC * put linux specific code behind target_os=linux * Clippy & Taplo * fix CI * Disable relocation * No unwrap in decode * No expect in decode * Better logging, smaller aux buffer * add IntelPTBuilder * some lints * Add exclude_hv config * Per CPU tracing and inheritance * Parametrize buffer size * Try not to break commandExecutor API pt.1 * Try not to break commandExecutor API pt.2 * Try not to break commandExecutor API pt.3 * fix baby PT * Support on_crash & on_timeout callbacks for libafl_qemu modules (AFLplusplus#2620) * support (unsafe) on_crash / on_timeout callbacks for modules * use libc types in bindgen * Move common code to bolts * Cleanup * Revert changes to backtrace_baby_fuzzers/command_executor * Move intel_pt in one file * Use workspace deps * add nr_addr_filter fallback * Cleaning * Improve decode * Clippy * Improve errors and docs * Impl from<PtError> for libafl::Error * Merge hooks * Docs * Clean command executor * fix baby PT * fix baby PT warnings * decoder fills the map with no vec alloc * WIP command executor intel PT * filter_map() instead of filter().map() * fix docs * fix windows? * Baby lints * Small cleanings * Use personality to disable ASLR at runtime * Fix nix dep * Use prc-maps in babyfuzzer * working ET_DYN elf * Cleanup Cargo.toml * Clean command executor * introduce PtraceCommandConfigurator * Fix clippy & taplo * input via stdin * libipt as workspace dep * Check kernel version * support Arg input location * Reorder stuff * File input * timeout support for PtraceExec * Lints * Move out method not needing self form IntelPT * unimplemented * Lints * Move intel_pt_baby_fuzzer * Move intel_pt_command_executor * Document the need for smp_rmb * Better comment * Readme and Makefile.toml instead of build.rs * Move out from libafl_bolts to libafl_intelpt * Fix hooks * (Almost) fix intel_pt command exec * fix intel_pt command exec debug * Fix baby_fuzzer * &raw over addr_of! * cfg(target_os = "linux") * bolts Cargo.toml leftover * minimum wage README.md * extract join_split_trace from decode * extract decode_block from decode * add 1 to `previous_block_ip` to avoid that all the recursive basic blocks map to 0 * More generic hook * fix windows * Update CI, fmt * No bitbybit * Fix docker? * Fix Apple silicon? * Use old libipt from crates.io --------- Co-authored-by: Romain Malmain <[email protected]> Co-authored-by: Dominik Maier <[email protected]> * libafl-fuzz: introduce nyx_mode (AFLplusplus#2503) * add nyx_mode * fix frida ci? * damn clippy * clippy * LibAFL: Remove `tui_monitor` from default features (AFLplusplus#2685) * No Usermode default * no tui * gg * try fix CI * fmt --------- Co-authored-by: Dominik Maier <[email protected]> * Actually make ConstMapObserver work, introduce `nonnull_raw_mut` macro (AFLplusplus#2687) * Actually make ConstMapObserver work * fixes * does that work? * mas * Feature: libafl-fuzzfuzzbench (AFLplusplus#2689) * fuzzbench * clippy * fmt * fix unicorn CI? * Move bitfields to bitbybit (AFLplusplus#2688) * move to bitbybit * Restore bitbybit dependent code * Clippy * Fix NautilusContext::from_file for python files (AFLplusplus#2690) * Bump to 0.14.0 (AFLplusplus#2692) * Fix versions in libafl and libafl_intelpt for crates.io (AFLplusplus#2693) * Fix versions in libafl and libafl_intelpt for crates * Add libafl_intelpt to publish * StdMOptMutator::new: remove unused type parameter (AFLplusplus#2695) `I` is unused in `::new` and thus requires callers to explicitly specify any type as it can't be determined by type inference. Clippy's `extra_unused_type_parameters` should pick this up, but is tuned a bit too conservative in order to avoid false positives AFAICT. * Move test_harness from source directory to OUT_DIR (AFLplusplus#2694) * remove test_harness from source directory * fmt * Add package.metadata.docs.rs for libafl_intelpt (AFLplusplus#2696) * libafl-fuzz: fix cmplog running on inputs more than once (AFLplusplus#2697) * libafl-fuzz: fix cmplog running on inputs more than once * fmt * fix afl++ cmplog header * update to latest afl stable commit * Libafl workspace internal deps in workspace Cargo.toml (AFLplusplus#2691) * Add internal deps to workspace * libafl: use workspace internal deps * libafl_bolts: use workspace internal deps * 0.14.0 * use workspace internal deps * Fix tui monitor for example fuzzers (AFLplusplus#2699) * Fix tui monitor for example fuzzers * New clippy lint * fix * Update pyo3-build-config requirement from 0.22.3 to 0.23.1 (AFLplusplus#2701) Updates the requirements on [pyo3-build-config](https://github.com/pyo3/pyo3) to permit the latest version. - [Release notes](https://github.com/pyo3/pyo3/releases) - [Changelog](https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md) - [Commits](PyO3/pyo3@v0.22.3...v0.23.1) --- updated-dependencies: - dependency-name: pyo3-build-config dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * bolts: fix build for tiers 3 platforms. (AFLplusplus#2700) cater to platforms knowingly support this feature instead. * Pre init module hooks (AFLplusplus#2704) * differenciate pre qemu init and post qemu init hooks * api breakage: Emulator::new_with_qemu is not public anymore. * Fix edge module generators (AFLplusplus#2702) * fix generators * fix metadata removal for ExecutionCountRestartHelper (AFLplusplus#2705) * Ignore pyo3 update (AFLplusplus#2709) * libafl-fuzz: feature-flag nyx mode (AFLplusplus#2712) * Bump ctor dependency to make nightly compile again (AFLplusplus#2713) * Batched timeout doc (AFLplusplus#2716) * timeout doc * clp * FMT * More batched timeout doc (AFLplusplus#2717) * timeout doc * clp * FMT * more * fixing an overexited cast * renaming variables * removing unnecessary brackets * fixing imports * fixing imports * renaming more variables * even more variable renaming * removing duplicate clap short options * reverting mistaken variable renaming * comparing the actual cores instead of an enumeration index --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Dominik Maier <[email protected]> Co-authored-by: Subhojeet Mukherjee, PhD <[email protected]> Co-authored-by: jejuisland87654 <[email protected]> Co-authored-by: Marco C. <[email protected]> Co-authored-by: Dongjia "toka" Zhang <[email protected]> Co-authored-by: Romain Malmain <[email protected]> Co-authored-by: Aarnav <[email protected]> Co-authored-by: Dominik Maier <[email protected]> Co-authored-by: Andrea Fioraldi <[email protected]> Co-authored-by: Mrmaxmeier <[email protected]> Co-authored-by: Sharad Khanna <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: David CARLIER <[email protected]> Co-authored-by: Henry Chu <[email protected]>
riesentoaster · Dec 11, 2024 · 2748326 · 2748326
1 parent ee70c78
commit 2748326
Show file tree

Hide file tree

Showing 27 changed files with 786 additions and 608 deletions.
diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml
@@ -194,7 +194,7 @@ jobs:
   cargo-fmt:
     runs-on: ubuntu-24.04
     env:
-       MAIN_LLVM_VERSION: 19
+      MAIN_LLVM_VERSION: 19
     steps:
       - uses: actions/checkout@v4
       - uses: ./.github/workflows/ubuntu-prepare

diff --git a/fuzzers/binary_only/frida_executable_libpng/src/fuzzer.rs b/fuzzers/binary_only/frida_executable_libpng/src/fuzzer.rs
@@ -5,7 +5,9 @@ use std::{path::PathBuf, ptr::null};
 use frida_gum::Gum;
 use libafl::{
     corpus::{CachedOnDiskCorpus, Corpus, OnDiskCorpus},
-    events::{launcher::Launcher, llmp::LlmpRestartingEventManager, EventConfig},
+    events::{
+        launcher::Launcher, llmp::LlmpRestartingEventManager, ClientDescription, EventConfig,
+    },
     executors::{inprocess::InProcessExecutor, ExitKind, ShadowExecutor},
     feedback_or, feedback_or_fast,
     feedbacks::{CrashFeedback, MaxMapFeedback, TimeFeedback, TimeoutFeedback},
@@ -93,13 +95,17 @@ unsafe fn fuzz(
 
     let shmem_provider = StdShMemProvider::new()?;
 
-    let mut run_client = |state: Option<_>, mgr: LlmpRestartingEventManager<_, _, _>, core_id| {
+    let mut run_client = |state: Option<_>,
+                          mgr: LlmpRestartingEventManager<_, _, _>,
+                          client_description: ClientDescription| {
         // The restarting state will spawn the same process again as child, then restarted it each time it crashes.
 
         // println!("{:?}", mgr.mgr_id());
 
-        if options.asan && options.asan_cores.contains(core_id) {
-            (|state: Option<_>, mut mgr: LlmpRestartingEventManager<_, _, _>, _core_id| {
+        if options.asan && options.asan_cores.contains(client_description.core_id()) {
+            (|state: Option<_>,
+              mut mgr: LlmpRestartingEventManager<_, _, _>,
+              _client_description| {
                 let gum = Gum::obtain();
 
                 let coverage = CoverageRuntime::new();
@@ -222,9 +228,11 @@ unsafe fn fuzz(
                 fuzzer.fuzz_loop(&mut stages, &mut executor, &mut state, &mut mgr)?;
 
                 Ok(())
-            })(state, mgr, core_id)
-        } else if options.cmplog && options.cmplog_cores.contains(core_id) {
-            (|state: Option<_>, mut mgr: LlmpRestartingEventManager<_, _, _>, _core_id| {
+            })(state, mgr, client_description)
+        } else if options.cmplog && options.cmplog_cores.contains(client_description.core_id()) {
+            (|state: Option<_>,
+              mut mgr: LlmpRestartingEventManager<_, _, _>,
+              _client_description| {
                 let gum = Gum::obtain();
 
                 let coverage = CoverageRuntime::new();
@@ -356,9 +364,11 @@ unsafe fn fuzz(
                 fuzzer.fuzz_loop(&mut stages, &mut executor, &mut state, &mut mgr)?;
 
                 Ok(())
-            })(state, mgr, core_id)
+            })(state, mgr, client_description)
         } else {
-            (|state: Option<_>, mut mgr: LlmpRestartingEventManager<_, _, _>, _core_id| {
+            (|state: Option<_>,
+              mut mgr: LlmpRestartingEventManager<_, _, _>,
+              _client_description| {
                 let gum = Gum::obtain();
 
                 let coverage = CoverageRuntime::new();
@@ -473,7 +483,7 @@ unsafe fn fuzz(
                 fuzzer.fuzz_loop(&mut stages, &mut executor, &mut state, &mut mgr)?;
 
                 Ok(())
-            })(state, mgr, core_id)
+            })(state, mgr, client_description)
         }
     };
 

diff --git a/fuzzers/binary_only/frida_libpng/src/fuzzer.rs b/fuzzers/binary_only/frida_libpng/src/fuzzer.rs
@@ -5,7 +5,9 @@ use std::path::PathBuf;
 use frida_gum::Gum;
 use libafl::{
     corpus::{CachedOnDiskCorpus, Corpus, OnDiskCorpus},
-    events::{launcher::Launcher, llmp::LlmpRestartingEventManager, EventConfig},
+    events::{
+        launcher::Launcher, llmp::LlmpRestartingEventManager, ClientDescription, EventConfig,
+    },
     executors::{inprocess::InProcessExecutor, ExitKind, ShadowExecutor},
     feedback_or, feedback_or_fast,
     feedbacks::{CrashFeedback, MaxMapFeedback, TimeFeedback, TimeoutFeedback},
@@ -73,7 +75,9 @@ unsafe fn fuzz(options: &FuzzerOptions) -> Result<(), Error> {
 
     let shmem_provider = StdShMemProvider::new()?;
 
-    let mut run_client = |state: Option<_>, mgr: LlmpRestartingEventManager<_, _, _>, core_id| {
+    let mut run_client = |state: Option<_>,
+                          mgr: LlmpRestartingEventManager<_, _, _>,
+                          client_description: ClientDescription| {
         // The restarting state will spawn the same process again as child, then restarted it each time it crashes.
 
         // println!("{:?}", mgr.mgr_id());
@@ -90,8 +94,10 @@ unsafe fn fuzz(options: &FuzzerOptions) -> Result<(), Error> {
             ExitKind::Ok
         };
 
-        if options.asan && options.asan_cores.contains(core_id) {
-            (|state: Option<_>, mut mgr: LlmpRestartingEventManager<_, _, _>, _core_id| {
+        if options.asan && options.asan_cores.contains(client_description.core_id()) {
+            (|state: Option<_>,
+              mut mgr: LlmpRestartingEventManager<_, _, _>,
+              _client_description| {
                 let gum = Gum::obtain();
 
                 let coverage = CoverageRuntime::new();
@@ -214,9 +220,11 @@ unsafe fn fuzz(options: &FuzzerOptions) -> Result<(), Error> {
                 fuzzer.fuzz_loop(&mut stages, &mut executor, &mut state, &mut mgr)?;
 
                 Ok(())
-            })(state, mgr, core_id)
-        } else if options.cmplog && options.cmplog_cores.contains(core_id) {
-            (|state: Option<_>, mut mgr: LlmpRestartingEventManager<_, _, _>, _core_id| {
+            })(state, mgr, client_description)
+        } else if options.cmplog && options.cmplog_cores.contains(client_description.core_id()) {
+            (|state: Option<_>,
+              mut mgr: LlmpRestartingEventManager<_, _, _>,
+              _client_description| {
                 let gum = Gum::obtain();
 
                 let coverage = CoverageRuntime::new();
@@ -349,9 +357,11 @@ unsafe fn fuzz(options: &FuzzerOptions) -> Result<(), Error> {
                 fuzzer.fuzz_loop(&mut stages, &mut executor, &mut state, &mut mgr)?;
 
                 Ok(())
-            })(state, mgr, core_id)
+            })(state, mgr, client_description)
         } else {
-            (|state: Option<_>, mut mgr: LlmpRestartingEventManager<_, _, _>, _core_id| {
+            (|state: Option<_>,
+              mut mgr: LlmpRestartingEventManager<_, _, _>,
+              _client_description| {
                 let gum = Gum::obtain();
 
                 let coverage = CoverageRuntime::new();
@@ -466,7 +476,7 @@ unsafe fn fuzz(options: &FuzzerOptions) -> Result<(), Error> {
                 fuzzer.fuzz_loop(&mut stages, &mut executor, &mut state, &mut mgr)?;
 
                 Ok(())
-            })(state, mgr, core_id)
+            })(state, mgr, client_description)
         }
     };
 

diff --git a/fuzzers/binary_only/frida_windows_gdiplus/src/fuzzer.rs b/fuzzers/binary_only/frida_windows_gdiplus/src/fuzzer.rs
@@ -22,7 +22,9 @@ use std::path::PathBuf;
 use frida_gum::Gum;
 use libafl::{
     corpus::{CachedOnDiskCorpus, Corpus, OnDiskCorpus},
-    events::{launcher::Launcher, llmp::LlmpRestartingEventManager, EventConfig},
+    events::{
+        launcher::Launcher, llmp::LlmpRestartingEventManager, ClientDescription, EventConfig,
+    },
     executors::{inprocess::InProcessExecutor, ExitKind, ShadowExecutor},
     feedback_and_fast, feedback_or, feedback_or_fast,
     feedbacks::{ConstFeedback, CrashFeedback, MaxMapFeedback, TimeFeedback, TimeoutFeedback},
@@ -82,7 +84,9 @@ unsafe fn fuzz(options: &FuzzerOptions) -> Result<(), Error> {
 
     let shmem_provider = StdShMemProvider::new()?;
 
-    let mut run_client = |state: Option<_>, mgr: LlmpRestartingEventManager<_, _, _>, core_id| {
+    let mut run_client = |state: Option<_>,
+                          mgr: LlmpRestartingEventManager<_, _, _>,
+                          client_description: ClientDescription| {
         // The restarting state will spawn the same process again as child, then restarted it each time it crashes.
 
         // println!("{:?}", mgr.mgr_id());
@@ -99,8 +103,10 @@ unsafe fn fuzz(options: &FuzzerOptions) -> Result<(), Error> {
             ExitKind::Ok
         };
 
-        if options.asan && options.asan_cores.contains(core_id) {
-            (|state: Option<_>, mut mgr: LlmpRestartingEventManager<_, _, _>, _core_id| {
+        if options.asan && options.asan_cores.contains(client_description.core_id()) {
+            (|state: Option<_>,
+              mut mgr: LlmpRestartingEventManager<_, _, _>,
+              _client_description| {
                 let gum = Gum::obtain();
 
                 let coverage = CoverageRuntime::new();
@@ -212,9 +218,11 @@ unsafe fn fuzz(options: &FuzzerOptions) -> Result<(), Error> {
                 fuzzer.fuzz_loop(&mut stages, &mut executor, &mut state, &mut mgr)?;
 
                 Ok(())
-            })(state, mgr, core_id)
-        } else if options.cmplog && options.cmplog_cores.contains(core_id) {
-            (|state: Option<_>, mut mgr: LlmpRestartingEventManager<_, _, _>, _core_id| {
+            })(state, mgr, client_description)
+        } else if options.cmplog && options.cmplog_cores.contains(client_description.core_id()) {
+            (|state: Option<_>,
+              mut mgr: LlmpRestartingEventManager<_, _, _>,
+              _client_description| {
                 let gum = Gum::obtain();
 
                 let coverage = CoverageRuntime::new();
@@ -340,9 +348,11 @@ unsafe fn fuzz(options: &FuzzerOptions) -> Result<(), Error> {
                 fuzzer.fuzz_loop(&mut stages, &mut executor, &mut state, &mut mgr)?;
 
                 Ok(())
-            })(state, mgr, core_id)
+            })(state, mgr, client_description)
         } else {
-            (|state: Option<_>, mut mgr: LlmpRestartingEventManager<_, _, _>, _core_id| {
+            (|state: Option<_>,
+              mut mgr: LlmpRestartingEventManager<_, _, _>,
+              _client_description| {
                 let gum = Gum::obtain();
 
                 let coverage = CoverageRuntime::new();
@@ -454,7 +464,7 @@ unsafe fn fuzz(options: &FuzzerOptions) -> Result<(), Error> {
                     .unwrap();
 
                 Ok(())
-            })(state, mgr, core_id)
+            })(state, mgr, client_description)
         }
     };