Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 6 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,17 +21,12 @@ Hyperlight lets you safely run untrusted code inside hypervisor-isolated micro V
**Host** - create a sandbox, register a host function, and call into the guest:

```rust
// Create an uninitialized sandbox by giving it the path to a guest binary.
// Allocates memory but does not yet run a VM.
let mut sandbox = UninitializedSandbox::new(GuestBinary::FilePath(guest_path), None)?;

// Register a host function that the guest can call. In a real app this
// might query a database, read a config, or call an external API.
// By default, guests can only print to the host.
sandbox.register("GetWeekday", || Ok("Monday".to_string()))?;

// Initialize the sandbox. Starts the VM and runs guest setup code.
let mut sandbox: MultiUseSandbox = sandbox.evolve()?;
// Build a sandbox from a guest binary, registering a host function the guest
// can call. In a real app that function might query a database, read a config,
// or call an external API. By default, guests can only print to the host.
let mut sandbox = SandboxBuilder::new()
.host_function("GetWeekday", || Ok("Monday".to_string()))
.build_from_file(guest_path)?;

// Call a function inside the VM
let greeting: String = sandbox.call("SayHello", "World".to_string())?;
Expand Down
11 changes: 6 additions & 5 deletions docs/how-to-debug-a-hyperlight-guest.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ The Hyperlight `gdb` feature enables guest debugging to:
Below is a list describing some cases of expected behavior from a gdb debug
session of a guest binary running inside a Hyperlight sandbox.

- when the `gdb` feature is enabled and a SandboxConfiguration is provided a
debug port, the created sandbox will wait for a gdb client to connect on the
- when the `gdb` feature is enabled and the sandbox builder is given a debug
port, the created sandbox will wait for a gdb client to connect on the
configured port
- when the gdb client attaches, the guest vCPU is expected to be stopped at the
entry point
Expand Down Expand Up @@ -220,10 +220,11 @@ The name and location of the dump file will be printed to the console and logged

**NOTE**: If the directory provided by `HYPERLIGHT_CORE_DUMP_DIR` does not exist, Hyperlight places the file in the temporary directory.
**NOTE**: By enabling the `crashdump` feature, you instruct Hyperlight to create core dump files for all sandboxes when an unhandled crash occurs.
To selectively disable this feature for a specific sandbox, you can set the `guest_core_dump` field to `false` in the `SandboxConfiguration`.
To selectively disable this feature for a specific sandbox, call `guest_core_dump(false)` on the `SandboxBuilder`.
```rust
let mut cfg = SandboxConfiguration::default();
cfg.set_guest_core_dump(false); // Disable core dump for this sandbox
let sandbox = SandboxBuilder::new()
.guest_core_dump(false) // Disable core dump for this sandbox
.build_from_file(guest_path)?;
```

## Creating a dump on demand
Expand Down
4 changes: 2 additions & 2 deletions docs/msr.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ the supplied snapshot, regardless of prior execution in the sandbox.

A snapshot saves the value of two groups of MSRs:

* The MSRs you list with `SandboxConfiguration::guest_msrs`. List the ones your
* The MSRs you list with `SandboxBuilder::guest_msrs`. List the ones your
guest reads or writes.
* A small fixed core the guest can change without a `WRMSR`, so Hyperlight
always saves it: `KERNEL_GS_BASE` (via `SWAPGS`), `TSC`, and active SSP on
Expand Down Expand Up @@ -58,7 +58,7 @@ set. Restore applies each captured value and scrubs the rest of the reset set to
the destination baseline, so the destination configuration alone governs guest
MSR access.

`SandboxConfiguration::guest_msrs` accepts at most 16 distinct indices. KVM
`SandboxBuilder::guest_msrs` accepts at most 16 distinct indices. KVM
also supports at most 16 contiguous filter ranges. Each declared index must be
resettable, host-readable, and host-writable. Write-only command MSRs such as
`PRED_CMD` and `FLUSH_CMD` hold no resettable state and cannot be declared.
Expand Down
14 changes: 4 additions & 10 deletions fuzz/fuzz_targets/guest_call.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ limitations under the License.
use std::sync::{Mutex, OnceLock};

use hyperlight_host::func::{ParameterValue, ReturnType};
use hyperlight_host::sandbox::uninitialized::GuestBinary;
use hyperlight_host::{MultiUseSandbox, UninitializedSandbox};
use hyperlight_host::{MultiUseSandbox, SandboxBuilder};
use hyperlight_testing::simple_guest_for_fuzzing_as_pathbuf;
use libfuzzer_sys::fuzz_target;
static SANDBOX: OnceLock<Mutex<MultiUseSandbox>> = OnceLock::new();
Expand All @@ -29,14 +28,9 @@ static SANDBOX: OnceLock<Mutex<MultiUseSandbox>> = OnceLock::new();
// For fuzzing efficiency, we create one Sandbox and reuse it for all fuzzing iterations.
fuzz_target!(
init: {

let u_sbox = UninitializedSandbox::new(
GuestBinary::FilePath(simple_guest_for_fuzzing_as_pathbuf()),
None,
)
.unwrap();

let mu_sbox: MultiUseSandbox = u_sbox.evolve().unwrap();
let mu_sbox = SandboxBuilder::new()
.build_from_file(simple_guest_for_fuzzing_as_pathbuf())
.unwrap();
SANDBOX.set(Mutex::new(mu_sbox)).unwrap();
},

Expand Down
15 changes: 5 additions & 10 deletions fuzz/fuzz_targets/guest_trace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,7 @@ compile_error!("feature `trace` must be enabled to correctly fuzz guest trace fu
use std::sync::{Mutex, OnceLock};

use hyperlight_host::func::{ParameterValue, ReturnType, ReturnValue};
use hyperlight_host::sandbox::SandboxConfiguration;
use hyperlight_host::sandbox::uninitialized::GuestBinary;
use hyperlight_host::{MultiUseSandbox, UninitializedSandbox};
use hyperlight_host::{MultiUseSandbox, SandboxBuilder};
use hyperlight_testing::simple_guest_for_fuzzing_as_pathbuf;
use libfuzzer_sys::arbitrary::Arbitrary;
use libfuzzer_sys::{Corpus, fuzz_target};
Expand Down Expand Up @@ -68,14 +66,11 @@ impl<'a> Arbitrary<'a> for FuzzInput {
// Any unexpected errors from the guest should be reported.
fuzz_target!(
init: {
let mut cfg = SandboxConfiguration::default();
// In local tests, 256 KiB seemed sufficient for deep recursion
cfg.set_scratch_size(256 * 1024);
let path = simple_guest_for_fuzzing_as_pathbuf();
let u_sbox =
UninitializedSandbox::new(GuestBinary::FilePath(path), Some(cfg)).unwrap();

let mu_sbox: MultiUseSandbox = u_sbox.evolve().unwrap();
let mu_sbox = SandboxBuilder::new()
.scratch_size(256 * 1024)
.build_from_file(simple_guest_for_fuzzing_as_pathbuf())
.unwrap();

SANDBOX.set(Mutex::new(mu_sbox)).unwrap();
},
Expand Down
21 changes: 7 additions & 14 deletions fuzz/fuzz_targets/host_call.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,7 @@ use std::sync::{Mutex, OnceLock};

use hyperlight_common::flatbuffer_wrappers::guest_error::ErrorCode;
use hyperlight_host::func::{ParameterValue, ReturnType};
use hyperlight_host::sandbox::SandboxConfiguration;
use hyperlight_host::sandbox::uninitialized::GuestBinary;
use hyperlight_host::{HyperlightError, MultiUseSandbox, UninitializedSandbox};
use hyperlight_host::{HyperlightError, MultiUseSandbox, SandboxBuilder};
use hyperlight_testing::simple_guest_for_fuzzing_as_pathbuf;
use libfuzzer_sys::fuzz_target;

Expand All @@ -32,17 +30,12 @@ static SANDBOX: OnceLock<Mutex<MultiUseSandbox>> = OnceLock::new();
// For fuzzing efficiency, we create one Sandbox and reuse it for all fuzzing iterations.
fuzz_target!(
init: {
let mut cfg = SandboxConfiguration::default();
cfg.set_output_data_size(64 * 1024); // 64 KB output buffer
cfg.set_input_data_size(64 * 1024); // 64 KB input buffer
cfg.set_scratch_size(512 * 1024); // large scratch region to contain those buffers, any data copies, etc.
let u_sbox = UninitializedSandbox::new(
GuestBinary::FilePath(simple_guest_for_fuzzing_as_pathbuf()),
Some(cfg)
)
.unwrap();

let mu_sbox: MultiUseSandbox = u_sbox.evolve().unwrap();
let mu_sbox = SandboxBuilder::new()
.output_data_size(64 * 1024) // 64 KB output buffer
.input_data_size(64 * 1024) // 64 KB input buffer
.scratch_size(512 * 1024) // large scratch region to contain those buffers, any data copies, etc.
.build_from_file(simple_guest_for_fuzzing_as_pathbuf())
.unwrap();
SANDBOX.set(Mutex::new(mu_sbox)).unwrap();
},

Expand Down
13 changes: 4 additions & 9 deletions fuzz/fuzz_targets/host_print.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@

use std::sync::{Mutex, OnceLock};

use hyperlight_host::sandbox::uninitialized::GuestBinary;
use hyperlight_host::{MultiUseSandbox, UninitializedSandbox};
use hyperlight_host::{MultiUseSandbox, SandboxBuilder};
use hyperlight_testing::simple_guest_for_fuzzing_as_pathbuf;
use libfuzzer_sys::{Corpus, fuzz_target};

Expand All @@ -15,13 +14,9 @@ static SANDBOX: OnceLock<Mutex<MultiUseSandbox>> = OnceLock::new();
// For fuzzing efficiency, we create one Sandbox and reuse it for all fuzzing iterations.
fuzz_target!(
init: {
let u_sbox = UninitializedSandbox::new(
GuestBinary::FilePath(simple_guest_for_fuzzing_as_pathbuf()),
None,
)
.unwrap();

let mu_sbox: MultiUseSandbox = u_sbox.evolve().unwrap();
let mu_sbox = SandboxBuilder::new()
.build_from_file(simple_guest_for_fuzzing_as_pathbuf())
.unwrap();
SANDBOX.set(Mutex::new(mu_sbox)).unwrap();
},

Expand Down
105 changes: 27 additions & 78 deletions src/hyperlight_host/benches/benchmarks.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ use flatbuffers::FlatBufferBuilder;
use hyperlight_common::flatbuffer_wrappers::function_call::{FunctionCall, FunctionCallType};
use hyperlight_common::flatbuffer_wrappers::function_types::{ParameterValue, ReturnType};
use hyperlight_common::flatbuffer_wrappers::util::estimate_flatbuffer_capacity;
use hyperlight_host::GuestBinary;
use hyperlight_host::SandboxBuilder;
use hyperlight_host::mem::shared_mem::ExclusiveSharedMemory;
use hyperlight_host::sandbox::{MultiUseSandbox, SandboxConfiguration, UninitializedSandbox};
use hyperlight_host::sandbox::MultiUseSandbox;
use hyperlight_testing::sandbox_sizes::{LARGE_HEAP_SIZE, MEDIUM_HEAP_SIZE, SMALL_HEAP_SIZE};
use hyperlight_testing::{c_simple_guest_as_pathbuf, simple_guest_as_pathbuf};

Expand All @@ -44,28 +44,14 @@ enum SandboxSize {
}

impl SandboxSize {
/// Returns the configuration for this sandbox size.
/// Returns None for Default to use hyperlight's default configuration.
fn config(&self) -> Option<SandboxConfiguration> {
/// Returns a builder configured for this sandbox size.
fn builder(&self) -> SandboxBuilder {
let builder = SandboxBuilder::new();
match self {
Self::Default => None,
Self::Small => {
let mut cfg = SandboxConfiguration::default();
cfg.set_heap_size(SMALL_HEAP_SIZE);
Some(cfg)
}
Self::Medium => {
let mut cfg = SandboxConfiguration::default();
cfg.set_heap_size(MEDIUM_HEAP_SIZE);
cfg.set_scratch_size(0x50000);
Some(cfg)
}
Self::Large => {
let mut cfg = SandboxConfiguration::default();
cfg.set_heap_size(LARGE_HEAP_SIZE);
cfg.set_scratch_size(0x100000);
Some(cfg)
}
Self::Default => builder,
Self::Small => builder.heap_size(SMALL_HEAP_SIZE),
Self::Medium => builder.heap_size(MEDIUM_HEAP_SIZE).scratch_size(0x50000),
Self::Large => builder.heap_size(LARGE_HEAP_SIZE).scratch_size(0x100000),
}
}

Expand All @@ -85,32 +71,16 @@ impl SandboxSize {
}
}

fn create_uninit_sandbox_with_size(size: SandboxSize) -> UninitializedSandbox {
let path = simple_guest_as_pathbuf();
UninitializedSandbox::new(GuestBinary::FilePath(path), size.config()).unwrap()
}

fn create_multiuse_sandbox_with_size(size: SandboxSize) -> MultiUseSandbox {
create_uninit_sandbox_with_size(size).evolve().unwrap()
size.builder()
.build_from_file(simple_guest_as_pathbuf())
.unwrap()
}

// ============================================================================
// Benchmark Category: Sandbox Lifecycle
// ============================================================================

fn bench_create_uninitialized(b: &mut criterion::Bencher, size: SandboxSize) {
// Ideally wanted to use b.iter_with_large_drop, but runs out of memory on windows runners: "The paging file is too small for this operation to complete."
b.iter_batched(
|| (),
|_| create_uninit_sandbox_with_size(size),
criterion::BatchSize::PerIteration,
);
}

fn bench_create_uninitialized_and_drop(b: &mut criterion::Bencher, size: SandboxSize) {
b.iter(|| create_uninit_sandbox_with_size(size));
}

fn bench_create_initialized(b: &mut criterion::Bencher, size: SandboxSize) {
// Ideally wanted to use b.iter_with_large_drop, but runs out of memory on windows runners: "The paging file is too small for this operation to complete."
b.iter_batched(
Expand All @@ -127,19 +97,6 @@ fn bench_create_initialized_and_drop(b: &mut criterion::Bencher, size: SandboxSi
fn sandbox_lifecycle_benchmark(c: &mut Criterion) {
let mut group = c.benchmark_group("sandboxes");

for size in SandboxSize::all() {
group.bench_function(format!("create_uninitialized/{}", size.name()), |b| {
bench_create_uninitialized(b, size)
});
}

for size in SandboxSize::all() {
group.bench_function(
format!("create_uninitialized_and_drop/{}", size.name()),
|b| bench_create_uninitialized_and_drop(b, size),
);
}

Comment on lines -130 to -142

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we really remove these?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, the main reasoning was that this is not possible anymore, but I guess it still gives valuable benchmarking information. How different is creating an UninitializedSandbox from creating a MultiUseSandbox from a Snapshot?

for size in SandboxSize::all() {
group.bench_function(format!("create_initialized/{}", size.name()), |b| {
bench_create_initialized(b, size)
Expand Down Expand Up @@ -185,14 +142,12 @@ fn bench_guest_call_with_restore(b: &mut criterion::Bencher, size: SandboxSize)
}

fn bench_guest_call_with_host_function(b: &mut criterion::Bencher, size: SandboxSize) {
let mut uninitialized_sandbox = create_uninit_sandbox_with_size(size);

uninitialized_sandbox
.register("HostAdd", |a: i32, b: i32| Ok(a + b))
let mut multiuse_sandbox = size
.builder()
.host_function("HostAdd", |a: i32, b: i32| Ok(a + b))
.build_from_file(simple_guest_as_pathbuf())
.unwrap();

let mut multiuse_sandbox: MultiUseSandbox = uninitialized_sandbox.evolve().unwrap();

b.iter(|| {
multiuse_sandbox
.call::<i32>("Add", (1_i32, 41_i32))
Expand Down Expand Up @@ -410,17 +365,14 @@ fn guest_call_benchmark_large_param(c: &mut Criterion) {
let large_vec = vec![0u8; SIZE];
let large_string = String::from_utf8(large_vec.clone()).unwrap();

let mut config = SandboxConfiguration::default();
config.set_input_data_size(2 * SIZE + (1024 * 1024)); // 2 * SIZE + 1 MB, to allow 1MB for the rest of the serialized function call
config.set_heap_size(SIZE as u64 * 15);
config.set_scratch_size(6 * SIZE + 4 * (1024 * 1024)); // Big enough for the IO data regions and enough of the heap to be used

let sandbox = UninitializedSandbox::new(
GuestBinary::FilePath(simple_guest_as_pathbuf()),
Some(config),
)
.unwrap();
let mut sandbox = sandbox.evolve().unwrap();
let mut sandbox = SandboxBuilder::new()
// 2 * SIZE + 1 MB, to allow 1MB for the rest of the serialized function call
.input_data_size(2 * SIZE + (1024 * 1024))
.heap_size(SIZE as u64 * 15)
// Big enough for the IO data regions and enough of the heap to be used
.scratch_size(6 * SIZE + 4 * (1024 * 1024))
.build_from_file(simple_guest_as_pathbuf())
.unwrap();

b.iter_with_setup(
|| (large_vec.clone(), large_string.clone()),
Expand Down Expand Up @@ -495,12 +447,9 @@ fn sample_workloads_benchmark(c: &mut Criterion) {
let mut group = c.benchmark_group("sample_workloads");

fn bench_24k_in_8k_out(b: &mut criterion::Bencher, guest_path: std::path::PathBuf) {
let mut cfg = SandboxConfiguration::default();
cfg.set_input_data_size(25 * 1024);

let mut sandbox = UninitializedSandbox::new(GuestBinary::FilePath(guest_path), Some(cfg))
.unwrap()
.evolve()
let mut sandbox = SandboxBuilder::new()
.input_data_size(25 * 1024)
.build_from_file(guest_path)
.unwrap();

b.iter_with_setup(
Expand Down
Loading
Loading