Memory access tracking enables quoting

[rndmcnlly]

This is a post about using V86 to enable creating playable quotes for PC games. We are seeking community input on how to support this in the V86 codebase without making a mess or creating an unmaintainable fork.

In my research lab at UC Santa Cruz, we're developing the idea of playable quotes. PQs feel like screen recording videos in that they capture how someone interacts with a system (e.g. VM/emulator). Unlike videos, they also feel like savestates in that they let you resume interaction at any moment you see and make new choices. Like a quote from a book, they contain only a tiny fraction of the data of the original work while still letting your audience experience the specific moments you wanted to share with them. Ideally, for a fraction of the storage cost associated with a single VM snapshot, you could perfectly record an interaction with the VM in a way that gives audiences limited ability to have novel but similar interactive choices in the future.

While my lab initial implemented PQs for Game Boy games, we want to do general PC software next using V86. The main obstacle in getting PQs working is the ability to track which chunks of memory are actively used during an interactive demonstration. Using some pretty tricky dynamic binary rewriting methods, we produced some fun demos with V86:

• Explainer video: https://youtu.be/z9JYOZWLMlo?t=1410
• Lightweight demo (Windows 1.0): http://v86-instrumentation.glitch.me/
• Heavyweight demo (Doom): https://v86-instrumentation-advanced.glitch.me/

We think our approach could be both simpler and more efficient if we tapped into the memory paging system that V86 already models. Instead of trying to monitor every time black-boxed V86 touches any part of wasm memory, we think V86 could actively report whenever it maps a new guest physical page for access using the TLB.

The same mechanism that allows memory access tracking for PQs could also enable demand paging for guest physical memory. V86 can already lazy-load disk blocks, but it can't do this for memory blocks yet.

Dream: TLB manipulations (e.g. those currently happening in do_page_walk of cpu.rs) would be reported back out to JavaScript. These calls would be blocking so that JavaScript code could just-in-time prepare pages that weren't really ready yet.

Does anyone have a vision for how to do this? I can't help but imagine the mechanisms needed to support PQs could also power a bunch of other applications.

[copy]

This looks pretty cool. I think we could support the required hooks in v86. To give you some pointers, have a look at cpu_exception_hook, which is used for some tests:

v86/tests/nasm/run.js

Line 331 in 368efb0

emulator.cpu_exception_hook = function(n)

v86/src/rust/cpu/cpu.rs

Line 3153 in 368efb0

if cpu_exception_hook(CPU_EXCEPTION_DE) {

v86/src/browser/starter.js

Line 134 in 368efb0

"cpu_exception_hook": n => this.cpu_exception_hook(n),

Deterministic replay could be very useful for various forms of oracle testing, e.g. running the emulator once with jit and once without, and checking if they diverge at any point.