WebGPU-based renderer for the editor

[Tyriar]

We're finally starting to look at implementing a WebGPU-based rendering in monaco, similar to what xterm.js uses. This issue is used to track all the work which is expected to take several months.

Project: VS Code Editor GPU Renderer (view)

Related issues

Here are some historical links that might be useful:

• Zero latency typing issue which investigated typing latency in monaco. This lead to some improvements but eventually came to the conclusion that the biggest win would be doing rendering manually using our own shaders Feature Request: Zero-latency Typing #27378
• Old exploration issue Explore GPU rendering in the editor #162445
• Internal issue describing my 2022 prototype which is based on xterm.js' webgl renderer https://github.com/microsoft/vscode-internalbacklog/issues/3157
  • The branch https://github.com/microsoft/vscode/tree/tyriar/webgl_monaco
• Internal issue describing my 2024 prototype which is based on a game renderer I was experimenting with in my spare time https://github.com/microsoft/vscode-internalbacklog/issues/4906
  • The branch https://github.com/microsoft/vscode/tree/tyriar/gpu_exploration

---

Below copied from https://github.com/microsoft/vscode-internalbacklog/issues/4906

GPU-based rendering

branch: tyriar/gpu_exploration

How GPU rendering works

It works by assembling array buffers which represent commands to run on the GPU, these are filled on the CPU with information like the texture to use (chracter, fg, bg), location, offset, etc. xterm.js for example allocates a cols x rows array buffer that represents the viewport only and updates it on every frame where the viewport changes.

There are 2 types of shaders:

• Vertex shader - This is run for every vertex (4 vertices per quad) and is used to transform the vertices into screen space.
• Fragment shader - This is run for every pixel in the quad and is used to determine the color of the pixel.

How the prototype works

The WebGPU prototype works by pre-allocating a buffer that represents up to 3000 lines in a file with a maximum column length of 200. The buffers* are lazily filled in based on what's the viewport. Meaning once a line is loaded, it doesn't need to be modified again. I think it updates more aggressively currently than needed due to my lack of knowledge around finding dirty lines in Monaco.

@vertex fn vs(
	vert: Vertex,
	@builtin(instance_index) instanceIndex: u32,
	@builtin(vertex_index) vertexIndex : u32
) -> VSOutput {
	let dynamicUnitInfo = dynamicUnitInfoStructs[instanceIndex];
	let spriteInfo = spriteInfo[u32(dynamicUnitInfo.textureIndex)];

	var vsOut: VSOutput;
	// Multiple vert.position by 2,-2 to get it into clipspace which ranged from -1 to 1
	vsOut.position = vec4f(
		(((vert.position * vec2f(2, -2)) / uniforms.canvasDimensions)) * spriteInfo.size + dynamicUnitInfo.position + ((spriteInfo.origin * vec2f(2, -2)) / uniforms.canvasDimensions) + ((scrollOffset.offset * 2) / uniforms.canvasDimensions),
		0.0,
		1.0
	);

	// Textures are flipped from natural direction on the y-axis, so flip it back
	vsOut.texcoord = vert.position;
	vsOut.texcoord = (
		// Sprite offset (0-1)
		(spriteInfo.position / textureInfoUniform.spriteSheetSize) +
		// Sprite coordinate (0-1)
		(vsOut.texcoord * (spriteInfo.size / textureInfoUniform.spriteSheetSize))
	);

	return vsOut;
}

@fragment fn fs(vsOut: VSOutput) -> @location(0) vec4f {
	return textureSample(ourTexture, ourSampler, vsOut.texcoord);
}

Texture atlas

Glyphs are rendered on the CPU using the browser's canvas 2d context to draw the characters into a texture atlas. The texture atlas can have multiple pages, this is an optimization problem as uploading images is relative expensive. xterm.js creates multiple small texture atlas pages, allocates using a shelf allocator and eventually merged them into larger immutable pages as they're more expensive to upload.

Currently the prototype uses a single large texture atlas page, but it warms it up in idle callbacks for the current font and all theme token colors in the background (using the TaskQueue xterm.js util).

Memory usage

text_data_buffer: [wgslX, wgslY, textureIndex, ...]

texture_atlas_buffer: [positionX, positionY, sizeX, sizeY, offsetX, offsetY, ...]

textureIndex in text_data_buffer maps to texture_atlas_buffer[textureIndex * 6]

In the above, each text_data_buffer cell is 12 bytes (3x 32-bit floats), so 3000x200 would be:

3000 * 200 * 12 = 7.2MB

This is pretty insignificant for a modern GPU.

* Double buffering is used as the GPU locks array buffers until it's done with it.

Scrolling

The prototype currently scrolls extremely smoothly as at most a viewport worth of data is filled but often no viewport data will change. Then we just need to update the scroll offset so the shadow knows which cells to render.

Input

So far, the above is highly optimized for readonly scrolling. For input/file changes there are a few cases we need to target. We essentially want to get these updates to take as little CPU time as possible, even if that means leaving stale and no-longer referenced data in the fixed buffers.

Adding new lines or deleting lines

This could be supported by uploading a map whose job is to map line numbers with the index in the fixed buffer:

That way we only need to update indexes, not the whole line data.

Inserting characters

Simple O(n) solution is to just update the entire line. We could do tricks to make this faster but it might not be worth the effort if line length is fixed.

Fixed buffers and long lines

My plan for how the characters will be send to the GPU is to have 1 or more fixed width buffers (eg. 80, 200?) with maps that point to indexes dynamically as described in the input section and then another more dynamic buffer which supports lines of arbitrary length. This dynamic buffer will be a little less optimized as it's the edge case when coding. The fixed buffers could also be dynamically allocated based on the file to save some memory.

Other things we could do

• Sub-pixel glyphs for smoother flow - eg. render characters at 4x the width and support offsetting the character every 0.25px.
• Proportional font support isn't in xterm.js but it's possible without too much effort, we will need to support this anyway if we want to render widths just like the DOM renderer. The main thing this requires is some way of getting the width of the glyphs and the offset of each character in a line. Again this is an optimization problem of getting and updating this width/offst data as fast as possible.
• I believe SPAA is possible to do on the GPU using grayscale textures.
• Custom glyphs are supported in the terminal which allows pixel perfect box drawing characters for example like ┌───┘. Whether this looks good in monaco is up to the font settings. Letter spacing and line height will always mess with these
• Texture atlas glyphs could be first drawn to a very small page and then packed more efficiently into a longer-term page in an idle callback or worker.
• Texture atlas pages could be cached to disk
• Canvas sharing - To optimize notebooks in particular we could have a shared canvas for all editors and tell the renderer that it owns a certain bounding box

Test results

These were done on terminalInstance.ts. Particularly slow frames of the test are showed.

The tyriar/gpu_exploration tests disabled all dom rendering (lines, sticky scroll, etc.) to get an idea of how fast things could be without needed to perform layouts on each frame. It's safe to assume that rendering other components would be less than or equal to the time of the most complex component (minimap is similar, but could potentially share data as well).

Scroll to top command

M2 Pro Macbook main

M2 Pro Macbook tyriar/gpu_exploration (all dom rendering disabled)

Windows gaming PC main

Windows gaming PC tyriar/gpu_exploration (all dom rendering disabled)

Scrolling with small text on a huge viewport

fontSize 6, zoomLevel -4

M2 Pro Macbook main

M2 Pro Macbook tyriar/gpu_exploration (all dom rendering disabled)

Windows gaming PC main

Windows gaming PC tyriar/gpu_exploration (all dom rendering disabled)

Very long line

Long lines aren't supported in the gpu renderer currently

Shaders run in parallel to microtasks and layout

The sample below from the Windows scroll to top test above demonstrates how the shaders execute in parallel with layout, as opposed to all after layout.

Before:

After:

---

Harfbuzz shaping engine is used by lots of programs including Chromium to determine various things about text rendering. This might be needed for good RTL/ligature/grapheme rendering.

[Tyriar]

Update on my end for last week. WIP branch #225413

General

• Rendering is fixed up when switching editors and resizing canvas
• Correct background color is drawn (instead of black)
  
• Bunch of general clean up and refactors. In particular improving of variable/constant names and simplifying of the main webgpu code
• Set up a GPULifecycle namespace with helpers that return IDisposables
• Sorted out some high level lifecycle/leak issues
• The GlyphRasterizer is now owned by GpuViewLayerRenderer. The idea here is that the texture atlas is shared across all editors, but different editors could have different font sizes so it's owned by the editor so multiple font sizes can be rendered (WIP, sizes aren't tracked in atlas keys yet).

Rasterization

• Bold and italic text is now rendered
  
• Less garbage collection by reusing hot objects

Texture atlas

• Multiple texture atlas pages are now addressable. There is no overflow logic yet, but glyphs are distributed whether they are alphabet chars in order to test multiple pages
• Glyphs are uniquely identified and stored by their metadata instead of just their foreground color
  

  vscode/src/vs/editor/browser/view/gpu/atlas/textureAtlasPage.ts

  Lines 81 to 84 in 3cfe905

  	// Ignore metadata that doesn't affect the glyph
  	metadata ^= (MetadataConsts.LANGUAGEID_MASK | MetadataConsts.TOKEN_TYPE_MASK | MetadataConsts.BALANCED_BRACKETS_MASK);
  	return this._glyphMap.get(chars, metadata) ?? this._createGlyph(rasterizer, chars, metadata);

• Texture atlas debug commands
  
  • Saving texture atlas pages:
    
  • Logging texture atlas page stats:
    
• Some basic unit tests for atlas and allocators
• Only the used portion of the atlas texture is uploaded, speeding up render time when there are new glyphs significantly, especially on initial render (~15ms -> ~2ms)

Explorations

• Explored approach for rendering of view parts, starting with the ruler.
  • I first tried to do multiple passes with a separate shader but it's more complicated than I initially thought and requires juggling some textures. Additionally, order of render passes and having them all run every time would be required for this.
  • I think the right approach here at least for simple view parts is to allow parts to register shapes into some render pass/command encoder object. This would make the ruler component even simpler than the DOM-based one as they would basically just register some fixed rectangles/lines and then refresh it when the setting changes.
• Explored the "scratch page" idea for the texture atlas.
  • This needed more logic in the shader than expected. Uploading only relevant parts of the page texture was a big win that makes this no longer needed.

[faheemstepsharp]

Hope this become default soon.

[Tyriar]

@faheemstepsharp I suspect it's going to be a long road to be the default (6 months, 1 year+?). We did eventually switch the terminal to default to GPU rendering, it'll be really bad if we ship an editor that breaks text rendering though.

[Tyriar]

Update for @hediet and myself for last week. WIP PR #225413

Architecture

We came up with a better approach for where to stick the implementation. GPU parts are now regular "view parts" instead of being more tightly tied to the view.

vscode/src/vs/editor/browser/view.ts

Line 161 in bd21f3c

this._viewLinesGpu = this._instantiationService.createInstance(ViewLinesGpu, this._context, this._viewGpuContext);

A new ViewGpuContext contains all objects needed for managing rendering to the frame (canvas element, GpuContext, command encoder, etc.). This is owned by View and will be injected to every GPU-related view part, similar to ViewContext.

vscode/src/vs/editor/browser/view.ts

Lines 146 to 148 in bd21f3c

	if (this._context.configuration.options.get(EditorOption.experimentalGpuAcceleration) === 'on') {
	this._viewGpuContext = new ViewGpuContext();
	}

❔ The term "context" is becoming a little overloaded (ViewContext, ViewGpuContext, GPUContext). Maybe there's a better name for ViewGpuContext?

Drawing shapes

Built out the ObjectCollectionBuffer data structure which allows creating type-safe objects that get encoded into a Float32Array which will be used to draw shapes via the ViewGpuContext interface. This will allows view parts to easily add, remove and change sections of the Float32Array in a fairly performant way without needing to deal with the actual buffer. Done right I think this should make the implementation of simple view parts like rulers to be even simpler than the DOM-based counterpart.

vscode/src/vs/editor/test/browser/view/gpu/objectCollectionBuffer.test.ts

Lines 42 to 54 in bd21f3c

	const buffer = store.add(createObjectCollectionBuffer([
	{ name: 'a' },
	{ name: 'b' },
	] as const, 5));
	store.add(buffer.createEntry({ a: 1, b: 2 }));
	const entry1 = buffer.createEntry({ a: 3, b: 4 });
	store.add(buffer.createEntry({ a: 5, b: 6 }));
	const entry2 = buffer.createEntry({ a: 7, b: 8 });
	store.add(buffer.createEntry({ a: 9, b: 10 }));
	entry1.dispose();
	entry2.dispose();
	// Data from disposed entries is stale and doesn't need to be validated
	assertUsedData(buffer, [1, 2, 5, 6, 9, 10]);

This object isn't hooked up yet, just the data structure and tests are mostly done.

General

• Lots of cleaning up of interfaces, adding jsdoc, etc.
  • Removed chars/tokenFg from the allocator interface, to makes it more clear that all an allocator's job is to take a rasterized glyph, put it into an atlas and track the usage.
• Fixed metadata key to properly remove metadata that doesn't affect the glyph's rendering.
• Fixed "null cells" rendering random characters to the middle of the renderer. This was happening because zeroed out sections of the buffer were all pointing at the first glyph of the first atlas page.
• The canvas is sized to fit .overflow-guard. This is probably the final size and position of the canvas.
  
• Added viewport offset which now renders the characters in approximately the right position (when dpr=1 at least). The top and the bottom lines in this picture show the gpu rendering overlaid on top of the DOM rendering.
  
• Added #regions and organized the webgpu init code a little better.
  
• Added a hidden setting to enable the GPU renderer so we can merge the code with minimal impact on default rendering.

Texture atlas

• Basic page overflow logic is done; when a page is filled it will start adding glyphs to a second page. Only 2 pages are currently supported in the shader though.
• Handle edge cases around glyphs too large for slab or page.
• Reduced search time for glyph's page to O(1) 740ba1c
• More tests!

Debugging

• New draw glyph command that will draw a single glyph.
  
• Consolidated all gpu-related debug command into a single Developer: Debug Editor GPU Renderer command that brings up a picker. This will let us create many debug-related commands without spamming the regular command palette.
  

[vincentdchan]

This may be a silly question but how do you draw glyphs on WebGPU? Are you drawing the glyph map with canvas or render the font manually?

[Tyriar]

@vincentdchan fonts are rasterized to a texture atlas using a 2d canvas context (mostly on CPU), then the texture atlas pages are uploaded and drawn by the shader where each cell is a texture mapped to 2 triangles. So we're leveraging the browser's font rendering stack and can avoid getting into that.

[aurexav]

Some feedback from VSCode Insiders:

1. Scrolling through a long comment causes noticeable lag, whereas scrolling through code remains smooth. I’m not sure why this happens.
2. Weird bracket in error lens line.

[Tyriar]

@aurexav thanks, I created #238849 for 1. For 2, could you create an issue with the code that caused that and what you did if anything to make it show? I'm not too familiar with Rust.

[Tyriar]

Update:

• New viewport rendering strategy which means we can not render files with many lines, as well as lines up to 2000 characters before falling back Editor GPU: Add viewport render strategy #238487
  
• Added sub-pixel offset rendering of glyphs Sub-pixel offset rendering #238699, this makes rendering of characters a lot closer to how the DOM and in general flow a lot better. It does this by rasterizing glyphs at 0.0, 0.1, ..., 0.9 offsets. Notice how these a's no longer look exactly the same:
  
• The horizontal scroll bar should now be sized by the GPU lines Editor GPU: Pass content width through to scroll bar #238679, thought there are still some remaining issues here Editor GPU: Horizontal scroll bar doesn't work yet #233790

[aurexav]

@aurexav thanks, I created #238849 for 1. For 2, could you create an issue with the code that caused that and what you did if anything to make it show? I'm not too familiar with Rust.

I’m currently unable to reproduce the issue.

I’m encountering a critical bug. When I attempt to open certain files within a project, the editor crashes.

[dougg0k]

I tested "editor.experimentalGpuAcceleration": "on" on the 1.97, but all text disappeared, they appeared if I selected them individually.

It was due to webgpu being disabled. So, by using / adding these two flags to ~/.config/code-flags.conf, fixed the problem.

--enable-unsafe-webgpu
--enable-features=Vulkan

ps: any other feature you try to enable, must be with a comma in the same line, a new enable-features line would overwrite the previous.

Even though it gives some warns, it works. Confirmed with code --status.

I am using a Arch Linux based distro.

---

Edit1:

One issue I found after, once I open a file and start selecting text randomly fast, it lags, not vscode, but the selection. After a bit, it stabilizes, till I open another file and do the same. And same thing with the minimap if you have it enabled.

---

Edit2:

After, I added these other flags to check, I dont which, but at least one helped with the issue.

--enable-gpu-rasterization
--enable-zero-copy

It can happen in bigger files, but stops much faster, then it's all fast.

---

Edit3:

I did more testing, I also tested other flags not mentioned here. Besides the slow effects mentioned already, when it's happening if you select with the keyboard word per word or hover with mouse the files tree, all are slower, till all stops being slow all at once. It's one thing for everything.

Sometimes, when you open the editor, the slow effect are there. Sometimes after opening, all are very fast, but after a while bit the slow appears.

If you are able to fix the issue, overall it would be an improvement.

---

Edit4:

I will share all the flags I used in attempt to fix the issue and more.

That slow effect are still random, be on big or small file. Sometimes only on the minimap, sometimes on other parts too. But again, once it stop in one part, it seem to stop in others too. Though it can come back after.

--enable-features=Vulkan,CanvasOopRasterization,UseSkiaRenderer,VulkanFromANGLE,DefaultANGLEVulkan,DirectRenderingDisplayCompositor,VizDisplayCompositor,HighEfficiencyModeAvailable,EarlyHintsPreloadForNavigation
# WaylandWindowDecorations cause slowdown - SkiaGraphite,RawDraw break webgpu
--disable-features=WaylandWindowDecorations,SkiaGraphite,RawDraw
--force-device-scale-factor=1
--force-color-profile=srgb
--enable-unsafe-webgpu
--enable-gpu-rasterization
--enable-oop-rasterization
--enable-zero-copy
--scheduler-configuration
--canvas-oop-rasterization
--enable-drdc
--enable-native-gpu-memory-buffers
--disable-wayland-ime
--enable-threaded-texture-mailboxes
--enable-threaded-compositing
--enable-threaded-animation
--enable-threaded-scrolling
--wm-window-animations-disabled
--animation-duration-scale=0.0
--cc-scroll-animation-duration-in-seconds=0.0
--disable-login-animations
--disable-modal-animations
--ui-enable-zero-copy
--enable-raw-draw
--enable-accelerated-2d-canvas
--use-skia-renderer
--enable-use-gpu-scheduler-dfs
--enable-accelerated-compositing
--enable-gpu-async-worker-context
--gtk-version=4
--new-canvas-2d-api
--disable-gpu-vsync
--num-raster-threads=8
--use-vulkan=native
--main-thread-compositing-priority
--enable-gpu-memory-buffer-video-frames
--ui-enable-shared-image-cache-for-gpu
--use-client-gmb-interface
--enable-waitable-swap-chain=1
--max-gum-fps=500
--disable-smooth-scrolling
--enable-viz-display-compositor
--disable-image-animation-resync
--allow-pre-commit-input
--back-forward-cache=force-caching-all-pages
--high-efficiency-mode-available
--unthrottled-nested-timeout
--force_high_performance_gpu
--disable-renderer-backgrounding
--enable-gpu-early-init
--disable-webgl
--disable-input-event-activation-protection
# --ignore-gpu-blocklist
# --disable-gpu-driver-workarounds
# --disable-gpu-driver-bug-workarounds
# --max-active-webgl-contexts=100 # GL usage only
# --use-gl=egl # Breaks webgpu
# --in-process-gpu # Breaks webgpu
# --enable-skia-graphite # Breaks webgpu
# --disable-frame-rate-limit # Cause slow down

If you try to use these flags, know that not all commands will work with the same end result in all systems, you must test and find any potential issue, and remove the cause.

Ideally there should be a way to easily test them or someone do them, individually comparing potential ms differences.

---

Update: I added more flags in the list, I dont see the slow happening anymore, I wonder if the skia_renderer fixed it, or at least minimized the effect even more.

direct_rendering_display_compositor that seem to improve gpu threading, at least on linux, it's not activating, regardless of feature enabling or flag setting.

---

I might edit this comment again if any issue appears, but so far, everything have been responsive and with no issues.

Those additional flags really made a difference, at least here. I dont know which ones are working and giving improvements, and which are not, but as long as they dont worsen things, it's not a problem.

---

Flags list are now complete.

---

I noticed slows when writing and selecting text, not as the slowdowns from before, but still not being snappy / responsive. Same slow behavior when selecting through the keyboard, though with mouse are very fast.

Also randomly some text that I started writing had the behavior of being invisible, just like without setting those flags. When activating the gpuacceleration. But this time was only for what was being written.

I found that --disable-webgl seem to help minimize but not completely remove the slow writing effect while webgpu are active.

---

To not use the WebGPU and stop vscode from using GPU completely. Comment or remove all flags and use these two.

--disable-gpu
--disable-software-rasterizer

---

On 1.98, RawDraw cause the UI to break. So, it's now disabled in the list above.

[pprchal]

I apologize for the possible offtopic.
But the previous article addresses the problem mentioned here:
#7770

I have a home PC-celeron on a 4K Benq 3840x2560.
If I run vscode by default, then it is unusable.

Those 2 options

--enable-unsafe-webgpu
--enable-features=Vulkan

are OK, and the code can be used normally (without --disable-gpu)

[mubaidr]

I tested "editor.experimentalGpuAcceleration": "on" on the 1.97, but all text disappeared, they appeared if I selected them individually.

It was due to webgpu being disabled. So, by using / adding these two flags to ~/.config/code-flags.conf, fixed the problem.

--enable-unsafe-webgpu
--enable-features=Vulkan

ps: any other feature you try to enable, must be with a comma in the same line, a new enable-features line would overwrite the previous.

Even though it gives some warns, it works. Confirmed with code --status.

I am on Arch Linux based distro.

Edit1:

One issue I found after, once I open a file and start selecting text randomly fast, it lags, not vscode, but the selection. After a bit, it stabilizes, till I open another file and do the same. And same thing with the minimap if you have it enabled.

Edit2:

After, I added these other flags to check, I dont which, but at least one helped with the issue.

--enable-gpu-rasterization
--enable-zero-copy

It can happen in bigger files, but stops much faster, then it's all fast.

Edit3:

I did more testing, I also tested other flags not mentioned here. Besides the slow effects mentioned already, when it's happening if you select with the keyboard word per word or hover with mouse the files tree, all are slower, till all stops being slow all at once. It's one thing for everything.

Sometimes, when you open the editor, the slow effect are there. Sometimes after opening, all are very fast, but after a while bit the slow appears.

If you are able to fix the issue, overall it would be an improvement.

Edit4:

I will share all the flags I used in attempt to bring an improvement to the issue.

That slow effect are still random, be on big or small file. Sometimes only on the minimap, sometimes on other parts too. But again, once it stop in one part, it seem to stop in others too. Though it can come back after.

--enable-features=Vulkan,CanvasOopRasterization,RawDraw,UseSkiaRenderer
--enable-unsafe-webgpu
--enable-gpu-rasterization
--enable-zero-copy
--high-efficiency-mode-available
--scheduler-configuration
--canvas-oop-rasterization
--enable-drdc
--enable-native-gpu-memory-buffers
--ignore-gpu-blocklist
--disable-wayland-ime
--enable-threaded-texture-mailboxes
--use-vulkan
--enable-threaded-compositing
--enable-threaded-animation
--enable-threaded-scrolling
--wm-window-animations-disabled
--animation-duration-scale=0
--disable-login-animations
--disable-modal-animations
--ui-enable-zero-copy
--enable-raw-draw
--enable-accelerated-2d-canvas
--use-skia-renderer
--disable-gpu-driver-bug-workarounds

Update: I added more flags in the list, I dont see the slow happening anymore, I wonder if the skia renderer fixed it, or at least minimized the effect even more.

Confirmed. Enabling webgpu using flag along with Vulkan solves the empy screen issue.

[dougg0k]

There is an issue that seem related to GPU accel, that can happen in all places in vscode, it would be nice to have it fixed when webgpu feature are stabilized and released.

That is, random text overlap, I've seen in the editor, and i've seen others reporting happening in the file tree and terminal.

#173335
#174721

@Tyriar

[larsgeb]

Just a question from someone who is just getting into the internals of VSCode, and thinks his issue is related.

I have a Jupyter Widget that uses 3D rendering with PyThreeJS (and then of course Three.JS). This is an interface to WebGL.

On my current VSCode 1.97.2 (Apple Silicon, this commit) and since about summer 2024 (though I wouldn't able to pinpoint a version number), this widget stopped working when executing notebooks directly in VSCode (though they work when launching a kernel from VSCode and going to Safari/FireFox/Chrome). I had a bit of a dive into it recently and found that when the widget gets created, it fails while trying to create a WebGL context.

I checked out a few releases (I think as early as October 2024) of VSCode over the past few months, and MacOS flavours (Intel, Universal), but no joy. I also don't want to regress too far just for this widget, as I'm a happy user of new features.

VSCode's GPU Info shows specifically the following:

*   WebGPU has been disabled via blocklist or the command line.
    Disabled Features: webgpu
*   WebGL2 has been disabled via blocklist or the command line.
    Disabled Features: webgl2
*   Accelerated rasterization has been disabled, either via blocklist, about:flags or the command line.
    Disabled Features: rasterization
*   Accelerated video encode has been disabled, either via blocklist, about:flags or the command line.
    Disabled Features: video_encode
*   Accelerated video decode has been disabled, either via blocklist, about:flags or the command line.
    Disabled Features: video_decode
*   WebGL has been disabled via blocklist or the command line.
    Disabled Features: webgl
*   Gpu compositing has been disabled, either via blocklist, about:flags or the command line. The browser will fall back to software compositing and hardware acceleration will be unavailable.
    Disabled Features: gpu_compositing
*   Accelerated 2D canvas is unavailable: either disabled via blocklist or the command line.
    Disabled Features: 2d_canvas

Additionally, I've tried most of the Electron command line flags to try and enable this, but also no succes. I think I am not sure what to expect on enabling WebGL at all for version 1.97.2. It seems that on the Insiders Release (1.98.0-insider), WebGL is at least supported, but the widget still crashes.

I guess this is expected then right? I'm not knowledgeable on the difference between terminal versus editor GPU rendering, so I am also not sure where the "Interactive - ..." window for Jupytext files or the Jupyter Notebook editor directly falls, but in general, on my MacOS Apple Silicon system, this is what VSCode Developer Tools tell me after trying to get a WebGL context.

Just verifying this is directly related to this issue, otherwise happy to document and write up in a new issue.

[Tyriar]

@dougg0k random overlaps in the tree is not related to editor GPU work, I saw another issue related to this recently. The terminal one was a regression #239917.

[Tyriar]

@larsgeb Chromium maintains a blacklist of hardware to disable webgl on and you might just have a setup that hits that. You could create a new issue and ping @deepak1556 on it since it's not directly related to this one.

[dougg0k]

From my testings on the gpu acceleration. Unless you can make webgpu as responsive as with GPU disabled, I dont really see the point in using it.

I included the information in my first comment here.

@Tyriar

---

In fact, could you tell the actual benefits of using gpu acceleration in a text editor other than taking load off the cpu?

[Tyriar]

From my testings on the gpu acceleration. Unless you can make webgpu as responsive as with GPU disabled, I dont really see the point in using it.

@dougg0k of course, this is a WIP and the focus is on feature parity not performance currently.

In fact, could you tell the actual benefits of using gpu acceleration in a text editor other than taking load off the cpu?

The goal is to minimize input latency and scroll latency by reduce the time it takes to render a frame as much as possible. This is mostly done by removing the need for browser-based layouts and dealing with DOM nodes which are expensive on the CPU side.

[axetroy]

When the layout changes, re-rendering can cause page content to flicker

2025-03-07.00.03.37.mov

[Tyriar]

Update on this: I was busy with terminal intellisense last month (which is now in preview!) and will be mostly improving terminal in copilot reliability and internal security stuff this month.

[Tyriar]

Update: I'll be out for a while soon, I don't expect work to pick back up on this until July at the earliest

[yennhi2906]