Wayland Compositor Thin Client

Continuing the discussion from How-to: Arch Linux on a Librem:

I would think something like this would describe what the goal is. LTSP (and usb passthrough) is a separate issue.

As for how to do it, something akin to how X handles it is not what you want. X’s approach is to run a full X server on each thin client, which requires the thin client to do the compositing, and the brainpower it needs. Yes, the 2d vesa driver is sufficient, so you don’t need a fancy card (any vesa compatible output should work), but it’s still badly inefficient. If you wanted to run a local Wayland compositor, but add in some remote programs too, and have them composited together with the local programs, X’s approach is the only way to go, but that isn’t what a thin client does, so we want a different approach.

Instead, we want to write an entire network-aware Wayland compositor, which will run on the server side. It will allocate buffers for the clients from a local DRM node, on a local graphics card. It will use the usual wayland calls (available either through libwayland, or wlroots (still considering the advantages of each)) to pass those buffers to its clients for drawing. It will then composite the output, but rather than rendering it locally, it will serialize the buffers and transmit them over the network. The initial implementation should probably just send the whole raw frame each time, but adding blitting and damage tracking is probably a good optimization, H264 encoding or similar may also be a good idea. On the thin client side, the frame gets rendered via mesa+DRM directly in a tty. The code for this may be able to be lifted from wlroots, weston, or similar. Meanwhile, the client sends libinput events across the network to the server, which processes them appropriately.

Now, I don’t really want to implement all the things for which a Wayland compositor is responsible, and people wanting to use a thin client probably want to have their preferred compositor available (gnome, kde, wayfire, sway, et cetera). Fortunately, there is a solution. The network-aware compositor only needs to be able to have a single, full-screen client, and only needs to be able to pass events through to that client. We then run the compositor of choice using the Wayland backend, which all important Wayland compositors support. Oh, this also handles the alternate use case I mentioned (a remote program mixed with local programs), since you could run the network-aware server using a Wayland backend for the remote client, but that would take some automation to get window sizing right.

OK, what you say sounds right. I am not a programmer, I am a mathematician using the ltsp technology and often the “network transparency” of X from server to pc, either directly or with xpra. I use ltsp for 16 years now and have configured our labs to work helping the local administrators (mainly Windows server admins) that had no experience in Linux.

But from what you say we get down to my initial complain. Wayland needs (for such applications) some kind of “network transparency” that currently does not have.

Let me mention another shortcoming I recalled, other that HD video. With X and ltsp programs such as riot or skype do not work with video. riot works with sound only. Note that both programs are tested running on the thin clients because they have to have access to the local camera. I do not know if your suggested approach could solve this; if wayland on server could have access to the clients local camera or cooperate with the clients programs. This ought to work somehow because the cameras for desktops typically give low resolution videos. But judging from this I would say that for such a setup the best is to have client running the absolute minimal set of applications. That is, minimize mixing of apps. Most apps must run on the server.

By the way I have a few videos for my students and colleagues so that they can configure their desktop on the clients. The 1st video shows the possibilities. It is here:

and you will see how nice is the situation with X right now. What you see in this video is just the desktop on the thin client.

Edit: the video is created on a pc because I have to use the simplescreenrecorder and the thin client is not so fast. But the desktop on the client is the same with what you see. No delays, everything works as smoothly as on the pc.