How Does Linux Display Subsystem Work?

The basic building block of displays is the framebuffer. A framebuffer is just a block of memory that holds a frame of image data that gets sent to the monitor and displayed. When you first start Linux, you will see a black console which consists of text data drawn directly to the output framebuffer. Linux systems also make use of two display frameworks that works with the output framebuffer.

X11 is the first and older framework that create virtual framebuffers for each application (client) and then composes the virtual framebuffers together on top of final framebuffer. It then switches the current output framebuffer to the newly rendered framebuffer and sends the data to the monitor to be displayed. The problem this method is two folds. First, the overhead associated with copying the framebuffers around which can be quite taxing especially on systems with limited memory bandwidth like SBCs. Second, to work around the performance problems, optimizations are made for performance over security. The benefit of this method is that it works very well over a network since the components are broken down to individual applications so it is possible to launch X11 applications on remote display servers.

Wayland is the newer framework that aims to exploit modern hardware capabilities to mitigate the issues mentioned with X11. Unlike X11 which is a display server, Wayland is a display protocol and multiple implementations of the compositor (like a display server). Wayland is not designed as a network display server and takes full advantage of local hardware accelerators such as video decoding to framebuffers with zero-copying.