📚 Considering starting a new oss project; some questions on how multiple monitors/displays work in Linux

Hello fellow penguins,

I've been toying around with the idea of starting an open source project.

The idea:

A setup to allow an iPad to be used as an additional monitor in Linux (I guess only on Wayland, since that's the future?).

I know it's already possible with RDP workarounds, but I was thinking about something more sophisticated and native.

Particularly, imo it would be extremely awesome if (in a future phase of the project of course) we can have Apple Pencil support. So you'd be able to draw directly in Krita/Gimp/etc right on the iPad with the apple pencil.

My background:

The problem is, I have no idea where to even start.

I'm a full stack dev with 20 years of experience, so I can write code, but my low-level programming experience is very limited in general, and what there was of it was limited to DOS programming and Windows driver debugging.

Lately I've been getting into Rust, and now that we're getting Rust support in the next version of the kernel, I'd love to use it for this.

What I'm envisioning:

(This is limited by my current understanding, so it may be completely off)

• There would be a Linux kernel-mode driver for, I am guessing, a virtual monitor (this is where my knowledge lacks the most).
  • For Apple Pencil support (in a future version), I am guessing we would also need an input driver. Maybe we can fork the wacom drivers and re-use some of their code for this?
• The kernel driver(s) would communicate via unix sockets(?) with a daemon/server app running in usermode.
  • Perhaps it can something like dbus so that they can be decoupled, though I don't think it is feasible or possible to make dbus calls from a kernel module. Maybe we would have another intermediate low-level layer, that would provide an interface between the kmod and the userland server app.
  • The server/daemon app would handle communications with our iOS client app, to facilitate screen output (and later input).
    • We can perhaps use something like RDP to get a quick working prototype out the door, and then work on our own protocol.
    • The protocol should be medium-agnostic, since then we could support wifi as well as a direct usb-c cable, and maybe bluetooth?
• There would need to be an iPad app built in Swift which would interface with our server app running on the Linux machine.

Questions:

• Where would I even start looking at how to implement a virtual display?
  • I'll be honest, the idea of "monitors" have always been a mystery to me even in the Windows world. I know there are video card drivers, but when it comes to displays I think there is just one universal monitor plug & play driver? I know there are monitor-specific drivers out there, but IME they mostly tend to deal with additional features of a specific monitor (like volume control, color settings, etc) but all displays have always worked out of the box for me on every OS so long as the video card driver was loaded. So this is my biggest confusion... Is there such thing as a monitor/display driver? Where in the kernel tree would I even look for it?
• How does Linux work with multiple displays? I am assuming that the kernel handles and abstracts the "presence" and interaction with them, but it is the display system (X or Wayland) that actually manages what to output to them (well, it would be the window system / desktop that would determine WHAT to output, but it is the underlying display server that exposes the APIs to work with them. Is this assumption correct?
  • Where in the Wayland codebase would be good starting points to look at how this whole setup works?
• And of course: Is anyone interested in working on this with me? :)
  • Particularly, if you can do the kernel/driver programming portion, then I wouldn't need to spend my time on that and can work on the client & server portion -- which is something I do know how to do very well and am good at. I would even be open to discussing some symbolic compensation (Nothing substantial obviously, since this would be a hobby/open source project, but your coffee & beer supply would be on me!)