GUACAMOLE-288: Add support for multi-monitor connections on RDP.

[corentin-soriano]

Anyone who wishes can test and feedback will be appreciated!

Requires using this branch on guacamole-server: apache/guacamole-server#560

• Display each monitor in separate window.
• Secondary windows automatically resize to the dimensions of the main window.
• Display user consent button to switch all window on fullscreen when the main window fullscreen is enabled.
• All monitors are on the size of the main window and ordered from left to right (left monitor is the primary).
• Handle CTRL+ALT+SHIFT in secondary monitors (open the menu on the main window).
• Close secondary monitor on client disconnect.
• Admin can limit the usage of multi-monitors on each connection.
• Drag and drop between windows without having to release the click and resume on the other side.

Admin settings:

Add new monitor:

Multiple monitors opened:

[stephzero1]

wow this is huge! i will test asap

[neandrake]

Exciting. Could you write up the technical implementation here? Here's what I gather from looking at the changes here and in the guacamole-server pull request.

• It looks like all the drawing instructions will now set an optional x-offset for the base/default layer. All other layers are relative to the base layer which is why they don't need the x-offset (correct?).
• A new optional parameter is added to the size instruction for indicating number of monitors.
• A new optional parameter for maximum number of secondary monitors is added to the connection parameters.

That leads me to understand,

1. There is a constraint that monitors must be laid out horizontally and be the same dimension.
2. Each window/tab/connection to the server is receiving graphical updates for all monitors but only displaying the subset for its selected monitor.

Could you clarify if that is correct? If so, how is each tab specifying which monitor/offset to use?

[corentin-soriano]

Exciting. Could you write up the technical implementation here? Here's what I gather from looking at the changes here and in the guacamole-server pull request.

• It looks like all the drawing instructions will now set an optional x-offset for the base/default layer. All other layers are relative to the base layer which is why they don't need the x-offset (correct?).

Only the main layer is affected by the offset because the others (eg: mouse) are relative to the browser window instead of the remote screen.

• A new optional parameter is added to the size instruction for indicating number of monitors.

With FreeRDP, the number of monitors is set via the display update channel (the same as for the dimensions). The setting is optional for backward compatibility.

• A new optional parameter for maximum number of secondary monitors is added to the connection parameters.

You can define it in your user-mapping.xml or in guacamole settings with a jdbc extension:

<param name="secondary-monitors">3</param>

In the example above I can open 3 additional monitors and then the button is grayed out (guacd would ignore the instruction):

That leads me to understand,

1. There is a constraint that monitors must be laid out horizontally and be the same dimension.

For now, all monitors have the same dimensions but there is client-side scaling if one of the windows needs to have different dimensions:

2. Each window/tab/connection to the server is receiving graphical updates for all monitors but only displaying the subset for its selected monitor.

Exactly, each window displays its visible part according to its x-offset. Each time a monitor is added/removed, an instruction is sent by sendMonitorsInfos() to all the windows to know their new position (allow to close a middle window).
The x-offset corresponds to the position * width of the monitor.

Could you clarify if that is correct? If so, how is each tab specifying which monitor/offset to use?

Secondary windows knows their position thanks to an ID defined in the route:

In the screenshot above:

• Main window is at the position 0 => x-offset = 0*width = 0 px
• monitor with ID 3 is at the position 1 => x-offset = 1*width.

Does this answer your questions?

[mike-jumper]

@corentin-soriano Rather than build on the size instruction, what about leveraging layer parameters and the established set instruction (similar to the multi-touch support)?

For example, we could do something like:

/* Hint that some_layer is best represented as a separate monitor/display */
guac_protocol_send_set(socket, some_layer,
    GUAC_PROTOCOL_LAYER_PARAMETER_PRESENTATION,
    GUAC_PROTOCOL_LAYER_PARAMETER_PRESENTATION_DETACHED);

/* Hint that the default layer will only contain other layers and need not have 
 * a graphical surface */
guac_protocol_send_set(socket, default_layer,
    GUAC_PROTOCOL_LAYER_PARAMETER_PRESENTATION,
    GUAC_PROTOCOL_LAYER_PARAMETER_PRESENTATION_VOID);

to advise compatible clients that a particular layer should be ideally rendered as a separate display in its own right, with its position within any parent layers to be interpreted as hinting for relative location.

That would make the same semantics available in a backward-compatible manner, while also allowing the concept of separating out parts of the display to be more broadly applicable (such as for popping out RemoteApp windows).

[gerdesj]

I'd like to test your code but I am not a github expert. I have checked out both guacamole-client and guacamole-server locally.

How do I put them in the correct state? I suspect --branch is involved.

I'd be grateful if you could note how to get the starting line set up and then I'll run with it.

[necouchman]

I'd like to test your code but I am not a github expert. I have checked out both guacamole-client and guacamole-server locally.

How do I put them in the correct state? I suspect --branch is involved.

@gerdesj

Should be something like:
git clone https://github.com/apache/guacamole-client
cd guacamole-client
git fetch origin pull/1061/head:test/GUACAMOLE-288
git checkout test/GUACAMOLE-288
mvn clean package

(and similarly for guacamole-server, just substituting in the correct pull request number for that repo, then autoreconf, configure, make, etc.)

[gerdesj]

Wow! It works perfectly. I'd say it is better than stock mstsc because you can add and remove screens at your whim.

So far I have tested with Firefox on Kubuntu 24.04, connecting to a Windows 2022 RDS box. Guac + Tomcat are running on Ubuntu 22.04. I only have two identical screens so haven't looked into what happens with varying sizes.

I'll move this to our live box soon and get some more testing done with real users.