Skip to content

feat(session): expose FrameEnd signal for protocol-level frame synchronization#1189

Closed
sfwwslm (sfwwslm) wants to merge 1 commit intoDevolutions:masterfrom
sfwwslm:feature/frame-end
Closed

feat(session): expose FrameEnd signal for protocol-level frame synchronization#1189
sfwwslm (sfwwslm) wants to merge 1 commit intoDevolutions:masterfrom
sfwwslm:feature/frame-end

Conversation

@sfwwslm
Copy link
Copy Markdown

Summary

This PR exposes the FrameEnd marker from the RDP Graphics Pipeline Extension ([MS-RDPEGFX]) to the ActiveStageOutput. This allows downstream consumers to synchronize their rendering loop with the server's logical frames, effectively eliminating screen tearing and improving frame pacing.

Background

In the [MS-RDPEGFX] protocol, the server sends RDPGFX_FRAME_MARKER_PDU with a frameAction of RDPGFX_FRAME_END (0x0001) to signal the completion of a logical frame. While ironrdp-pdu already supports parsing these markers, the ActiveStage in ironrdp-session was previously consuming them internally (for Acknowledge packets) without notifying the application layer.

Without this signal, RDP clients often have to rely on arbitrary timers (e.g., 16ms) to flush pending graphics updates. This frequently leads to "half-rendered" updates appearing on screen when a server-side frame is split across multiple PDUs, causing visible tearing and "rolling" artifacts during fast window dragging or scrolling.

Changes

  • ironrdp-session:
    • fast_path.rs: Added FrameEnd variant to the internal UpdateKind enum.
    • fast_path.rs: Updated Processor::process_surface_commands to detect FrameMarker with FrameAction::End and propagate it as an UpdateKind.
    • active_stage.rs: Added FrameEnd variant to the public ActiveStageOutput enum.
    • active_stage.rs: Updated ActiveStage::process to transform the internal UpdateKind::FrameEnd into an external ActiveStageOutput::FrameEnd.

Impact

By consuming the FrameEnd signal, applications can now:

  1. Eliminate Tearing: Trigger a UI flush/present only when a complete logical frame is received.
  2. Implement Frame Coalescing: Effectively aggregate multiple fragmented PDUs into a single atomic update.
  3. Better Frame Pacing: Align client-side rendering (e.g., requestAnimationFrame in web clients) with the server's actual production rate.

Empirical testing in a Tauri-based WebView RDP client showed that using this marker allowed for a stable 60+ FPS experience with significantly improved visual coherence during intense UI movements.

Comment on lines -311 to 321
/// See [\[MS-RDPBCGR\] 2.2.15.1].
/// See [[MS-RDPBCGR] 2.2.15.1].
///
/// [\[MS-RDPBCGR\] 2.2.15.1]: https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-rdpbcgr/de783158-8b01-4818-8fb0-62523a5b3490
/// [[MS-RDPBCGR] 2.2.15.1]: https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-rdpbcgr/de783158-8b01-4818-8fb0-62523a5b3490
MultitransportRequest(MultitransportRequestPdu),
}

/// End of a logical frame.
FrameEnd,
}
impl TryFrom<x224::ProcessorOutput> for ActiveStageOutput {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: The new code looks broken syntactically?

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR exposes the RDPEGFX FrameEnd (frame marker end) signal up through ironrdp-session’s ActiveStageOutput, enabling protocol-aligned “present”/flush timing for downstream render loops.

Changes:

  • Add a FrameEnd marker to fast_path::UpdateKind and detect SurfaceCommand::FrameMarker(FrameAction::End) in surface command processing.
  • Propagate the internal UpdateKind::FrameEnd to a new public ActiveStageOutput::FrameEnd.
  • Minor doc-link formatting change in ActiveStageOutput docs.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
crates/ironrdp-session/src/fast_path.rs Detect RDPEGFX frame-end markers during surface command processing and surface them as a new update kind.
crates/ironrdp-session/src/active_stage.rs Expose frame-end as a new ActiveStageOutput variant and map from fast-path updates.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +151 to +152
processor_updates.push(UpdateKind::Region(update_region));
processor_updates.extend(markers);
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

process_surface_commands aggregates updates across all commands, but process_single_update always emits UpdateKind::Region(...) before any FrameEnd markers. This reorders FrameEnd relative to the actual command stream and makes it impossible to associate a FrameEnd with the precise set of updates preceding it (e.g., if multiple frame markers appear in one SurfaceCommands update, or if an End marker is not the last command). Consider emitting UpdateKind values in command order (and flushing the accumulated region when encountering FrameAction::End) so FrameEnd matches the image state at that point.

Suggested change
processor_updates.push(UpdateKind::Region(update_region));
processor_updates.extend(markers);
processor_updates.extend(markers);
processor_updates.push(UpdateKind::Region(update_region));

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe false positive, but double check

Copy link
Copy Markdown
Member

@CBenoit Benoît Cortier (CBenoit) left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change is looking good to me! But I think this may be breaking other crates in the workspace (I can think of at least ironrdp-web and ironrdp-client), and this will need to be addressed before we can merge. Also, can you confirm you tested this on your tauri client? Just to be certain about the expected result.

@sfwwslm
Copy link
Copy Markdown
Author

Thanks for the review!

I have tested this change in my Tauri-based RDP client, and everything is working as expected on my side.

For context, my project does not use the ironrdp-web or ironrdp-client crates. Due to upstream security dependency issues, I am currently using a forked branch, and I have verified that this change works correctly within my setup.

However, I’m not able to fully assess the impact of this change on other crates in the workspace that I’m not using.


Reference

Branch:

Dependencies:

ironrdp = { git = "https://github.com/sfwwslm/IronRDP", branch = "fluxterm-runtime", features = ["session", "input", "graphics", "dvc", "svc", "displaycontrol", "connector"] }
ironrdp-tls = { git = "https://github.com/sfwwslm/IronRDP", branch = "fluxterm-runtime", package = "ironrdp-tls", features = ["rustls"] }
ironrdp-tokio = { git = "https://github.com/sfwwslm/IronRDP", branch = "fluxterm-runtime", package = "ironrdp-tokio", features = ["reqwest"] }

@glamberson
Copy link
Copy Markdown
Contributor

Good to see this. We hit the same "session layer consumes protocol data without propagating it" pattern with share_id (#1147), correctly decoded but never stored for downstream use.

Nice approach with the bool flag refactor. Would you find exposing other protocol signals alongside FrameEnd valuable? There are several other PDUs in a similar situation (FrameStart, codec statistics, connection health signals) that application code could use if they were surfaced. It could be great follow-up material.

I'm always glad to see more IronRDP clients. I have several testing clients for my server work but I'd be interested in trying yours if it's available. I've tested frame pacing against lamco-rdp-server and can confirm the tearing artifacts you describe are visible during fast window dragging.

@sfwwslm
Copy link
Copy Markdown
Author

Thank you for the detailed feedback! Regarding the points raised, I’d like to share some context from my side:

On FrameEnd Signaling: During development, I noticed other signaling mechanisms within the protocol. I agree that FrameEnd (and potentially other related signals) should be exposed or handled in a way that allows downstream implementations to utilize them properly. This would ensure better alignment with the RDP specifications.

Current Project Status: My client is still in an early development stage, and I am working on fully eliminating screen tearing. As I am still getting up to speed with the complexities of the RDP protocol, my immediate goal is to achieve a "functional/usable" state for my personal tool.

Contribution Limits: Honestly, due to my current experience level with the protocol and limited personal time for this project, I may not be able to provide deep architectural feedback or implement complex upstream enhancements at this moment.

@glamberson
Copy link
Copy Markdown
Contributor

sfwwslm (@sfwwslm) Well that all sounds great to me. I'm very pleased to hear you're focusing on an extremely important quality issue like tearing. That's jsut the sort of thing I never have time to work on.

I work on architecture a great deal, and my products use a web of protocols, strategies and methods/quirks to construct an RDP session. It's all a little maddening, really, especially with all these protocols being on the bleeding edge of near-stability. If you have a particular quality issue you want to work on, post an issue here, and I'll do what i can to investigate the tooling that might be desirable to help you out.

Thanks and regards,
Greg

@sfwwslm
Copy link
Copy Markdown
Author

Thank you for the detailed feedback and for your time reviewing this!

After a round of testing on my side, I've confirmed that adding the FrameEnd flag did not achieve the expected results for my current implementation. Therefore, I’ve decided to close this PR for now.

If I have any new findings or a more robust solution in the future, I’ll be sure to open a new issue or PR to discuss it. Thanks again for the guidance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants