Skip to content

Draft of branch coverage support#118305

Closed
Zalathar wants to merge 8 commits intorust-lang:masterfrom
Zalathar:branch
Closed

Draft of branch coverage support#118305
Zalathar wants to merge 8 commits intorust-lang:masterfrom
Zalathar:branch

Conversation

@Zalathar
Copy link
Member

@Zalathar Zalathar commented Nov 26, 2023

EDIT: I've closed this and re-opened a non-draft version at #122322.

This is a work-in-progress snapshot of my implementation of branch coverage.

It works, and produces correct results in many cases. However, it sometimes produces confusing or incorrect results for if expressions that contain the !/&&/|| operators, since the THIR-to-MIR lowering of those is non-trivial.

@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Nov 26, 2023
@Zalathar
Copy link
Member Author

@rustbot label +A-code-coverage

@rustbot rustbot added the A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) label Nov 26, 2023
@Zalathar
Copy link
Member Author

@rustbot author

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 26, 2023
Comment on lines +21 to +16
LL| | if
LL| | !
LL| 15| cond
------------------
| Branch (LL:9): [True: 10, False: 5]
------------------
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two problems here:

  • The span doesn't include !.
  • The true/false counts are reversed, since they're tracking cond instead of !cond.

Comment on lines +37 to +64
LL| | if
LL| 15| a
------------------
| Branch (LL:9): [True: 12, False: 3]
------------------
LL| | &&
LL| 12| b
------------------
| Branch (LL:9): [True: 8, False: 4]
------------------
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks correct, which is nice.

Comment on lines +77 to +135
LL| | if
LL| | let
LL| | true
LL| | =
LL| 15| cond
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This currently doesn't get branch coverage at all. I'd like to fix that if possible, but for me it's a lower priority than getting the logical ops correct.

@Zalathar
Copy link
Member Author

cc @Swatinem in case you're curious.

The MIR building stuff will have to change once I figure out how to handle the logical ops better.

@bors
Copy link
Collaborator

bors commented Nov 26, 2023

☔ The latest upstream changes (presumably #118300) made this pull request unmergeable. Please resolve the merge conflicts.

@Zalathar
Copy link
Member Author

Well, this definitely indicates a need to add macro-expanded spans to the list of things I need to figure out.

The existing coverage code has some functions in spans.rs for dealing with this sort of thing, which I can probably look to for inspiration.

@Zalathar
Copy link
Member Author

I tried passing each of the branch spans through function_source_span. That gets rid of the bogus spans, but it also results in a lot of unhelpful noise for assertions.

@Zalathar Zalathar force-pushed the branch branch 2 times, most recently from 996fbe5 to 9000afd Compare November 28, 2023 09:04
@Zalathar
Copy link
Member Author

Now that the || bug is fixed, there are no known correctness issues remaining. The ! mappings are non-ideal, but they're not wrong.

So I'm thinking it might make sense for me to polish the current functionality into a review-ready state, so that people can start playing with branch coverage on nightly and start flushing out any lingering correctness issues.

@rust-log-analyzer

This comment has been minimized.

@Zalathar Zalathar force-pushed the branch branch 3 times, most recently from 811cb89 to 934e104 Compare November 29, 2023 02:13
@Zalathar
Copy link
Member Author

I found a way to fix the mappings for !: pass an extra enclosing_not_expr: Option<&Expr<'tcx>> argument to the recursive call.

We also need to propagate it through the ExprKind::Scope case, but the other recursive cases reset it to None.

@Zalathar
Copy link
Member Author

Zalathar commented Feb 8, 2024

Big rebase!

I had to resolve some non-trivial merge conflicts, and I also changed the MIR builder code to be specific to branch coverage only, since currently there are no concrete plans to use it for recording non-branch coverage info (though this still remains a future possibility).

@bors
Copy link
Collaborator

bors commented Feb 13, 2024

☔ The latest upstream changes (presumably #121036) made this pull request unmergeable. Please resolve the merge conflicts.

@Dylan-DPC Dylan-DPC added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Feb 17, 2024
@Zalathar
Copy link
Member Author

The main concern I have with this PR at the moment is that the way it handles ! expressions is pretty fragile. Because of things like #111752, the current MIR building code makes it a little awkward to determine which arm of an if is the true-value arm, and which is the false-value arm.

(The current approach tries to keep track of the outermost enclosing !, and then walks back down the THIR tree counting negations until it hits a non-negation node. This avoids major changes to the existing MIR-building code, and gets the job done, but I don't feel good about it.)

My preferred approach would be to rearrange the MIR-building code in a way that naturally satisfies the needs of branch coverage (and beyond), without adversely affecting non-coverage builds. But that's going to require some care, because this code already has some important and subtle responsibilities.

@Zalathar
Copy link
Member Author

Updated for #121370 (//@ directives in tests).

@Zalathar
Copy link
Member Author

I've adjusted how I handle the issue of || producing incorrect counts (diff).

Instead of sneakily creating an extra block when inserting the block marker, I decided to just fix the issue at its source, by making the || lowering create a new block for the LHS and RHS success paths to converge at.

This adds one extra block to the lowering of || even when coverage is disabled, but that extra block is soon removed by the MIR simplifier.

@Zalathar
Copy link
Member Author

I received some questions about this implementation, which I have answered in a Zulip thread: https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Branch.20coverage.20questions

@bors
Copy link
Collaborator

bors commented Mar 1, 2024

☔ The latest upstream changes (presumably #121859) made this pull request unmergeable. Please resolve the merge conflicts.

@bors
Copy link
Collaborator

bors commented Mar 5, 2024

☔ The latest upstream changes (presumably #121998) made this pull request unmergeable. Please resolve the merge conflicts.

@bors
Copy link
Collaborator

bors commented Mar 8, 2024

☔ The latest upstream changes (presumably #122151) made this pull request unmergeable. Please resolve the merge conflicts.

@Zalathar
Copy link
Member Author

Zalathar commented Mar 10, 2024

Switched to a different mechanism for handling ! expressions (diff), which is less intrusive to rustc_mir_build::build::matches, and should be a bit easier to maintain/extend.

@Zalathar
Copy link
Member Author

Now that I've resolved my own concerns, I think it might be time to close this and open a new PR for actual review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants