Skip to content

Commit c8fadd9

Browse files
fhahngithub-actions[bot]
authored andcommitted
Automerge: [LV] Retrieve middle VPBB via scalar ph to fix epilogue resumephis (NFC)
If ScalarPH has predecessors, we may need to update its reduction resume values. If there is a middle block, it must be the first predecessor. Note that the first predecessor may not be the middle block, if the middle block doesn't branch to the scalar preheader. In that case, fixReductionScalarResumeWhenVectorizingEpilog will be a no-op. In preparation for llvm/llvm-project#106748.
2 parents 347c88f + cdff7f0 commit c8fadd9

File tree

1 file changed

+14
-5
lines changed

1 file changed

+14
-5
lines changed

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7786,7 +7786,6 @@ DenseMap<const SCEV *, Value *> LoopVectorizationPlanner::executePlan(
77867786

77877787
BestVPlan.execute(&State);
77887788

7789-
auto *MiddleVPBB = BestVPlan.getMiddleBlock();
77907789
// 2.5 When vectorizing the epilogue, fix reduction resume values from the
77917790
// additional bypass block.
77927791
if (VectorizingEpilogue) {
@@ -7801,10 +7800,20 @@ DenseMap<const SCEV *, Value *> LoopVectorizationPlanner::executePlan(
78017800
Phi.addIncoming(Phi.getIncomingValueForBlock(BypassBlock), Pred);
78027801
}
78037802
}
7804-
7805-
for (VPRecipeBase &R : *MiddleVPBB) {
7806-
fixReductionScalarResumeWhenVectorizingEpilog(
7807-
&R, State, State.CFG.VPBB2IRBB[MiddleVPBB], BypassBlock);
7803+
VPBasicBlock *ScalarPH = BestVPlan.getScalarPreheader();
7804+
ArrayRef<VPBlockBase *> ScalarPreds = ScalarPH->getPredecessors();
7805+
if (!ScalarPreds.empty()) {
7806+
// If ScalarPH has predecessors, we may need to update its reduction
7807+
// resume values. If there is a middle block, it must be the first
7808+
// predecessor. Note that the first predecessor may not be the middle
7809+
// block, if the middle block doesn't branch to the scalar preheader. In
7810+
// that case, fixReductionScalarResumeWhenVectorizingEpilog will be a
7811+
// no-op.
7812+
auto *MiddleVPBB = cast<VPBasicBlock>(ScalarPreds[0]);
7813+
for (VPRecipeBase &R : *MiddleVPBB) {
7814+
fixReductionScalarResumeWhenVectorizingEpilog(
7815+
&R, State, State.CFG.VPBB2IRBB[MiddleVPBB], BypassBlock);
7816+
}
78087817
}
78097818
}
78107819

0 commit comments

Comments
 (0)