-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Rework exception compiler #6909
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Caution Review failedThe pull request is closed. 📝 WalkthroughWalkthroughRemoved per-block handler/stack/lasti fields and helpers; simplified fblock push APIs; SetupFinally/SetupCleanup/SetupWith now carry targets; added IR passes to mark/label exception handlers and convert pseudo-ops; unwind paths now emit explicit SetupFinally/SetupCleanup/PopBlock sequences. Changes
Sequence Diagram(s)sequenceDiagram
participant CG as Codegen (compile.rs)
participant IR as IR layer (ir.rs)
participant BC as Bytecode defs (instruction.rs)
participant EM as Emitter/VM
CG->>IR: emit CodeInfo -> Blocks
IR->>IR: mark_except_handlers(blocks)
IR->>IR: label_exception_targets(blocks)
IR->>IR: convert_pseudo_ops(blocks, varnames_len)
IR->>BC: annotate blocks (except_handler, start_depth, preserve_lasti)
BC->>EM: emit pseudo-ops as SetupFinally{target}/SetupCleanup{target}/PopBlock
EM->>EM: ensure PopBlock before POP/EXIT and build exception table
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Code has been automatically formatted The code in this PR has been formatted using:
git pull origin exceptions |
92f8696 to
41538d0
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@crates/codegen/src/compile.rs`:
- Around line 3178-3186: The code emits a SetupFinally and calls
push_fblock(FBlockType::FinallyTry, ...) but stores FBlockDatum::None so the
finally body is not inlined on unwind; replace the push_fblock call in the try*
path (the block that emits PseudoInstruction::SetupFinally and uses
finally_block) with push_fblock_full and pass
FBlockDatum::FinallyBody(finalbody.to_vec()) so unwind_fblock_stack can inline
finalbody on return/break/continue; apply the same change at the other
occurrence around the lines noted (also where PseudoInstruction::SetupFinally
and FBlockType::FinallyTry are used).
🧹 Nitpick comments (3)
crates/codegen/src/ir.rs (3)
247-256: Consider consolidating NOP removal.NOPs are removed here after
convert_pseudo_ops, but there's also a separateremove_nops()call at Line 195. The early call handles NOPs from constant folding, but this creates a two-phase removal pattern that could be confusing.
1266-1269: POP_BLOCK is converted to NOP twice.
POP_BLOCKis converted toInstruction::Nophere inlabel_exception_targets(Line 1269) and again inconvert_pseudo_ops(Lines 1348-1350). The second conversion is redundant but harmless. The comment on Line 1347 acknowledges this.Consider removing the redundant case in
convert_pseudo_opssince it's already handled:♻️ Proposed simplification
// POP_BLOCK → NOP (should already be NOP from label_exception_targets) PseudoInstruction::PopBlock => { - info.instr = Instruction::Nop.into(); + // Already converted to NOP in label_exception_targets + debug_assert!(matches!(info.instr.real(), Some(Instruction::Nop))); }
1360-1367: Unimplemented panic for unexpected pseudo instructions.Using
unimplemented!will cause a panic if these pseudo-ops are encountered. If these represent compiler bugs rather than user errors, consider usingunreachable!with a clearer message, or returning an error.♻️ Proposed change
// These should have been resolved earlier PseudoInstruction::AnnotationsPlaceholder | PseudoInstruction::JumpIfFalse { .. } | PseudoInstruction::JumpIfTrue { .. } | PseudoInstruction::StoreFastMaybeNull(_) => { - unimplemented!( - "Unexpected pseudo instruction in convert_pseudo_ops: {pseudo:?}" - ) + unreachable!( + "Pseudo instruction should have been resolved earlier: {pseudo:?}" + ) }
2ab50de to
0ec4349
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 3
🤖 Fix all issues with AI agents
In `@crates/codegen/src/compile.rs`:
- Around line 2938-2945: The HandlerCleanup fblock is being pushed with
FBlockDatum::None which drops bound exception names and can unbalance
SetupCleanup/PopBlock for except*; update the named-handler path that currently
emits PseudoInstruction::SetupCleanup and calls
push_fblock(FBlockType::HandlerCleanup, cleanup_end, cleanup_end) to carry the
ExceptionName in the fb_datum (i.e., push FBlockDatum::ExceptionName(name)
instead of None) so unwind_fblock/compile_break_continue will clear the binding
correctly, and for except* ensure the number of SetupCleanup emitted matches
PopBlock handling (either emit the second cleanup for the multi-except* case or
adjust unwind_fblock to detect and handle single-cleanup HandlerCleanup
specially) so cleanup counts remain balanced.
In `@crates/codegen/src/ir.rs`:
- Around line 921-932: The subtraction computing handler.stack_depth can
underflow: replace the expression h_start - 1 - (handler.preserve_lasti as u32)
with a saturating subtraction using start_depths/ExceptHandlerInfo values (e.g.,
compute let sub = 1 + (handler.preserve_lasti as u32); handler.stack_depth =
h_start.saturating_sub(sub)) and add a debug_assert!(h_start != u32::MAX) or
similar debug_assert to catch unexpected sentinel values; update the loop in
blocks -> instructions where ExceptHandlerInfo (handler) is adjusted to use this
saturating logic to prevent underflow.
In `@crates/compiler-core/src/bytecode/instruction.rs`:
- Around line 1037-1051: stack_effect for the pseudo-instructions SetupCleanup,
SetupFinally, and SetupWith currently returns 0 while stack_effect_info returns
non-zero handler-entry depths, causing inconsistency; update the
stack_effect_info implementation for the variants SetupCleanup, SetupFinally,
and SetupWith to return (0,0) to match stack_effect(), and add a short doc
comment near stack_effect_info explaining that these instructions' handler-entry
depths are computed separately in max_stackdepth (so stack_effect() reflects
fall-through semantics). Ensure you modify the match arms for those variants in
the stack_effect_info implementation and include the explanatory comment in the
same module.
0ec4349 to
ee4fe41
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
crates/codegen/src/compile.rs (2)
1519-1537:⚠️ Potential issue | 🟠 MajorRestore symbol-table cursor after inline finally compilation.
Inline finalbody compilation during unwind can advancenext_sub_table, desyncing later normal/exception-path compilation for finalbody with nested scopes.🛠️ Suggested fix
// Push PopValue fblock if preserving tos if preserve_tos { self.push_fblock( FBlockType::PopValue, saved_fblock.fb_block, saved_fblock.fb_block, )?; } - self.compile_statements(&body)?; + let sub_table_cursor = + self.symbol_table_stack.last().map(|t| t.next_sub_table); + self.compile_statements(&body)?; + if let Some(cursor) = sub_table_cursor + && let Some(current_table) = self.symbol_table_stack.last_mut() + { + current_table.next_sub_table = cursor; + }
4893-4906:⚠️ Potential issue | 🟠 MajorPop the ExceptionHandler fblock on the suppression path.
Currently the suppress branch jumps toafter_blockwithoutPopBlock/pop_fblock, leaving stale handler state for subsequent unwinds.🛠️ Suggested fix
emit!(self, Instruction::PopTop); // pop __exit__ emit!(self, Instruction::PopTop); // pop lasti + emit!(self, PseudoInstruction::PopBlock); + self.pop_fblock(FBlockType::ExceptionHandler); emit!( self, PseudoInstruction::Jump { target: after_block } );
🧹 Nitpick comments (1)
crates/codegen/src/ir.rs (1)
1204-1326: Consider asserting consistent except stacks at merge points.Because
visitedshort-circuits, a block reached with a different except stack will be ignored. A stored entry stack + debug assert would guard invariants.💡 Suggested defensive check
- #[derive(Clone)] + #[derive(Clone, PartialEq, Eq)] struct ExceptEntry { handler_block: BlockIdx, preserve_lasti: bool, } let mut visited = vec![false; num_blocks]; let mut block_stacks: Vec<Option<Vec<ExceptEntry>>> = vec![None; num_blocks]; + let mut entry_stacks: Vec<Option<Vec<ExceptEntry>>> = vec![None; num_blocks]; // Entry block visited[0] = true; block_stacks[0] = Some(Vec::new()); + entry_stacks[0] = Some(Vec::new()); @@ - if target != BlockIdx::NULL && !visited[target.idx()] { - visited[target.idx()] = true; - block_stacks[target.idx()] = Some(stack.clone()); - todo.push(target); - } + if target != BlockIdx::NULL { + if let Some(existing) = entry_stacks[target.idx()].as_ref() { + debug_assert_eq!(existing, &stack, "mismatched except stack"); + } else { + entry_stacks[target.idx()] = Some(stack.clone()); + } + if !visited[target.idx()] { + visited[target.idx()] = true; + block_stacks[target.idx()] = Some(stack.clone()); + todo.push(target); + } + } @@ - if target != BlockIdx::NULL && !visited[target.idx()] { - visited[target.idx()] = true; - block_stacks[target.idx()] = Some(stack.clone()); - todo.push(target); - } + if target != BlockIdx::NULL { + if let Some(existing) = entry_stacks[target.idx()].as_ref() { + debug_assert_eq!(existing, &stack, "mismatched except stack"); + } else { + entry_stacks[target.idx()] = Some(stack.clone()); + } + if !visited[target.idx()] { + visited[target.idx()] = true; + block_stacks[target.idx()] = Some(stack.clone()); + todo.push(target); + } + }
ee4fe41 to
a0752fd
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
crates/codegen/src/compile.rs (1)
4889-4913:⚠️ Potential issue | 🟠 MajorClose the SetupCleanup region on the suppressed-with path.
The suppressed branch exits the exception handler without emitting
PopBlock, so theSetupCleanupregion can leak intoafter_blockand misroute later exceptions tocleanup_block. Add aPopBlockbeforePopExceptin the suppress path.🛠️ Suggested fix
self.switch_to_block(suppress_block); emit!(self, Instruction::PopTop); // pop True (TO_BOOL result) + emit!(self, PseudoInstruction::PopBlock); emit!(self, Instruction::PopExcept); // pop exc and restore prev_exc emit!(self, Instruction::PopTop); // pop __exit__ emit!(self, Instruction::PopTop); // pop lasti
🧹 Nitpick comments (1)
crates/codegen/src/ir.rs (1)
863-867: Persist computedstart_depthintoBlock.start_depth.
start_depthsis computed but never stored, leaving the new fieldNoneand out of sync with the comment. Consider populating it for downstream uses.♻️ Suggested update
if DEBUG { eprintln!("DONE: {maxdepth}"); } + + // Persist block entry depths for downstream consumers + for (idx, block) in self.blocks.iter_mut().enumerate() { + let depth = start_depths[idx]; + if depth != u32::MAX { + block.start_depth = Some(depth); + } + } // Fix up handler stack_depth in ExceptHandlerInfo using start_depths
a0752fd to
41860a9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@crates/codegen/src/compile.rs`:
- Around line 3021-3026: The finally-entry path currently emits
PseudoInstruction::PopBlock but leaves the FinallyTry fblock on the
fblock/except stack, causing handlers inside finally to re-trigger unwind logic;
update the handler-to-finally path to also pop the FinallyTry fblock before
jumping into the finally body (same behavior as the else_block path) so the
except stack is consistent when entering finally_block — locate the logic around
finalbody, finally_block, else_block and the emitted PseudoInstruction::PopBlock
and ensure the corresponding fblock (FinallyTry) is removed from the
fblock/except stack prior to entering the finally body.
In `@crates/codegen/src/ir.rs`:
- Around line 824-858: The SEND jump path is being treated with jump_effect = -1
which undercounts target block start depth because END_SEND (which actually pops
the receiver) may not have executed yet; modify the SEND handling in the block
(the match on instr.real() / Instruction::Send) to use jump_effect = 0 by
default for SEND, and only use -1 when the jump target is strictly after the
END_SEND instruction for that send (i.e., detect the END_SEND instruction
index/label corresponding to this Send and compare ins.target to it); update the
computation that sets target_depth and maxdepth and leave stackdepth_push usage
unchanged (symbols to locate: instr.real(), Instruction::Send, ins.target,
END_SEND detection logic, stackdepth_push).
fa2c13f to
ed6e28f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
crates/codegen/src/compile.rs (1)
4897-4905:⚠️ Potential issue | 🟠 MajorClear
ExceptionHandlerfblock on the suppressedwithpath.The suppressed path jumps to
after_blockwithoutPOP_BLOCK/pop_fblock, leaving the handler active and corrupting later unwinds/exception table coverage.🛠️ Suggested fix
self.switch_to_block(suppress_block); emit!(self, Instruction::PopTop); // pop True (TO_BOOL result) + emit!(self, PseudoInstruction::PopBlock); + self.pop_fblock(FBlockType::ExceptionHandler); emit!(self, Instruction::PopExcept); // pop exc and restore prev_exc emit!(self, Instruction::PopTop); // pop __exit__ emit!(self, Instruction::PopTop); // pop lasti
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@crates/codegen/src/ir.rs`:
- Around line 1221-1224: The POP_BLOCK handling silently pops an empty exception
stack; instead guard the pop and surface a failure for malformed CFGs by
checking whether stack.is_empty() before calling stack.pop() inside the is_pop
branch for POP_BLOCK; if empty, return or panic with a diagnostic containing the
block index (bi) and instruction index (i) (or otherwise propagate an Err) so
the invalid CFG is detected, otherwise perform stack.pop() and set
blocks[bi].instructions[i].instr = Instruction::Nop.into() as before.
🧹 Nitpick comments (2)
crates/codegen/src/ir.rs (2)
824-838: Use checked_add for handler depth to keep overflow handling consistent.
The rest of the stack-depth computation uses checked arithmetic; this path currently doesn’t.♻️ Proposed fix
- let handler_depth = depth + handler_effect; + let handler_depth = depth + .checked_add(handler_effect) + .ok_or(InternalError::StackOverflow)?;
873-889: Consider persisting computed start depths intoBlock.start_depth.
The field is added but never populated, which can mislead downstream consumers.♻️ Proposed fix
- // Fix up handler stack_depth in ExceptHandlerInfo using start_depths + // Persist block start depths for downstream consumers (if any) + for (i, block) in self.blocks.iter_mut().enumerate() { + block.start_depth = (start_depths[i] != u32::MAX).then_some(start_depths[i]); + } + + // Fix up handler stack_depth in ExceptHandlerInfo using start_depths
| } else if is_pop { | ||
| stack.pop(); | ||
| // POP_BLOCK → NOP | ||
| blocks[bi].instructions[i].instr = Instruction::Nop.into(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Guard POP_BLOCK underflow to catch malformed exception stacks.
Currently an empty stack is silently ignored, which can hide invalid CFGs.
🛡️ Proposed fix
} else if is_pop {
+ debug_assert!(
+ !stack.is_empty(),
+ "POP_BLOCK without matching SETUP_*"
+ );
stack.pop();
// POP_BLOCK → NOP
blocks[bi].instructions[i].instr = Instruction::Nop.into();
} else {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| } else if is_pop { | |
| stack.pop(); | |
| // POP_BLOCK → NOP | |
| blocks[bi].instructions[i].instr = Instruction::Nop.into(); | |
| } else if is_pop { | |
| debug_assert!( | |
| !stack.is_empty(), | |
| "POP_BLOCK without matching SETUP_*" | |
| ); | |
| stack.pop(); | |
| // POP_BLOCK → NOP | |
| blocks[bi].instructions[i].instr = Instruction::Nop.into(); |
🤖 Prompt for AI Agents
In `@crates/codegen/src/ir.rs` around lines 1221 - 1224, The POP_BLOCK handling
silently pops an empty exception stack; instead guard the pop and surface a
failure for malformed CFGs by checking whether stack.is_empty() before calling
stack.pop() inside the is_pop branch for POP_BLOCK; if empty, return or panic
with a diagnostic containing the block index (bi) and instruction index (i) (or
otherwise propagate an Err) so the invalid CFG is detected, otherwise perform
stack.pop() and set blocks[bi].instructions[i].instr = Instruction::Nop.into()
as before.
Move exception handler tracking from compile-time fblock metadata to
a post-codegen CFG analysis pass, matching flowgraph.c's pipeline.
Compiler changes (compile.rs):
- Remove fb_handler, fb_stack_depth, fb_preserve_lasti from FBlockInfo
- Remove handler_stack_depth() and current_except_handler() helpers
- Emit SetupFinally/SetupCleanup/SetupWith/PopBlock pseudo instructions
instead of manually tracking handlers in fblocks
- Add dead blocks after raise/break/continue/return to prevent
dead code from corrupting the except stack
- Add missing PopTop after CallIntrinsic1 ImportStar
- Fix async comprehension SetupFinally/GetANext emission order
- Fix try-except* handler: move BUILD_LIST/COPY before handler loop
- Rework unwind_fblock for HandlerCleanup, TryExcept, FinallyTry,
FinallyEnd, and With/AsyncWith to emit proper PopBlock sequences
New CFG analysis passes (ir.rs):
- mark_except_handlers(): mark blocks targeted by SETUP_* instructions
- label_exception_targets(): walk CFG with except stack to set
per-instruction handler info and convert POP_BLOCK to NOP
- convert_pseudo_ops(): lower remaining pseudo ops after analysis
- Compute handler entry depth from SETUP_* type in max_stackdepth()
(SETUP_CLEANUP=+2, SETUP_FINALLY/SETUP_WITH=+1)
- Fix SEND jump_effect from -1 to 0 (receiver stays until END_SEND)
- Add underflow guard for handler stack_depth calculation
Instruction metadata (instruction.rs):
- SetupCleanup/SetupFinally/SetupWith now carry target: Arg<Label>
- Add is_block_push()/is_pop_block() to PseudoInstruction/AnyInstruction
- Fix LoadSpecial stack_effect from (2,2) to (1,1)
- Set Setup* stack_effect_info to (0,0) for fall-through consistency
Fixes 3 test_coroutines expectedFailures:
- test_with_8, test_for_assign_raising_stop_async_iteration{,_2}
ed6e28f to
7704b71
Compare
📦 Library DependenciesThe following Lib/ modules were modified. Here are their dependencies: [x] test: cpython/Lib/test/test_coroutines.py (TODO: 22) dependencies: dependent tests: (no tests depend on coroutines) [ ] lib: cpython/Lib/enum.py dependencies:
dependent tests: (10 tests)
Legend:
|
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.