I recently went through the Anthropic Fellow online assessment, which came in two parts. OA1 is a 90-minute coding build split into five escalating levels; within half an hour of submitting, OA2, a debugging task, landed in my inbox. This debrief captures the format, the real pace, and the core focus of both parts. If you're prepping for an Anthropic-style OA that leans into systems and ML engineering, I hope this note saves you some detours. If you want OA assist or OA live support, you can align your pace with the approaches below.
OA Flow Overview
| Stage | Time | Format | Focus |
|---|---|---|---|
| OA1 | 90 minutes | Five independent levels, one .py file each, calling given system code (system code cannot be modified) |
Systems implementation, state-machine modeling, escalating difficulty |
| OA2 | Sent within half an hour of submitting | Debugging task: fix bugs in given code (ML-flavored) | Code reading, defect localization, keeping vectorization |
The two parts feel completely different. OA1 is more of a systems implementation: you build the scheduling logic on top of interfaces someone else provides. OA2 is more like firefighting in production: it tests reading and debugging code. Let's break down each.
OA1: LLM Inference Request Scheduler (Five Escalating Levels)
Background
The five levels of OA1 revolve around one theme: implement an LLM inference request scheduler in the style of vLLM / SGLang. Each level is an independent .py file that must call functions from the provided "system code," and the system code itself cannot be changed. Each level implements something different, escalating in difficulty. Here I'll use the level that best captures the scheduling core.
A request's lifecycle looks like this:
- The request first enters the
waitingqueue. - After one Prefill pass it becomes
admitted. - Then it Decodes token by token.
- Once the generated token count reaches
max_tokens, it isfinished.
Each time step, GPU work has a ceiling max_work, and one step cannot consume more than that. The scheduling order is a hard requirement:
- First, Decode every
admittedrequest in arrival order, each consuming 1 unit. - Then, with remaining capacity, Prefill
waitingrequests in arrival order, each consumingprompt_len. - Key constraint: if the request at the head of
waitingdoesn't fit, stop immediately for this round. You cannot skip it to schedule a smaller later request.
Approach
Maintain three state groups: waiting / admitted / finished. Each step starts from remaining = max_work:
- Decode phase: iterate
admittedin arrival order, each consuming 1 unit; a request whosegeneratedreachesmax_tokensmoves tofinished, otherwise it staysadmitted. - Prefill phase: take the request at the head of
waiting, and Prefill it intoadmittedonly ifprompt_len <= remaining; the moment the head doesn't fit,break. - Each step returns the tasks it scheduled, and updates states and generated-token counts.
Note one detail: a request Prefilled in this step will not Decode in the same step, because Prefill runs after Decode, so it only starts emitting tokens on the next step.
Python Implementation
First, the interface sketch of the provided "system code" (read-only, cannot be modified):
from collections import deque
from typing import Dict, List
# ==== System code (cannot be modified, interface sketch only) ====
class Request:
def __init__(self, req_id: int, prompt_len: int, max_tokens: int):
self.req_id = req_id
self.prompt_len = prompt_len # GPU units required by Prefill
self.max_tokens = max_tokens # total tokens to Decode
self.generated = 0 # tokens generated so far
def prefill(req: Request) -> int:
"""System code: run Prefill, return the GPU units consumed."""
return req.prompt_len
def decode_one(req: Request) -> int:
"""System code: Decode one token, return units consumed (always 1)."""
req.generated += 1
return 1
Complexity: both prefill and decode_one are O(1).
Next, the scheduler we implement:
class Scheduler:
"""LLM inference scheduler: Decode all admitted first, then Prefill waiting with what's left."""
def __init__(self, max_work: int):
self.max_work = max_work
self.waiting: deque[Request] = deque() # not yet Prefilled
self.admitted: deque[Request] = deque() # Prefilled, currently Decoding
self.finished: List[Request] = []
def add(self, req: Request) -> None:
self.waiting.append(req)
def step(self) -> Dict[str, List[int]]:
remaining = self.max_work
decoded: List[int] = []
prefilled: List[int] = []
# Phase 1: Decode -- serve all admitted in arrival order, 1 unit each
carry: deque[Request] = deque()
while self.admitted:
req = self.admitted.popleft()
if remaining < 1:
carry.append(req) # out of capacity, Decode next step
continue
remaining -= decode_one(req) # consume 1 unit, generated += 1
decoded.append(req.req_id)
if req.generated >= req.max_tokens:
self.finished.append(req) # hit the cap, finished
else:
carry.append(req)
self.admitted = carry
# Phase 2: Prefill -- start from the head of waiting, stop the moment the head doesn't fit
while self.waiting:
head = self.waiting[0]
if head.prompt_len > remaining:
break # head doesn't fit: cannot skip it for a smaller later request
self.waiting.popleft()
remaining -= prefill(head)
self.admitted.append(head)
prefilled.append(head.req_id)
return {"decode": decoded, "prefill": prefilled}
Complexity: O(A + P) per step, where A is the current admitted count and P is the number newly Prefilled this step; space O(N), N being the total in-flight requests.
Dry Run Walkthrough
Let max_work = 4, and add three requests up front (arrival order R0, R1, R2):
| Request | prompt_len | max_tokens |
|---|---|---|
| R0 | 2 | 2 |
| R1 | 3 | 1 |
| R2 | 1 | 1 |
Stepping through step():
| step | starting remaining | Decode | Prefill | end state (waiting / admitted / finished) |
|---|---|---|---|---|
| 1 | 4 | — | R0 (-2); R1 needs 3 > 2, stop | [R1,R2] / [R0] / [] |
| 2 | 4 | R0 (-1, generated=1) | R1 (-3); R2 needs 1 > 0, stop | [R2] / [R0,R1] / [] |
| 3 | 4 | R0 (generated=2, finished), R1 (generated=1, finished) | R2 (-1) | [] / [R2] / [R0,R1] |
| 4 | 4 | R2 (generated=1, finished) | waiting empty | [] / [] / [R0,R1,R2] |
Focus on step 1: with remaining = 2, the head R1 needs 3 and doesn't fit, and even though the later R2 needs only 1 you cannot skip R1 to Prefill R2. That's the trap the problem is most likely to catch you on.
Scoring Strategy
The five levels escalate, so the safe play is to nail the early base levels first. Once the state machine (waiting -> admitted -> finished) and the main trunk ("Decode first, then Prefill, stop when the head doesn't fit") are correct, the later levels usually stack constraints on top of it (preemption, priority, batch-size caps). Don't grind on the last level from the start. Lock in the base points, then push up.
OA2: Debug Extremely Randomized Trees
Background
Within half an hour of submitting OA1, OA2 arrived in my inbox: a debugging task, ML-flavored. It gives you a mostly-working but buggy Extremely Randomized Trees (Extra-Trees) implementation whose defects cause crashes or bad accuracy. You read trees.py and tests/test_trees.py, and locate and fix issues starting from the failing tests. The tests and docs cannot be changed; you're not required to optimize for speed, but you must not replace NumPy vectorized ops with obviously slower multi-level loops.
Approach
Don't read all the code up front. Instead, work backward from the errors and failing tests:
- Run the full test suite first, and see which cases fail and what errors they throw.
- Narrow down per failing case, localizing to a specific function.
- Focus on these areas:
- Whether the tree's stopping conditions (depth / sample count / purity) are correct.
- Random feature and random threshold selection (the core of Extra-Trees is a threshold drawn uniformly within the feature's value range).
- The left/right split mask boundaries (
<vs<=, and whether left and right are complementary and complete). - Empty-node / leaf handling.
- Prediction aggregation (majority vote for classification, mean for regression).
- The random-seed convention the tests expect.
- Keep the original interface and NumPy vectorization; re-run relevant tests after each fix, then the full suite.
A Typical Bug and Fix
The most common defect lives in the left/right split: the right subset should be the complement of the left, but a copy-paste error leaves it pointing the same direction, so the right subtree is always empty. Either the recursion never stops, or accuracy collapses.
# Before (bug): left and right use the same comparison direction, right subset always empty
left_mask = X[:, feat] <= threshold
right_mask = X[:, feat] <= threshold # copy-paste error, should be the complement
# After (fix): right subset is the complement, keeping the split mutually exclusive and complete, and still vectorized
left_mask = X[:, feat] <= threshold
right_mask = ~left_mask
Another frequent trap is an off-by-one in random feature sampling: np.random.randint(0, n_features - 1) can never pick the last feature, so it should be np.random.randint(0, n_features) (exclusive upper bound).
# Before (bug): the last feature is never selected
feat = np.random.randint(0, n_features - 1)
# After (fix): randint's upper bound is exclusive, so use n_features to cover every feature
feat = np.random.randint(0, n_features)
Note: these fixes are constant-size edits and don't change the overall complexity. Tree building stays on the order of O(n * d * depth), and the vectorized split stays O(n). Never degrade it into per-sample multi-level loops just to make it "look right."
Prep Strategy
- OA1: lock in the base points first. Solidify the request state machine and the main trunk ("Decode first, the waiting head can't be skipped"). Secure the early levels, then stack constraints toward the harder ones.
- OA2: start from the errors. Run the full suite first and localize with failing cases rather than reading all the code; fix one spot, run once, converge step by step.
- Respect the given constraints. System code can't change, tests can't change, vectorization can't degrade. These are implicit scoring points, and crossing the line costs you directly.
- Pace the handoff between parts. OA2 comes within half an hour of submitting OA1, so don't fully switch off in between; keep the energy to shift into "read code plus debug" mode.
FAQ
Q1: Do I have to finish all five levels of OA1?
Not necessarily. The five levels escalate and scoring is cumulative. The safe strategy is to fully solve and max out the early base levels first, then push toward the harder ones. Get the main trunk (state machine plus scheduling order) right, and the later levels are usually constraints stacked on top of it.
Q2: In scheduling, why stop when the head doesn't fit instead of skipping to a smaller request?
That's the exact scheduling semantics the problem specifies: Prefill follows arrival order strictly, and when the waiting head doesn't fit, this round stops immediately, guaranteeing first-come-first-served and preventing large requests from being deferred indefinitely. Skipping the head to Prefill a smaller later request violates the intent and gets marked wrong.
Q3: Does a request Prefilled in this step also Decode in the same step?
No. Each step is "Decode all admitted first, then Prefill waiting." A request is only Prefilled and added to admitted after the Decode phase, so it starts Decoding token by token on the next step.
Q4: For the OA2 debugging task, should I read all the code first?
Not recommended. The more efficient move is to run the full test suite first, work backward from the failing cases and error messages, and localize to a specific function before reading the local code. Focus on stopping conditions, random feature / threshold selection, the left/right split mask, empty-node handling, and prediction aggregation.
Q5: OA2 says not to degrade vectorization. What does that mean exactly?
The problem lets you skip peak performance, but forbids rewriting NumPy's vectorized ops (boolean-mask splits, batch comparisons) into obviously slower per-sample multi-level loops. When fixing bugs, keep the original vectorized form and hold the change to constant size.
Prepping for an Anthropic Fellow-style OA that blends systems implementation with ML engineering? We know vLLM / SGLang-style inference scheduling problems and Extra-Trees-style debugging tasks, and can help you polish state-machine modeling, scheduling semantics, and debugging pace in one pass, with end-to-end OA assist and OA live support.
Add WeChat Coding0201 now to get a one-on-one custom prep plan.
Contact
- WeChat: Coding0201
- Email: [email protected]
- Telegram: @OAVOProxy