## Round 1 - MLE Coding ## Problem Implement the backward pass (backpropagation) for a single fully-connected layer with ReLU activation. You are given the gradient flowing in from the next layer and must compute the gradients with respect to the layer's weights, biases, and input. ```python import numpy as np def relu(x: np.ndarray) -> np.ndarray: ... def relu_grad(x: np.ndarray) -> np.ndarray: ... # 1 where x>0, else 0 def linear_backward( dout: np.ndarray, # gradient from upstream, shape (batch, out_features) x: np.ndarray, # input to this layer, shape (batch, in_features) W: np.ndarray, # weights, shape (in_features, out_features) b: np.ndarray, # biases, shape (out_features,) pre_activation: np.ndarray # Wx + b before ReLU, shape (batch, out_features) ) -> tuple: # Returns (dx, dW, db) ... ``` ``` Shapes check: dout: (32, 128) x: (32, 256) W: (256, 128) pre_activation: (32, 128) dx: (32, 256) <- gradient w.r.t. input dW: (256, 128) <- gradient w.r.t. weights db: (128,) <- gradient w.r.t. bias ``` ## Follow-ups 1. What does multiplying `dout` by `relu_grad(pre_activation)` represent in the chain rule? 2. Why is `dW = x.T @ delta` and not `delta @ x.T`? 3. How would the backward pass change for sigmoid activation instead of ReLU? 4. How would you numerically verify your gradient implementation?

InterviewDB Experience

ML Puzzle Coding - Implement Neural Network Backpropagation Step

phone

Interview Experience

Round 1 - MLE Coding Problem Implement the backward pass (backpropagation) for a single fully-connected layer with ReLU activation. You are given the gradient flowing in from the next layer and must compute the gradients with respect to the layer's weights, biases, and input. Follow-ups What does multiplying dout by relu_grad(pre_activation) represent in the chain rule? Why is dW = x.T @ delta and not delta @ x.T? How would the backward pass change for sigmoid activation instead of ReLU? How wou…

Full Details

🔒

Unlock all OpenAI questions

Full insider details, leaked discussions, and candidate experiences.

or every company, $100/year →

About This Question

This is a candidate experience report from a openai interview during the phone round.

It covers the following topics: Mle, Onsite, Phone .

Topics

Mle Onsite Phone

ML Puzzle Coding - Implement Neural Network Backpropagation Step

Interview Experience

Full Details

About This Question

Topics

More OpenAI Interview Questions