Scoring Server - Design a Low-Latency Real-Time Scoring Service
Interview Experience
Round 1 System Design
Problem
Design a scoring server that receives raw feature vectors in real time and returns model scores within 20ms p99. The model is a gradient boosted tree (100 MB serialized). The server handles 50K requests/sec at peak.
Requirements
- Latency: p99 < 20ms end-to-end (network + inference).
- Throughput: 50K RPS peak, 10K RPS average.
- The model is updated daily; zero-downtime rollout required.
- Feature input: JSON payload, ~50 float fields.
Design Points
Load Balancer -> Scoring Fleet (stateless workers)
Workers: deserialize JSON -> validate -> run model ->
**return** score
Model loaded in-process (no subprocess call)
Blue/Green deploy: new model warmed up, traffic shifted atomically
Discussion Questions
- How do you manage model warm-up time when spinning up new instances?
- How do you validate incoming features for schema drift before scoring?
- What metrics do you instrument: latency histogram, score distribution, error rate?
Follow-ups
- How do you A/B test two model versions in production with consistent user assignment?
- What happens when a feature is missing in the payload — impute, reject, or score with default?
- How do you handle a latency spike caused by a single slow feature transformation?
- How would the design differ for a deep learning model that requires a GPU?
Full Details
Round 1 System Design
Problem
Design a scoring server that receives raw feature vectors in real time and returns model scores within 20ms p99. The model is a gradient boosted tree (100 MB serialized). The server handles 50K requests/sec at peak.
Requirements
- Latency: p99 < 20ms end-to-end (network + inference).
- Throughput: 50K RPS peak, 10K RPS average.
- The model is updated daily; zero-downtime rollout required.
- Feature input: JSON payload, ~50 float fields.
Design Points
Load Balancer -> Scoring Fleet (stateless workers)
Workers: deserialize JSON -> validate -> run model ->
**return** score
Model loaded in-process (no subprocess call)
Blue/Green deploy: new model warmed up, traffic shifted atomically
Discussion Questions
- How do you manage model warm-up time when spinning up new instances?
- How do you validate incoming features for schema drift before scoring?
- What metrics do you instrument: latency histogram, score distribution, error rate?
Follow-ups
- How do you A/B test two model versions in production with consistent user assignment?
- What happens when a feature is missing in the payload — impute, reject, or score with default?
- How do you handle a latency spike caused by a single slow feature transformation?
- How would the design differ for a deep learning model that requires a GPU?
About This Question
This is a candidate experience report from a ixl interview during the phone round.
It covers the following topics: Phone, System Design, System Design, Coding, Onsite .
Topics
About Ixl Interview Reports
This question was reported by a candidate who interviewed at Ixl. LeakCode aggregates interview reports from 10+ sources, including 1Point3Acres, Glassdoor, LeetCode Discuss, Blind, Reddit, Indeed, and Nowcoder. Each report is translated where necessary, deduplicated against existing entries, and tagged by company, role, round type, and reporting date.
Use this question as one calibration data point, not a memorization target. Companies typically rotate their question pools every 2-4 months; the exact wording of a 2024 question may differ from what you encounter today. The underlying pattern, difficulty level, and follow-up depth at Ixl are the higher-signal extractions to take from this report.
For broader preparation context, the Ixl interview process typically includes a recruiter screen, one or two technical phone screens, and a 4-5 round on-site loop covering coding, system design (at L4+ levels), and behavioral. Reports tagged on LeakCode show the round-by-round distribution and typical difficulty calibration. To browse questions filtered by round type and seniority, use the company hub linked above.
How To Practice This Type of Question
Solve similar problems on LeetCode under timed conditions (25-35 minutes per medium difficulty). The goal is pattern recognition: recognize the underlying technique (sliding window, two-pointer, BFS, memoized recursion, etc.) within 60-90 seconds of reading. Strong candidates verbalize their hypothesis out loud before coding, then iterate based on feedback. Weak candidates dive into implementation immediately, lose time on the wrong approach, and run out of time for follow-ups.
Companies update their question pools every 2-4 months. The exact wording of any given question may have been retired by the time you interview. Focus your prep on the pattern, not the specific problem. The patterns that appear in Ixl reports consistently are the ones worth investing in; one-off niche problems are not.
During Your Ixl Round
Apply the standard interview round template: clarify requirements (2-3 minutes), state your approach out loud and confirm direction with the interviewer (3-5 minutes), code with narration (15-25 minutes), test with concrete examples including edge cases (5 minutes), discuss optimization or trade-offs if time permits (5 minutes). This template is universally accepted across FAANG and adjacent companies; deviating from it produces weaker interviewer feedback signal.
The single most predictive failure mode in Ixl reports tagged "no hire": not asking clarifying questions. Interviewers are explicitly trained to weight this. Strong candidates ask 3-5 clarifying questions even on problems that look obvious; weak candidates dive into code immediately. The clarifying-question check is often the first signal recorded in the interviewer's written notes.