Inference Engineer, vLLM & CUDA
Halide Labs · Remote (EU / US)
$180k–$240k
PyTorchTritonCUDA
IDsigned
Ines Duarte
The AI Talent Layer.
Built exclusively for companies hiring in AI. Discover engineers, researchers, and builders through intelligent matching—not resume databases.
Every candidate is vetted by someone who's actually built and shipped AI systems — not a keyword filter.
3rdYour company
2ndYour product
1stYour AI systems
0thThe talent that builds all of it
What the product produces
Candidate 4 of 7
“Doesn't just build demos — ships systems that hold up in production. Found a reliability bug in our pipeline our own team had missed for weeks.”
Ines Duarte
Member of Technical Staff · Halide Labs
The layer under the automation.
reliabilityperformance20m ago
How fast we catch problems before they cost you
ID
Ines Duarte
Member of Technical Staff, Halide Labs
41 replies12 useful
The problem
You can't retrain your way to an AI team.
Most companies hiring for AI don't have the in-house knowledge to tell who's actually good at it — so they send existing engineers to a course and hope. That's not how this works. New systems need people who've already built and shipped them, and a hiring process wasn't built to find those people. We're not another database. We're your hiring partner: we learn your company and your vision, then find the people who can actually get you there.
Good candidate, auto-rejectedKeyword match: 61%Filtered by job title, not skill412 resumes. 3 worth a call.Sourced from a stale databaseStill open after 6 months
We've recently joined hands with their team and we're building a sustainable tech ecosystem together. It's been amazing working with their team — the coordination and execution is simple and efficient, and their talent is brilliant.
How it works
One human step, put back where it belongs.
Five steps. The only automated part is the email that tells you which step you're on.
- 0th
Create your profile
Just the basics — no essay, no résumé formatting. A few details about you and what you're looking for.
- 1st
Apply, or reach out directly
Apply to roles on the board, or email hire@0thlayer.com if you're after something specific and want to talk to our team directly.
- 2nd
We match you to the right role
When we find a role that fits, we reach back out to you before anything else moves forward.
- 3rd
We submit you and set up the interview
Your application goes to the hiring team. Once they give the green light, we set up your interview.
- 4th
You're in — we handle the paperwork
Once the interview process wraps up, we take care of the documentation and you start your new role.
0thlayer.com/jobs
0Halide Labs
BoardSaved12Applications3ForumActivity
ID
Ines Duarte
reviewer
214 open roles · updated 12m ago
inference, CUDA, remote⌘K
stackPyTorchphasescaleremoteEU / US6 of 214
Role CompensationReviewer
Inference Engineer, vLLM & CUDA
signedHalide Labs · Remote (EU / US)
PyTorchTritonCUDAvLLM
$180k–$240k
8–64 GPUs
ID
Ines Duarte
Halide Labs
Training Infrastructure Engineer
Cadence-9 · London · Hybrid
JAXRay
£145k–£190k
>1k accelerators
MO
Marcus Oyelaran
Cadence-9
Evals Lead, Post-training
signedNorthrail · Remote (US)
PythonPyTorchWeights & Biases
$200k–$265k
Single-node
WT
Wei Tan
Northrail
RL Engineer, Manipulation
Verge Robotics · Zürich
PyTorchMuJoCo
€155k–€205k
Multi-node
SM
Sofia Marchetti
Verge Robotics
ML Data Engineer
Trellis Health · Remote (US) · Boston
SparkdbtAirflowDuckDB
$160k–$205k
Petabyte-scale
DO
Daniel Okonkwo
Trellis Health
Applied ML Engineer, Ranking
Kestrel · Remote (EU)
PyTorchFeast
€120k–€158k
Single-node
AL
Anna Lindqvist
Kestrel
esc
shortlist signed
Inference Engineer, vLLM & CUDA
Halide Labs · Remote (EU / US)
- $180k–$240k
- 8–64 GPUs
- 213
- 7 of 213
ID
Ines Duarte
0thlayer.com/jobs/inference-engineer-vllm-cuda
openPosted 2d ago · 213 applicants
Inference Engineer, vLLM & CUDA
Halide Labs · Remote (EU / US) · Full-time
$180k–$240k
8–64 GPUs
Inference & serving
Priya Raman · Director of Inference
PyTorchTritonCUDAvLLM
You would own the serving path for our 8B and 70B models end to end — the scheduler, the KV-cache, the CUDA kernels underneath it, and the eval harness that tells us when a change made things worse. That path serves about 40 million requests a week across 8 to 64 GPUs, and its p99 is the number the whole company watches on a Monday.
The first thing you would touch is the eviction policy. We already know it is wrong; there is a thread about it on the forum and a graph nobody likes. We would rather hire the person who argues with us in that thread than the person who agrees with the job post.
- You have profiled a real inference server under load and can say what the bottleneck turned out to be.
- CUDA or Triton at the level of reading a kernel and knowing why it is slow — not at the level of having heard of them.
- You have shipped something with an eval harness attached to it, rather than a demo.
- Multi-node is a plus, not a requirement. Say so if you do not have it; the reviewer would rather know.
0thlayer.com/forum/kv-cache-eviction-p99
reliabilityperformance41 replies · last 20m ago
How fast we catch problems before they cost you
ID12 marked useful
Ines Duarte
Member of Technical Staff, Halide Labs
We were evicting the least-recently-used block in the pool, which sounds like the neutral choice and is not. The blocks that go longest without a touch belong to the requests that are still decoding — the long ones, the ones with the most prefill already sunk into them. So the policy quietly targets exactly the work you least want to throw away.
Mean latency did not move. p99 tripled inside forty minutes. Every eviction there costs a full prefill replay, and that replay queues behind the arrivals that caused the eviction in the first place. Graph below; the marker is where we noticed.
cache/policy.py
1# LRU across the whole pool. Looks fair. Is not.2def evict(self, need: int) -> None:3 freed = 04 for block in sorted(self.blocks, key=lambda b: b.last_used):5 if block.pinned:6 continue7 freed += self.release(block) # 16 tokens per block8 if freed >= need:9 return10 raise OutOfCache("pool exhausted")Rollout at −13m. Mean held. The tail did not.
sorted by useful marks
MO
Marcus Oyelaran
Staff Engineer, Training Infrastructure, Cadence-9412 useful · 2h agoLRU on a KV-cache is a scheduling policy wearing a memory policy’s clothes. You are not choosing which bytes are cold, you are choosing which request to punish, and you picked the one with the deepest sunk cost. The tail is the replay, not the miss.
8 marked usefulWT
Wei Tan
Research Engineer, Northrail197 useful · 1h agoBefore you rewrite the policy, plot evictions-per-request against sequence length. We had this exact shape and it turned out 4% of requests were absorbing 60% of the evictions. Once that was on screen the fix was an admission check at the front door, not a cleverer rule at the back.
SM
Sofia Marchetti
Principal ML Engineer, Verge Robotics341 useful · 20m agoSeconding the replay theory. Cheap mitigation while you rewrite: pin the blocks of any request that is already past its p50 decode length. Costs a few percent of the pool. Our p99 was back under 210ms within a day and stayed there.
0thlayer.com/companies/shortlists/inference-engineer-vllm-cuda
Inference Engineer, vLLM & CUDA
Halide Labs · Remote (EU / US) · closes in 4d
signedsigned 1d ago
7 of 213 readread in full · none skimmed
01Candidate 1 of 7readnote signed
02Candidate 2 of 7readnote signed
03Candidate 3 of 7readnote signed
04Candidate 4 of 7shortlistedreviewed by Ines Duarte · 1d ago
Doesn't just build demos — ships systems that hold up in production. Found a reliability bug in our pipeline our own team had missed for weeks.
Hasn't worked at large scale yet. Everything here was built for one team, not thousands of users.
Ask how they'd handle it if traffic tripled overnight.
Ines Duarte
Member of Technical Staff · Halide Labs
1d ago
Reviewer is accountable for having read all 213 in full.
05Candidate 5 of 7readnote signed
06Candidate 6 of 7readnote signed
07Candidate 7 of 7readnote signed
I've been working with them for a few years and they've supported my brand a lot — in tech operations, support, and digital activities. They've been our long-term support partner.
The forum
Practitioners worldwide, building this together.
We're not live yet — we're building the forum into a real home for AI practitioners everywhere to think out loud, and for hiring teams to see how they think. Join the waitlist and help shape it from day one.
0thlayer.com/forum/kv-cache-eviction-p99
reliabilityperformance41 replies · last 20m ago
How fast we catch problems before they cost you
ID12 marked useful
Ines Duarte
Member of Technical Staff, Halide Labs
We were evicting the least-recently-used block in the pool, which sounds like the neutral choice and is not. The blocks that go longest without a touch belong to the requests that are still decoding — the long ones, the ones with the most prefill already sunk into them. So the policy quietly targets exactly the work you least want to throw away.
Mean latency did not move. p99 tripled inside forty minutes. Every eviction there costs a full prefill replay, and that replay queues behind the arrivals that caused the eviction in the first place. Graph below; the marker is where we noticed.
cache/policy.py
1# LRU across the whole pool. Looks fair. Is not.2def evict(self, need: int) -> None:3 freed = 04 for block in sorted(self.blocks, key=lambda b: b.last_used):5 if block.pinned:6 continue7 freed += self.release(block) # 16 tokens per block8 if freed >= need:9 return10 raise OutOfCache("pool exhausted")Rollout at −13m. Mean held. The tail did not.
sorted by useful marks
MO
Marcus Oyelaran
Staff Engineer, Training Infrastructure, Cadence-9412 useful · 2h agoLRU on a KV-cache is a scheduling policy wearing a memory policy’s clothes. You are not choosing which bytes are cold, you are choosing which request to punish, and you picked the one with the deepest sunk cost. The tail is the replay, not the miss.
8 marked usefulWT
Wei Tan
Research Engineer, Northrail197 useful · 1h agoBefore you rewrite the policy, plot evictions-per-request against sequence length. We had this exact shape and it turned out 4% of requests were absorbing 60% of the evictions. Once that was on screen the fix was an admission check at the front door, not a cleverer rule at the back.
SM
Sofia Marchetti
Principal ML Engineer, Verge Robotics341 useful · 20m agoSeconding the replay theory. Cheap mitigation while you rewrite: pin the blocks of any request that is already past its p50 decode length. Costs a few percent of the pool. Our p99 was back under 210ms within a day and stayed there.
Threads, not takes
Long posts with code, plots, and numbers — not hot takes. Threaded replies, ordered by time, nothing boosted for engagement. No feed, ever.
Reputation you earned
Reputation comes from people in your field marking an answer useful — not from posting volume. It'll be on your profile, visible to every hiring team on the platform.
Attach a thread to an application
Point at something you've written and say “start here.” It becomes part of how a hiring team gets to know you — not just your résumé.
Open to practitioners everywhere
Wherever you're building AI — inside a lab, a startup, or on your own — this is a place to think out loud with people doing the same work.
We needed engineers who'd actually shipped AI systems before, not just interviewed well. 0th Layer matched us with people who could hit the ground running on our internal AI and automation work — the vetting alone saved us months of bad interviews.
Two sides, one rule
Both sides work with the same team.
Candidates get matched by people who understand the work. Companies get a hiring partner, not a database. Same team, same process, both directions.
Get matched, not filtered.
- A simple profile — not a résumé rewrite
- We match you to roles that actually fit
- Real interviews with real people
- Straight to the hiring team once you're a fit
- No recruiter spam. We don't have your inbox to sell.
For candidates
Engineers, researchers, infra
Get a hiring partner, not a database.
- A call to actually understand the role
- AI ranks every application, a real expert reviews it
- Candidates matched to your vision, not just keywords
- We run interviews built around what we learned
- We stay involved after you sign
For companies
Founders, ML leads, hiring managers
What's on the board
Narrow on purpose. Every role here is an ML role.
We turn down more listings than we run. If it's “AI-adjacent,” it's somewhere else.
Hiring for AI used to mean wading through hundreds of resumes hoping someone was real. Through 0th Layer we found people who'd genuinely built this stuff before, and they've been building out our internal AI and automation ever since.
We hired the engineers who now run our internal AI tooling and automation through 0th Layer. Every one of them had actually built the kind of systems we needed — that's rare, and it's why we keep coming back to hire more.
Where the line is
What we don't do.
Short list. Worth reading before you sign up.
No AI-only decisions.
AI ranks and identifies fit across every application, but no model has the final say. A real expert reviews every shortlist before it reaches you.
No selling your profile.
We don't sell, license, or “enrich” your data into sourcing tools. A company sees your application when you apply to that company. That's the whole distribution list.
No profile you didn't choose to make public.
Forum posts are public — that's the point of a forum. Your applications, your links, and the fact that you're looking are not. Not to your employer. Not to anyone.
No lock-in.
Delete the account and it's gone, threads included if you want them gone. Export first if you'd rather keep it.
FAQs
Questions about who reads what, what it costs, and what happens to your data. If yours isn't here, email us at hire@0thlayer.com.
Our team does — people who understand AI and ML work, not a resume database. AI helps us rank and organize every application so nothing gets lost in volume, but a person reviews every one before we get back to you.
Yes, and it's the point — AI is what lets us actually understand every application instead of skimming résumés. It reads and ranks candidates against the role. But no decision ships on AI's judgement alone: a real person on our team reviews everyone it surfaces before you ever hear from us.
No. The forum isn't live yet, so it isn't part of applying at all right now. Once it launches, it'll be a way to show your thinking alongside your profile — never a requirement.
You'll hear from us either way — no ghosting. If nothing's a fit right now, we'll tell you, and your profile stays with us for the next role that might be.
Those are distribution. Nobody is responsible for reading the other end. Here, our team reviews every application and follows up personally, whether or not you're a fit. That's the difference.
AI and ML work only: research, training and inference infrastructure, data, evals, RL, applied ML, MLOps, safety, robotics, and the adjacent systems work. We turn down general software roles and anything where “AI” is in the title but not in the job.
Applications that come in through the board, plus our own network — we actively search for people who match what you've told us about the role, not just whoever happened to apply.
Free for candidates — creating a profile, applying, all of it, permanently. Companies pay per role. Talk to us and we'll tell you the number.
One application. One person. One answer.
Free for candidates, permanently. Companies pay per role. No sourcing spam in either direction.