VibeHire

Assess engineers through real-work trials.

VibeHire helps companies evaluate engineers by having them solve realistic job-relevant tasks using modern coding and AI tools.

See how it works
What VibeHire is

VibeHire moves real work into the shortlisting stage. Candidates receive the task as a ticket and work in a configured engineering environment with the tools and support they would expect on the job. We capture both the outcome and how they got there, then build a report with hiring bands for each metric and the evidence behind them.

CompanyCandidateVibeHire
  1. Share the roleRole requirements and candidate pool
  2. Build and inviteWe configure the work trial with you and invite the candidates.
  3. Work trialCandidates work independently in a configured engineering environment.
  4. EvaluatePerformance is evaluated across the signals that matter for the role.
  5. ReviewHiring bands, evidence and interview probes for every candidate.
EVALUATION SIGNALS10+Agentic and deterministic signals, evaluated on telemetry evidence and session variables.
ENGINEERING SETUPS5+From VS Code and Chrome workflows to Android Studio, Spark and Hadoop.
SESSION CAPTURE100%The complete work trial is recorded, from tool usage and decisions to validation and final output.

For companies

A trial that looks like the job, built with you.

app.vibehirelabs.com
←
Role briefBackend Engineer — Payments · Acme Corp Private Ltd
Draft saved just nowRM
Tell us about the role
Job requirement plus the applicant sheet. We take it from there.
Role
Job requirement
CANDIDATES WHO APPLIED
Drop the applicant sheet here.xlsx or .csv · name, email, resume
XLSX
applicants_backend_payments.xlsxUploading · 18 KB
✓ 24 candidates
NameEmailResume
Anita Krishnan[email protected]✓ linked
Rohan Mehta[email protected]✓ linked
Priya Sharma[email protected]✓ linked
Daniel Okafor[email protected]✓ linked
Mei-Lin Chen[email protected]✓ linked
Arjun Nair[email protected]✓ linked
Sofia Reyes[email protected]✓ linked
Assessment built by VibeHire · invites sent from VibeHireSubmit details →
←
Assessment buildBackend Engineer — Payments · Acme Corp Private Ltd
Drafting with Acme · round 2VH
Workspace profile and questions
Drafted from the job requirement, agreed with your team.
CODING & AI TOOLS · FROM THE PROFILE LIBRARY
VS Code
Chrome
Spark
Hadoop
Android Studio
Claude Code
QUESTION BANK · CURATED BY VIBEHIRE OPS
Rate limiter for the payments APIPAY-231 · Python · FastAPI · 90 min · $10.00 AI budget
✓ SelectedPreview
Idempotent retries for webhook deliveryPAY-214 · Python · FastAPI · 90 min · $10.00 AI budget
✓ SelectedPreview
Ledger reconciliation jobPAY-198 · Python · Postgres · 60 min · $6.00 AI budget
✓ SelectedPreview
90 min · $10.00 AI budget · 1 questionReview and Publish Assessment →
Question bank / PAY-231Add to assessment✕
TICKET PANEL · CANDIDATE VIEW
✓PAY-231 · Sprint 14
Rate limiter for the payments API
MEDIUMPythonFintech90 min · $10.00 budget
SUMMARY.MD

Implement a sliding-window RateLimiter(max_requests, window_seconds) with an allow(client_id) method. O(1) amortized per call; only allowed requests consume quota.

ACCEPTANCE_CRITERIA.MD
✓Amortized O(1) per allow() call
✓Rejected calls never consume quota
✓Per-client isolation under concurrency
TEST SUITE
6 PUBLIC — VISIBLE TO CANDIDATE
test_allows_under_limitUNIT
Requests under max_requests are allowed
PAGE 1 / 6
4 PRIVATE — SCORED ON SUBMIT
▶BOUNDARY2
▶CONCURRENCY1
▶PERFORMANCE1
←
Run evaluationConfigure agents, deterministic metrics and weightage before the run
Reviewed with AcmeVH
1Agents 2Deterministic metrics 3Weights & review
AI agent metrics4 / 4 ENABLED+ New agent
✦
Functional ownershipDid the candidate own the solution shape, or accept whatever the assistant producedclaude-sonnet-4-5 · temp 0.2 · 24 turns
Included
✦
AI orchestration qualityPrompt specificity, verification of AI output, recovery from bad suggestionsclaude-sonnet-4-5 · temp 0.3 · 30 turns
Included
✦
Implementation maintainabilityStructure, naming, error handling and module boundaries in the final workspaceclaude-sonnet-4-5 · temp 0.2 · 24 turns
Included
✦
Validation ownershipSelf-authored tests, inspection of generated assertions, boundary coverageclaude-sonnet-4-5 · temp 0.2 · 18 turns
Included
Deterministic metrics3 / 3 ENABLED+ New metric
Private test successABSOLUTE
private_test_passIncluded
Public test successABSOLUTE
public_test_passIncluded
Outcome efficiencyRELATIVE
— no formula yet —Included
Computed from session variables — no agent involved. Scores map to band votes via the cut-offs in each metric.
WeightageTOTAL 0%Distribute equally
Functional ownership
0%
AI orchestration quality
0%
Implementation maintainability
0%
Validation ownership
0%
Private test success
0%
Public test success
0%
Outcome efficiency
0%
Run evaluation
✦
AI orchestration qualityAGENT · JUDGED FROM SESSION EVIDENCE
✕
AGENT NAME
AI orchestration quality
SYSTEM PROMPT
You review the full transcript between the candidate and the AI editor. Assess prompt specificity, verification of AI output, and recovery from bad suggestions. Distinguish delegation with judgement from blind acceptance. Citing transcript turns, vote one verdict band.
BAND DEFINITIONS
STRONG HIRE
Directs the AI with precision and verifies every output
HIRE
Good prompts and mostly verified outputs
LEAN HIRE
Uses AI effectively but accepts some output unverified
LEAN NO HIRE
Vague prompting; frequent blind acceptance of suggestions
NO HIRE
Fully dependent on AI with no judgement or verification
DuplicateDone
ƒx
Outcome efficiencyDETERMINISTIC · COMPUTED FROM SESSION VARIABLES
✕
FORMULA · SCORES 0–100
30+70×quality_lift×cost_score÷100
✓ Formula valid
VARIABLES — CLICK TO INSERT
quality_liftPrivate-test lift after the baseline
cost_scoreClamped inverse of spend vs cohort median
budget_used_percentage% of the allotted AI budget consumed
OPERATORS
+−×÷()100
BAND GRADING
AbsoluteRelative
STRONG HIREtop 10%
HIRE10 – 35%
LEAN HIRE35 – 60%
Done
←
ResultsBackend Engineer — Payments · 16 candidates evaluated
Reports releasedRM
RANKEDAll badges ▾
1Priya SharmaSTRONG HIRE
2Daniel OkaforSTRONG HIRE
3Mei-Lin ChenHIRE
4Arjun NairHIRE
5Sofia ReyesHIRE
6Tom WhitfieldLEAN HIRE
7Kavya ReddyLEAN HIRE
8Imran ShaikhLEAN HIRE
1–8 of 16‹  1 / 2  ›
PS
Priya SharmaRANKED 1ST OF 16
STRONG HIRE
ScorecardSWOTChatInterview probes
✦Functional ownershipWEIGHT 20%STRONG HIRE
EVIDENCE POINTERS
›Corrected an AI-proposed O(n) scan to a per-client deque before implementation.
›Stated the design constraint (O(1) amortized) in the first prompt before any code existed.
›Kept ownership of the final shape across both sessions without deferring to the assistant.
✦AI orchestration qualityWEIGHT 20%STRONG HIRE
✦Outcome efficiencyWEIGHT 10%LEAN NO HIRE
✦Implementation maintainabilityWEIGHT 15%LEAN HIRE
✦Validation ownershipWEIGHT 15%NO HIRE
ƒPrivate test successWEIGHT 15%HIRE
ƒPublic test successWEIGHT 5%STRONG HIRE
Strengths

Owned the rate-limiter design end to end, correcting an AI-suggested data structure that would have degraded to O(n). Prompts were specific, with acceptance criteria given before generation.

Weaknesses

Wrote her own edge-case tests only after the second failed private-test run; one boundary condition remained uncovered at submission.

Opportunities

Her spec-first prompting, then targeted diffs, would transfer well to larger codebases. A test-first habit would close the validation gap quickly.

Threats

Under time pressure she deferred to AI output without verification twice; with weaker test harnesses this could let regressions through.

Claude Code — Session 1 transcript14:12 – 14:20 · 6 messages · $6.21
PS
Priya Sharma

/plan the implementation of a sliding-window RateLimiter(max_requests, window_seconds) with an allow(client_id) method as per @README.md. Acceptance: O(1) amortized per call, and only allowed requests consume quota.

Claude Code

Generated rate_limiter.py using a per-client list of timestamps scanned on each call.

PS
Priya Sharma

The timestamp-list scan is O(n) per call. Switch to a per-client deque and pop expired entries from the left instead.

Claude Code

Refactored to collections.deque with popleft-based expiry — amortized O(1) per call. Only allowed requests are appended, so rejected calls never consume quota.

?Walk me through how you decided what to hand to the AI versus write yourself in the rate limiter.

Why ask: Confirms the functional-ownership signal — the design decisions in the transcript should be hers and reproducible without the assistant. · Functional ownership

?When the assistant proposed the timestamp-list approach, what made you push back?

Why ask: Checks the O(n)-to-deque correction was reasoned rather than pattern-matched; a strong answer references the complexity constraint she set up front. · AI orchestration quality

?Your own edge-case tests appeared after the second failed run — what would make you write them first?

Why ask: Targets the validation-ownership band; distinguishes a habit gap from a deliberate time-pressure tradeoff. · Validation ownership

For candidates

Work the way you actually work.

app.vibehirelabs.com/assessment/TASK-0001
AC
Backend Engineer AssessmentAcme Corp Private Ltd · Priya Sharma
Proctored
01:12:40remaining
‹
Question 1 of 3
›
AI BUDGET$8.62 / $10.00
▷ Run testsSaveSubmit
≡←→
rate-limiter
rate_limiter.py×
tests_user.py
M↓README.md
⋯
ProblemsOutputDebug ConsoleTerminalPorts
Claude Code⋯
Claude Code v2.1.270
Opus 5 (1M context) with high effort · API Usage Billing
~/workspace/rate-limiter
✻Thinking… (esc to interrupt)
⏺Write(rate_limiter.py)
⎿ Wrote 21 lines to rate_limiter.py
⏺Implemented RateLimiter with a per-client list of timestamps scanned on each allow() call: drop entries older than window_seconds, then compare the remaining count to max_requests.
✻Thinking… (esc to interrupt)
⏺Update(rate_limiter.py)
⎿ Updated rate_limiter.py with 2 additions and 1 removal
14- hits = [ts for ts in hits if now - ts < self.window]14+ while q and now - q[0] >= self.window:15+ q.popleft()
⏺Switched to collections.deque with popleft-based expiry — amortized O(1) per call. Only admitted requests are appended, so rejected calls never consume quota.
❯Try "fix lint errors"
⏵⏵ auto mode on (shift+tab to cycle)
main00Ln 20, Col 21Spaces: 4UTF-8Python 3.11.9
Payments API - Swagger UI×
+
localhost:8000/docs
PS
loading preview…
Payments API

FastAPI · rate limited per client · 100 requests / 60 s

POST/v1/chargesCreate a charge
GET/v1/charges/{id}Retrieve a charge
POST/v1/refundsRefund a charge
GET/v1/limits/{client_id}Remaining quota
ElementsConsoleSourcesNetworkPerformanceMemoryApplication»
1
No throttling
FilterAllFetch/XHRDocCSSJSFontImgMediaManifestWSWasmOther
NameStatusTypeInitiatorSizeTimeWaterfall
docs200documentOther3.1 kB24 ms
swagger-ui.css200stylesheetdocs(disk cache)6 ms
swagger-ui-bundle.js200scriptdocs(disk cache)11 ms
openapi.json200fetchswagger-ui-bundle.js:21.9 kB18 ms
charges201fetchswagger-ui-bundle.js:2412 B41 ms
charges429fetchswagger-ui-bundle.js:296 B9 ms
6 requests5.5 kB transferred63.4 kB resourcesFinish: 1.02 sDOMContentLoaded: 218 msLoad: 391 ms
Test resultsNot run yet
● Compiling and running tests…
✓Public tests6 / 6 passed
●Private tests3 / 4 passed
✓Your tests2 / 2 passed
Switch to browser preview
VS Code — open
Hide task panel
Open assistance chat
Focus mode — full-screen editor
Switch to dark mode
☾
Show
Contact us

Run your first work trial.

Tell us about the roles you are hiring for and we will set up the workspace with you.

Contact us
FAQ

Questions we get asked.

Anything else, contact us.

AI-generated resumes and isolated coding tests can screen out high-signal candidates. VibeHire sits at the shortlisting stage and gives every candidate a chance to prove their skills through real work.

VibeHire adapts the work trial to the experience level. Entry-level candidates build a small feature from scratch, junior engineers extend an existing codebase, and mid-to-senior engineers work through debugging or production-style incident scenarios.

Take-homes usually show you only the final output. VibeHire also shows the candidate's approach to the problem, their thinking and decision-making, so you can make more informed hiring decisions.

VibeHire pricing depends on the number of candidates and the assessment setup. Reach out to [email protected] and we’ll suggest a structure based on your hiring volume.