DIXON JUSTIN
New York,NY
Summary
AI training and evaluation specialist with experience improving large language models through human feedback, rubric-based scoring, prompt rewriting, and data annotation. Skilled in evaluating model outputs for accuracy, safety, reasoning quality, and instruction-following. Strong productivity and quality in remote, US-aligned workflows.
Overview
6
6
years of professional experience
Work History
AI Training Contributor / LLM Evaluator
Outlier AI
New York, New York
01.2023 - 02.2026
Independent Contractor | Project-Based
- Evaluated LLM responses using strict rubrics for correctness, helpfulness, tone, and safety.
- Performed response ranking and preference labeling to improve model behavior (RLHF/HITL).
- Flagged hallucinations, policy violations, and unsafe outputs; wrote concise rationales.
- Rewrote prompts/responses to improve instruction adherence and reduce ambiguity.
Data Collection / Task Contributor
Atlas Capture
New York, New York
03.2021 - 12.2022
Independent Contributor
- Completed structured data collection and quality-controlled submissions per task requirements.
- Followed platform guidelines, privacy rules, and delivery standards.
Remote Support / Online Academic Services
Independent / Freelance (Remote)
New York, New York
06.2020 - 10.2021
- Provided structured, deadline-driven academic support and documentation for clients remotely.
- Managed communication, file organization, and delivery across time zones.
Education
Master of Science - Data Science
Columbia University
New York, NY05-2025
Bachelor of Science - Data Science
Columbia University
New York, NY12-2022
Skills
LLM / RLHF Tasks
- Response evaluation (helpfulness, factuality, tone, safety)
- Preference ranking (best response selection)
- Chain-of-thought quality checks (reasoning coherence without revealing private reasoning)
- Hallucination detection fact-check flagging
- Prompt rewriting instruction tightening
- Safety/policy labeling (toxicity, harassment, self-harm, illegal instructions)
Annotation / QA
- Text classification, tagging, categorization
- Dataset QA and consistency checks
- Edge-case detection error taxonomy reporting
- Calibration: aligning scores to rubrics/examples
- Spreadsheet tracking: Google Sheets, Excel
Remote Execution
- Asynchronous delivery, SLA adherence, documentation quality
- Clear written rationales and reviewer-style notes
Timeline
AI Training Contributor / LLM Evaluator
Outlier AI
01.2023 - 02.2026
Data Collection / Task Contributor
Atlas Capture
03.2021 - 12.2022
Remote Support / Online Academic Services
Independent / Freelance (Remote)
06.2020 - 10.2021
Master of Science - Data Science
Columbia University
Bachelor of Science - Data Science
Columbia University