Note: The job is a remote job and is open to candidates in USA.
Company Overview
iMerit Technology is focused on designing human-in-the-loop data systems for AI labs and enterprise ML teams. The role involves solution design, data quality evaluation, and hands-on execution of workflows, requiring strong technical skills and experience in AI/ML data systems.
Responsibilities
- Translate ambiguous GenAI use cases into scoped, buildable data workflows
- Design end-to-end workflows: task schemas, guidelines, quality frameworks, sampling strategies, delivery models
- Lead technical discovery; surface constraints in data availability, task design, talent requirements, and tooling
- Produce clear proposals that align scope, pricing, and delivery assumptions
- Build quality frameworks: rubrics, acceptance criteria, audit models
- Design evaluation approaches for subjective, high-ambiguity tasks (reasoning, multimodal, safety/policy)
- Contribute to red teaming and adversarial testing tied to deployment risks
- Analyze datasets; validate outputs; debug workflows
- Prototype task designs and evaluation pipelines (Python, Ango Hub, or client tooling)
- Partner with Delivery to stand up workflows; stay engaged through pilot and early production
- Act as technical counterpart to ML researchers, product leads, and data teams
- Drive alignment across Sales, Solutions, Product, and Delivery
- Contribute to reusable patterns for workflows, pricing, and quality frameworks
Skills
- 5–10 years in AI/ML data systems or human-in-the-loop (or advanced degree + 3–5 years)
- Experience across at least two of: annotation, data generation/collection, evaluation, RLHF, red teaming
- Client-facing experience in a technical role
- Strong Python; can analyze datasets, prototype workflows, and debug issues
- Comfortable with unstructured and multimodal data (text, image, PDF, audio, video)
- Working understanding of ML workflows, especially training data and evaluation
- Experience with GenAI data programs (SFT, preference data, evals, red teaming)
- Strong grasp of quality frameworks: rubrics, scoring, sampling, audit, adjudication
- Ability to reason about ambiguity, subjectivity, and model behavior
- Strong decision-making under uncertainty; can balance quality, cost, and speed
- Comfortable operating hands-on in ambiguous, early-stage workflows
- Clear, concise communicator with technical and non-technical audiences
- Effective working across cross-functional, global teams
- Research background (PhD or equivalent) in linguistics, STEM, or social science
- Experience working directly with frontier AI labs
- Experience with multimodal or agentic/interactive systems
About iMerit
iMerit is a leading AI data solutions company providing high-quality data across computer vision, natural language processing, and content services that power machine learning and artificial intelligence applications for large enterprises. It was founded in 2012, and is headquartered in San Jose, California, USA, with a workforce of 1001-5000 employees. Its website is http://imerit.net/.
H1B Sponsorship
iMerit Technology has a track record of offering H1B sponsorships, with 1 in 2022. Please note that this does not guarantee sponsorship for this specific role.