Experience and Skills
- Senior-level experience building and operating production AI systems, including tool-using, multi-step LLM agents.
- Production experience designing evaluation sets, metrics, quality thresholds, go-live gates, and regression detection across model or prompt changes.
- Strong applied statistics skills, including review sample sizing, confidence bounds, and error-rate estimation from samples.
- Experience designing safety controls enforced in code, including approvals, deterministic checks, and kill switches.
- Strong Python or TypeScript skills, including testing, code review, and development of maintainable services.
- Ability to write clear design notes and document technical decisions before implementation.
- Hands-on experience with Agent Orchestration, LLM Evaluation, Evaluation Metrics, Software Testing, Safety Engineering, Runtime Governance, Approval Workflows, Model Observability, Data Provenance, Model Provider Abstraction.
Skills
- LLM Agents
- Agent Orchestration
- LLM Evaluation
- Evaluation Metrics
- Applied Statistics
- Python
- TypeScript
- Software Testing
- Code Review
- Safety Engineering
- Runtime Governance
- Approval Workflows
- Model Observability
- Data Provenance
- Model Provider Abstraction
Funciones
- Build an evaluation framework with datasets, metrics, agreed thresholds, and a go-live gate for each agent capability.
- Re-evaluate agent capabilities after every model or instruction change and identify quality regressions.
- Build a governed agent runtime with accountable ownership for each action, human approval where required, and deterministic checks before irreversible actions.
- Implement rule-level or campaign-level approval for high-volume actions, including statistically sized review samples before release.
- Configure automatic suspension when quality thresholds are not met.
- Design review and evaluation flows for AI-generated content sent externally.
- Record every model call, including the model, instruction version, data provenance, and quality checks.
- Maintain platform independence from any single model provider.
- Convert written governance rules, ontologies, and acceptance scenarios into executable validation with the Principal Platform Engineer.
- Collaborate with cloud engineering on runtime infrastructure, isolation, and model-provider connectivity, and with product stakeholders on capability requirements, datasets, and thresholds.
Deseable
- Knowledge Graphs
- Ontologies
- LinkML
- SHACL
- OWL
- RDF
- Human-in-the-Loop Systems
- Retrieval-Augmented Generation
- Access Control
- Data Classification
- Financial Services
- Privacy
- Multi-Provider LLM Integration
Beneficios
We provide the opportunity to participate in impactful and technically rigorous industrial data projects that drive innovation and professional growth. Our work environment emphasizes technical excellence, collaboration, and continuous innovation.
Niuro supports a 100% remote work model, allowing flexibility in work location globally. We invest in career development through ongoing training programs and leadership opportunities, ensuring continuous growth and success.
Upon successful completion of the initial contract, there is potential for long-term collaboration and stable, full-time employment, reflecting our long-term commitment to our team members.
Joining Niuro means becoming part of a global community dedicated to technological excellence and benefiting from a strong administrative support infrastructure that enables you to focus on impactful work without distraction.