Praca evaluation Warszawa. Znaleziono 351 ofert pracy.

Agentic AI MCP & A2A Specialist

PORT BLUE SKY Sp. z o.o. Sp.k. PL, 52.21519, 21.2453, warszawa, mazowieckie, Warszawa 14 dni temu

... integration, model abstraction, telemetry, evaluation, and production operations. This role ... tests, observability-driven evaluation, escalation paths, human-in-the- ... servers, workflow engines, and evaluation observability stacks. Work across engineering ...

Senior AI Engineer with .NET

YOUR ITEAMS sp. z o.o. PL, 52.21519, 21.2453, warszawa, mazowieckie, Warszawa, mazowieckie 17 dni temu

... AI Engineer to own the evaluation and quality layer of a ... services, RAG pipelines, agents, and evaluation tooling. Strong, demonstrable experience integrating ... -technical stakeholders on what model evaluation means in practice. Fluent English ( ...

Senior AI Engineer

YOUR ITEAMS sp. z o.o. PL, 52.21519, 21.2453, warszawa, mazowieckie, Warszawa, mazowieckie 26 dni temu

... Engineer responsible for ensuring the evaluation and quality assurance of a ... services, RAG pipelines, agents and evaluation tooling. Proven track record integrating ... Engineer focused on quality and evaluation. Significant technical ownership and influence ...

QA Engineer

Andersen PL, 52.21519, 21.2453, warszawa, mazowieckie, Warszawa 11 dni temu

... adapted for AI workflows, including evaluation‑based testing, scenario testing, and ... issues with detailed repro steps, evaluation evidence, logs, and accuracy quality ... . Nice to have: - Exposure to evaluation frameworks (LangFuse evals, Ragas, TruLens, ...

AI Engineering Team Leader

XTB PL, 52.21519, 21.2453, warszawa, mazowieckie, Warszawa 12 dni temu

... , enterprise-grade systems where evaluation-driven development and full-stack ... services, data access layers, evaluation, observability, and production operations, Own ... for LLMOps, AgentOps, deployment, evaluation-driven development, prompt and tool ...

Senior Python Engineer (AI evaluations platform)

ACAISOFT POLAND Sp. z o.o. PL, 52.21519, 21.2453, warszawa, mazowieckie, Warszawa, mazowieckie 11 dni temu

... a leading provider of AI evaluation and optimization solutions, trusted by ... a leading provider of AI evaluation and optimization solutions, directly contributing ... that support large-scale agent evaluation and reinforcement learning experiments Build ...

Senior AI QA Engineer Automation & Manual AI-based Applications Testing

EPAM Systems PL, , , warszawa, mazowieckie, Warsaw 3 dni temu

... best practicesDesign and automate evaluation of Gen AI features — grounding, ... prompt instruction testing and evaluation of agentic workflowsProficiency in at ... frameworks, prompt engineering and evaluation metrics for LLM-based systemsDemonstrated ...

Senior Fullstack Engineer (AI evaluations platform) @ Acaisoft

Acaisoft PL, , , pl, PL, Remote, Warszawa 7 dni temu

... production-ready agentic systems (custom evaluation harnesses) 5 years building scalable, ... a leading provider of AI evaluation and optimization solutions, trusted by ... performance user interfaces for LLM evaluation workflows, advanced analytics, search, and ...

Senior AI Developer with LLM Orchestration Intelligent Knowledge Platform

DataArt PL, 52.21519, 21.2453, warszawa, mazowieckie, Warszawa 3 dni temu

... APIs, Docker, Kubernetes, ML evaluation frameworks, observability and tracing tools ... services Create and maintain evaluation frameworks including ground truth datasets ... graph concepts Experience implementing evaluation methodologies for AI systems including ...

Data Science Lead AI

Capgemini Polska PL, 52.21519, 21.2453, warszawa, mazowieckie, Warszawa 3 dni temu

... You will guide experimentation, evaluation, and refinement of AI models, ... establish experimentation pipelines and evaluation frameworks. Strong knowledge of LLMs, ... frameworks and AI evaluation tooling. Comfortable working cross‑functionally ...

Senior AI Quality Analyst

Svitla Systems PL, 52.21519, 21.2453, warszawa, mazowieckie, Warszawa 5 dni temu

... , designing and executing comprehensive evaluation strategies to identify model weaknesses, ... for model testing and evaluation. Conduct data annotation and validation ... environments to ensure reliable evaluation pipelines. Reporting & Insights: Analyze test ...

DevOps with Agentic Systems knowledge (LLM Agents, Tool Calling) @ Acaisoft Poland Sp. z o.o.

Acaisoft Poland Sp. z o.o. PL, , , pl, PL, Remote, Warszawa 7 dni temu

... a leading provider of AI evaluation and optimization solutions, trusted by ... learning (RL) environments and scalable evaluation systems that guide and shape ... through rigorous, real-world agent evaluation. Due to the client’s time ...

Senior Data Scientist @ Bayer

Bayer PL, 52.21519, 21.2453, warszawa, mazowieckie, Warszawa 7 dni temu

... forecasting experience is a plus. Evaluation focus: design and run offline online tests, rubric-based GenAI evaluation, safety checks, and error analysis; ... knowledge use cases, Establish rigorous evaluation: offline metrics, human-in-the- ...

Mid/Senior AI engineer with networking experience @ CodiLime

CodiLime PL, , , pl, PL, Remote, Warszawa 7 dni temu

... LLM-based reasoning LLM evaluation tooling: Experience with frameworks and ... to-end agent responses, Designing evaluation workflows combining deterministic checks, human ... using DeepEval or custom evaluation prompts, Refining prompts, tool descriptions, ...

Lead AI Engineer - Agentic Supply Chain Systems

PORT BLUE SKY sp. z o.o. sp.k. PL, 52.21519, 21.2453, warszawa, mazowieckie, Warszawa 7 dni temu

... orchestration, token efficiency, prompt security, evaluation, model upgrades, and long-term ... input and output. Knowledge of evaluation approaches for agentic applications, including ... and Bedrock Agent Core evaluation features. Experience with supply-chain ...

Data Science Lead (AI) @ Capgemini Polska Sp. z o.o.

Capgemini Polska Sp. z o.o. PL, , , pl, PL, Kraków, Katowice, Lublin, Opole, Wrocław, Poznań, Warszawa, Gdańsk 7 dni temu

... establish experimentation pipelines and evaluation frameworks. Strong knowledge of LLMs, ... frameworks and AI evaluation tooling. Comfortable working cross‑functionally ... You will guide experimentation, evaluation, and refinement of AI models, ...

Infra Software Engineer with Agentic AI experience @ Acaisoft Poland Sp. z o.o.

Acaisoft Poland Sp. z o.o. PL, , , pl, PL, Remote, Warszawa 7 dni temu

... production-ready agentic systems (custom evaluation harnesses) Strong hands-on experience ... learning (RL) environments and scalable evaluation systems that guide and shape ... through rigorous, real-world agent evaluation. Due to the client’s time ...

Site Reliability Engineer – AI/LLM & Infra @ Acaisoft Poland Sp. z o.o.

Acaisoft Poland Sp. z o.o. PL, , , pl, PL, Remote, Warszawa 7 dni temu

... a leading provider of AI evaluation and optimization solutions, trusted by ... learning (RL) environments and scalable evaluation systems that guide and shape ... through rigorous, real-world agent evaluation. Due to the client’s time ...

Site Reliability Engineer – AI/LLM & Infra

ACAISOFT POLAND Sp. z o.o. PL, 52.21519, 21.2453, warszawa, mazowieckie, Warszawa, mazowieckie 11 dni temu

... a leading provider of AI evaluation and optimization solutions, trusted by ... learning (RL) environments and scalable evaluation systems that guide and shape ... through rigorous, real-world agent evaluation. Due to the client’s time ...

DevOps Engineer with AI

ACAISOFT POLAND Sp. z o.o. PL, 52.21519, 21.2453, warszawa, mazowieckie, Warszawa, mazowieckie 11 dni temu

... a leading provider of AI evaluation and optimization solutions, trusted by ... learning (RL) environments and scalable evaluation systems that guide and shape ... through rigorous, real-world agent evaluation. Due to the client’s time ...