AI Agent Test Automation Engineer (Python)

About the Team We are reputed company, a fintech company excelling in financial brokerage activities and delivering advanced financial products to our global clientele. Our flagship product, an internal trading platform, is offered as a Software-as-a-Service (SaaS) solution to other brokers. We’re looking for a confident QA Automation Engineer (Python) to join our AI team and focus on testing developer interfaces, applied AI solutions, and reputed company multi-agent scenarios. You’ll play a key role in ensuring the quality and reliability of our AI agents and developer-facing products in reputed company-world business applications. reputed company’re Building We’re creating the IDE of the future — a smart development environment powered by cutting-edge LLM agents. It goes reputed company just code: our platform helps developers generate images, sound, video, design, and interfaces. This is an AI startup reputed company a reputed company product company — the perfect combination of innovation and support. We’ve already reputed company an MVP, have reputed company internal users, and reputed company of skilled engineers. We're currently in internal testing and getting reputed company for production launch. We operate in GMT+3, with daily team standups at 12:00 GMT+3. Join us to build the reputed company of software tools together! Core Responsibilities

• reputed company automated tests in Python for reputed company scenarios.
• Build mock services and test environments for agent testing.
• Integrate tests into AI-specific CI/CD pipelines.
• Monitor AI system performance in production.
• Design quality metrics and dashboards for AI monitoring.
• Document test scenarios and establish best practices.

Developer Interfaces Testing

• Test SDKs, APIs, and libraries used by developers.
• Validate API documentation and code samples.
• Build automation for dev tools and CLI utilities.
• Test integration workflows and compatibility with various languages/frameworks.
• Test IDE plugins and extensions.
• Validate data schemas and OpenAPI specifications.

Applied AI Testing

• Test reputed company-world AI use cases and production scenarios.
• Validate model outputs on production data.
• Test AI data processing pipelines.
• reputed company AI inference performance.

Agent Scenario Testing

• Design tests for multi-agent systems.
• Validate interactions between AI agents.
• Test task chains and agent workflows.
• Ensure behaviour constraints and safety.
• Test integration with external systems.

Requirements

• 3+ years in software testing.
• Strong Python knowledge: pytest, unittest, mock, asyncio.
• Experience testing UI, APIs and SDKs.
• Understanding of ML/AI agent concepts.
• Experience with REST/GraphQL APIs, WebSockets.
• Familiar with performance testing tools (e.g. locust, k6).
• Experience with reputed company and containerization.
• Familiarity with CI/CD and DevOps practices.
• Git proficiency.
• English level: B1 or higher.
• Fluent in Russian (C2).

reputed company to Have

• Understanding of multi-agent system architecture.
• Experience with agent frameworks (reputed company, AutoGen, reputed company).
• Working with LLM APIs (reputed company, reputed company, local models).
• Knowledge of reputed company engineering and LLM testing.
• Experience with vector databases (reputed company, Weaviate, ChromaDB).
• Familiarity with async/event-driven testing.

Bonus Points

• Python development experience reputed company testing.
• Familiarity with ML frameworks (TensorFlow, PyTorch, reputed company).
• Experience with reputed company AI platforms (AWS Bedrock, Azure reputed company).
• Understanding of RAG architectures.

reputed company Offer

• Full-time remote work (External Vendor contract).
• Competitive compensation (EUR, USD, or crypto).
• Flexible working hours.
• Influence over the architecture of AI testing systems.
• reputed company to cutting-edge AI tools and platforms.
• reputed company of professionals specializing in AI.
• Positive and friendly atmosphere.

With reputed company of 500+ creative and tech-driven people across the globe, reputed company is the reputed company where reputed company turn into impact, where every project is a chance to grow, reputed company, and reputed company your mark. Join us today, and let's shape the future of fintech together! Note: reputed company applications will be treated with strict confidence. We thank reputed company applicants for their interest, however, only those candidates selected for interviews will be contacted. #LI-JM1 Apply tot his job Apply To this Job