We are looking for a highly skilled Senior Test Engineer to ensure the quality, reliability, and performance of our Data Hubs and data processing pipelines in Azure & Databricks. In this role, you will design and implement automated test frameworks, validate large-scale data workflows, and collaborate with cross-functional teams to integrate testing into CI/CD pipelines.

• Design, develop, and maintain automated test frameworks for data processing and orchestration.
• Conduct regression, performance, and scalability testing for data ingestion and transformation pipelines.
• Implement data quality validation and anomaly detection for Azure-based data solutions.
• Collaborate with Data Engineers and DevOps teams to integrate testing into CI/CD pipelines.
• Ensure compliance with data security, governance, and privacy standards.
• Define test strategies, best practices, and documentation for Data & AI solutions.

• Strong expertise in Python, PySpark, SQL, and test automation frameworks.
• Experience with Azure Data Services (Databricks, Data Factory, Storage Accounts).
• Proficiency in testing big data pipelines, distributed systems, and ETL workflows.
• Hands-on experience with CI/CD tools (Azure DevOps, Jenkins) and test automation.
• Strong understanding of data governance, quality frameworks, and monitoring tools.
• Excellent problem-solving skills and ability to work in Agile/Scrum environments.

• Experience with ML model validation and AI pipeline testing.
• Familiarity with data lineage tracking and metadata management tools.