Data Engineer

Role: As an integral part of our team, you will drive the day to day operation, which in turn will influence how our product evolves and how customers interact with our product. You will be joining a small team of Data Engineers, ensuring that real-time data is available to our analysts through the design and development of robust data pipelines.

You will: 

• Manage and scale BI data infrastructure by developing and maintaining our data pipelines from ingestion through to storage 
• Model data using solid data lake/warehouse concepts and optimize performance (partitioning, file formats, performance tuning) 
• As a central figure of a small team, you will champion standards, processes and best practices in your area 
• implement least-privilege IAM, encryption, data classification, PII handling, and auditability and contribute to data governance policies 
• Conduct data analysis experiments, providing bespoke solutions to our stakeholders giving them the information they need to succeed 
• collaborate with analysts and domain stakeholders to translate requirements into robust data solutions, review code
• We want to hear your ideas! You will bring all of your previous experiences and thoughts to the table and help us bring in new tools, topics and methodologies

Project duration: 3 months

Requirements: 

• 5+ years working experience as a Data Engineer. 
• You are an expert in designing and developing robust Data Pipelines using technologies such as Apache Spark, Hadoop ecosystem with a deep understanding of data processing 
• You have a strong background in cloud infrastructure (especially AWS) and hands-on experience of data lakes and modern data warehouses, including familiarity with orchestration, security, and network 
• You have proven programming experience with Scala and Python and you are also happy to review code 
• You have working knowledge of DBT (models, tests, documentation) and a general understanding of Tableau or similar BI tools 
• You work with relational databases, data warehouses and advanced SQL queries 
• You have experience with CI/CD & bash scripting and deployment/release best practices 
• You have practical experience with Docker and working knowledge of Kubernetes 
• You collaborate with analysts and other stakeholders to understand data requirements and provide efficient data solutions 
• You contribute to the development and maintenance of data governance and security practices 
• Excellent written and verbal communication skills in English and have a clear and adaptive communication style 
• You are solution-oriented and have excellent attention to detail