Senior Data Engineer

• Building a highly scalable and fault-tolerant data ingestion platform for millions of customers, collecting 5 billion clickstream events daily (up to 150k/sec) from all sites and mobile applications.
• Working with a hybrid solution using on-premise and Google Cloud Platform services like Spark, Kafka, Beam, BigQuery, Pubsub, or Dataflow.
• Implementing projects in the Data & AI area, including BigData processes, ML models, search engine, recommendations, and buyer profile.
• Collaborating within a team of over 150 experienced engineers organized into specialized teams.
• Using good practices such as clean code, code review, TDD, and CI/CD.
• Working in a hybrid model (4/1) with flexible start hours between 7:00 and 10:00 and 30 days of occasional remote work.

Key requirements:

• Programming in Scala, Java, or Python
• Strong understanding of distributed systems, data storage, and processing frameworks like dbt, Spark, or Apache Beam
• Knowledge of GCP (especially Dataflow and Composer) or other public cloud environments
• Use of good practices (clean code, code review, TDD, CI/CD)
• Efficient navigation within Unix/Linux systems
• Positive attitude and team-working skills
• Eagerness for personal development
• English at B2 level

Nice to have:

• Experience with Azure or AWS