Senior Databricks Engineer

• Architecture & Development: Design and implement scalable data pipelines using Apache Spark (PySpark) on Databricks. Build solutions, not just run scripts.
• Optimization: Analyze and optimize complex Spark jobs for performance and cost (partitioning, z-ordering, Photon engine utilization).
• Modernization: Migrate legacy data warehouses to the Lakehouse architecture using Delta Lake.
• Standards: Enforce high code quality standards through rigorous code reviews and CI/CD implementation (Databricks Asset Bundles).
• Mentorship: Share knowledge with the team and act as a technical advisor for clients.

Key requirements:

• Senior Experience: 5+ years in Data Engineering with a strong focus on distributed systems.
• Databricks Expertise: Minimum 3 years of hands-on experience with the Databricks platform. Understand the internals of Delta Lake.
• Cloud Native: Production experience with Azure (ADLS Gen2, ADF) or AWS (S3, Glue).
• Coding: Advanced proficiency in Python and SQL.
• Modern Governance: Experience with Unity Catalog implementation.
• Communication: Fluent English (C1+) is mandatory for direct client collaboration.

Nice to have:

• Experience building data applications using Databricks Apps.
• Familiarity with Delta Live Tables (DLT).
• Databricks Certifications (Data Engineer Professional).
• Experience with dbt.