Deskripsi Pekerjaan
Join Capgemini's innovative team as an Azure Databricks Engineer, where you'll architect and implement cutting-edge data solutions on Microsoft's cloud platform. This pivotal role requires you to design, build, and maintain scalable big data ecosystems that drive business intelligence and analytics. You'll collaborate with cross-functional teams to translate complex data requirements into robust cloud-native architectures while ensuring optimal performance, security, and compliance. As a key player in our digital transformation initiatives, you'll leverage Azure Databricks to unlock actionable insights from massive datasets, enabling data-driven decision-making across global enterprise clients. Your expertise will directly impact our clients' ability to harness the power of artificial intelligence and machine learning in their operations.
Tanggung Jawab
- Design and implement scalable data pipelines using Azure Databricks, Delta Lake, and Spark
- Optimize data processing workflows for performance, cost-efficiency, and reliability
- Develop and maintain ETL/ELT processes for large-scale data integration
- Collaborate with data scientists to support ML model deployment and lifecycle management
- Ensure data governance, security protocols, and regulatory compliance (GDPR, etc.)
- Monitor system health and troubleshoot performance issues proactively
- Document architecture, workflows, and best practices for knowledge sharing
Kualifikasi
- Bachelor's degree in Computer Science, Engineering, or related field
- 3+ years of experience in Azure Databricks and big data engineering
- Expertise in Spark, Delta Lake, and Databricks SQL
- Strong proficiency in Python, Scala, and SQL for data processing
- Experience with Azure services (Data Factory, Synapse, Blob Storage)
- Knowledge of CI/CD practices for data pipelines
- Certifications (Azure DP-203, Databricks Certified Developer) preferred