Deskripsi Pekerjaan
Join our dynamic team at Clicktek Consulting Inc as a Data Engineer and play a pivotal role in shaping our data infrastructure. We are seeking a skilled professional to design, build, and maintain robust batch and streaming data pipelines using cutting-edge Microsoft Fabric technologies including Data Factory, Dataflows Gen2, and Spark notebooks. This is an exciting opportunity to work with large-scale data ecosystems and make a significant impact on our organization's data-driven decision-making processes.
In this role, you will collaborate closely with data scientists, analysts, and business stakeholders to understand data requirements and translate them into efficient, scalable data solutions. You will be responsible for ensuring data quality, optimizing data workflows, and implementing best practices in data engineering. If you are passionate about data infrastructure, automation, and creating value through data, we encourage you to apply and become an integral part of our innovative team.
Tanggung Jawab
- Design, build, and maintain batch and streaming data pipelines using Microsoft Fabric technologies
- Develop and optimize data transformations using Dataflows Gen2 and Spark notebooks
- Ensure data quality, consistency, and reliability across all data systems
- Collaborate with cross-functional teams to understand data requirements and deliver solutions
- Implement data monitoring, alerting, and logging systems
- Automate data processes and workflows to improve efficiency
- Stay updated with emerging data technologies and best practices
- Document data architecture, processes, and procedures
Kualifikasi
- Bachelor's degree in Computer Science, Information Technology, or related field
- 3+ years of experience in data engineering or similar roles
- Strong proficiency in Microsoft Fabric, Data Factory, and Dataflows Gen2
- Experience with Spark and distributed computing frameworks
- Familiarity with SQL and NoSQL databases
- Knowledge of data warehousing concepts and best practices
- Experience with cloud platforms (Azure preferred)
- Strong problem-solving and analytical skills