You’ll be a key role in Data Protection Services team, an innovative team with many brand new services provided at enterprise level, to provide data protection related services. As a Senior Data Engineer, you will work with partners to design, build, configure and deploy technical solutions to obfuscate confidential PROD data using popular Azure tools: Azure Databricks, Azure SQL DB, ADF and Azure Blob/ADLS.
How You’ll Succeed
-
Work with partners to design, build, configure and deploy technical solutions to obfuscate confidential PROD data using popular Azure tools: Azure Databricks, Azure SQL DB, ADF and Azure Blob/ADLS.
-
Extensive experience with Azure Databricks and Pyspark based data processing, optimization, and performance tuning.
-
Strong Hands-on experiences of ETL processes, data lakes and data pipeline architecture.
-
Strong Hands-on experiences with languages used in Databricks development: Python, Scala and SQL.
-
Have knowledge/experience of DevOps: GitHub/Git Actions and Artifactory
-
Document the solution design and prepare playbook for other team members to follow