Data Engineering Azure/Databricks: Expert level engineer at Onsite
1. Azure Cloud DevOps experience involving Azure cloud architecture, data movement from external sources to ADLS and snowflake.
2. Azure Data Engineer strong experience in ADF, Azure Databricks
4. Logging and Auditing experience.
5. CICD integrations frameworks (GitLab (preferably)) used.
6. Experience with Job scheduling using ADF trigger/Control-M etc.
7. Experience with Data Encryption/Decryption within Azure/snowflake/postgres etc
1. Azure Cloud DevOps experience involving Azure cloud architecture, data movement from external sources to ADLS and snowflake.
2. Azure Data Engineer strong experience in ADF, Azure Databricks
- a. Databricks experience using Scala/Python coding.
- b. Experience working with full load/incremental and streaming data.
- c. Experience working with delta tables, data transformations using python/scala.
- d. Experience working with structured, semi-structured (JSON, XML etc.) and unstructured data.
- e. Nice to have Azure logic app experience.
- f. Experience with application development with Azure App services, API and Microservices development.
4. Logging and Auditing experience.
5. CICD integrations frameworks (GitLab (preferably)) used.
6. Experience with Job scheduling using ADF trigger/Control-M etc.
7. Experience with Data Encryption/Decryption within Azure/snowflake/postgres etc