Azure Data Engineer
Position Description
Join this exciting new project as an Azure Data Engineer! CGI is looking for an Azure Data Engineer to join the team in Lafayette, LA.! With your required expertise in developing and deploying application on MS Azure Databricks, you will work with a high-performing team to implement a Cloud Modernization Project. You will support and interact with the client daily in an Agile way of working. You will have the opportunity to show/grow with technology in a customer-facing role. You should be self-motivated to drive solutions and proactively create (in sufficient detail) the documentation required to support and describe technical solutions. You should be able to work independently under limited supervision and apply your knowledge of Azure-based ETL. You will work with cloud architects to establish pipelines and develop spark code using Databricks platform. You should have sufficient knowledge and maturity to effectively deal with technical issues and help to support the broader team. Your success will be directly related to CGI's and our client's success.
Your future duties and responsibilities
• The data engineer will be responsible for development, deployment, maintenance, diagnostics and support of spark ETL jobs on the MS Databricks platform.
• Will streamline code, rationalize datasets and tables to arrive at single source of truth
• Will build detailed architecture diagrams for all workflows and processes
jobs, cron, ESP etc.
• Excellent communication skills
Required qualifications to be successful in this role
• 3 years of hands on experience with cloud automation and scripting in an Azure environment.
• Experience in development of apache Spark code using PySpark/Scala
• Experience in using Azure Databricks Platform
• Experience in using Azure Data Factory to call Azure Databricks notebook activities
• Experience in using Azure Storage (Blob/Data Lake Store), Databricks Delta
• Experience in using Git based repositories
• Good fundamental knowledge about distributed computing, RDBMS and Dimensional modeling concepts
• Exposure to Azure DevOps
Skill Set/Years of Experience/Proficiency Level
Information Technology
6 years
High
ETL
4 years
High
SQL
4 years
High
Azure Databricks
3 years
High
Minimum Education Required: Bachelors Degree
#doubledown
Build your career with us.
It is an extraordinary time to be in business. As digital transformation continues to accelerate, CGI is at the center of this change-supporting our clients' digital journeys and offering our professionals exciting career opportunities.
At CGI, our success comes from the talent and commitment of our professionals. As one team, we share the challenges and rewards that come from growing our company, which reinforces our culture of ownership. All of our professionals benefit from the value we collectively create.
Be part of building one of the largest independent technology and business services firms in the world.
Learn more about CGI at www.cgi.com .
No unsolicited agency referrals please.
CGI is an equal opportunity employer.
Qualified applicants will receive consideration for employment without regard to their race, ethnicity, ancestry, color, sex, religion, creed, age, national origin, citizenship status, disability, pregnancy, medical condition, military and veteran status, marital status, sexual orientation or perceived sexual orientation, gender, gender identity, and gender expression, familial status, political affiliation, genetic information, or any other legally protected status or characteristics.
CGI provides reasonable accommodations to qualified individuals with disabilities. If you need an accommodation to apply for a job in the U.S., please email the CGI U.S. Employment Compliance mailbox at [ Email address blocked ] - Click here to apply to Azure Data Engineer . You will need to reference the requisition number of the position in which you are interested. Your message will be routed to the appropriate recruiter who will assist you. Please note, this email address is only to be used for those individuals who need an accommodation to apply for a job. Emails for any other reason or those that do not include a requisition number will not be returned.
We make it easy to translate military experience and skills! Click here to be directed to our site that is dedicated to veterans and transitioning service members.
All CGI offers of employment in the U.S. are contingent upon the ability to successfully complete a background investigation. Background investigation components can vary dependent upon specific assignment and/or level of US government security clearance held. CGI will consider for employment qualified applicants with arrests and conviction records in accordance with all local regulations and ordinances.
CGI will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. However, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with CGI's legal duty to furnish information.
Skills
Pyspark
Relational Database Management Systems
Workflows
Dimensional Modeling
Extract Transform Load (Etl)
Position Description
Join this exciting new project as an Azure Data Engineer! CGI is looking for an Azure Data Engineer to join the team in Lafayette, LA.! With your required expertise in developing and deploying application on MS Azure Databricks, you will work with a high-performing team to implement a Cloud Modernization Project. You will support and interact with the client daily in an Agile way of working. You will have the opportunity to show/grow with technology in a customer-facing role. You should be self-motivated to drive solutions and proactively create (in sufficient detail) the documentation required to support and describe technical solutions. You should be able to work independently under limited supervision and apply your knowledge of Azure-based ETL. You will work with cloud architects to establish pipelines and develop spark code using Databricks platform. You should have sufficient knowledge and maturity to effectively deal with technical issues and help to support the broader team. Your success will be directly related to CGI's and our client's success.
Your future duties and responsibilities
• The data engineer will be responsible for development, deployment, maintenance, diagnostics and support of spark ETL jobs on the MS Databricks platform.
• Will streamline code, rationalize datasets and tables to arrive at single source of truth
• Will build detailed architecture diagrams for all workflows and processes
jobs, cron, ESP etc.
• Excellent communication skills
Required qualifications to be successful in this role
• 3 years of hands on experience with cloud automation and scripting in an Azure environment.
• Experience in development of apache Spark code using PySpark/Scala
• Experience in using Azure Databricks Platform
• Experience in using Azure Data Factory to call Azure Databricks notebook activities
• Experience in using Azure Storage (Blob/Data Lake Store), Databricks Delta
• Experience in using Git based repositories
• Good fundamental knowledge about distributed computing, RDBMS and Dimensional modeling concepts
• Exposure to Azure DevOps
Skill Set/Years of Experience/Proficiency Level
Information Technology
6 years
High
ETL
4 years
High
SQL
4 years
High
Azure Databricks
3 years
High
Minimum Education Required: Bachelors Degree
#doubledown
Build your career with us.
It is an extraordinary time to be in business. As digital transformation continues to accelerate, CGI is at the center of this change-supporting our clients' digital journeys and offering our professionals exciting career opportunities.
At CGI, our success comes from the talent and commitment of our professionals. As one team, we share the challenges and rewards that come from growing our company, which reinforces our culture of ownership. All of our professionals benefit from the value we collectively create.
Be part of building one of the largest independent technology and business services firms in the world.
Learn more about CGI at www.cgi.com .
No unsolicited agency referrals please.
CGI is an equal opportunity employer.
Qualified applicants will receive consideration for employment without regard to their race, ethnicity, ancestry, color, sex, religion, creed, age, national origin, citizenship status, disability, pregnancy, medical condition, military and veteran status, marital status, sexual orientation or perceived sexual orientation, gender, gender identity, and gender expression, familial status, political affiliation, genetic information, or any other legally protected status or characteristics.
CGI provides reasonable accommodations to qualified individuals with disabilities. If you need an accommodation to apply for a job in the U.S., please email the CGI U.S. Employment Compliance mailbox at [ Email address blocked ] - Click here to apply to Azure Data Engineer . You will need to reference the requisition number of the position in which you are interested. Your message will be routed to the appropriate recruiter who will assist you. Please note, this email address is only to be used for those individuals who need an accommodation to apply for a job. Emails for any other reason or those that do not include a requisition number will not be returned.
We make it easy to translate military experience and skills! Click here to be directed to our site that is dedicated to veterans and transitioning service members.
All CGI offers of employment in the U.S. are contingent upon the ability to successfully complete a background investigation. Background investigation components can vary dependent upon specific assignment and/or level of US government security clearance held. CGI will consider for employment qualified applicants with arrests and conviction records in accordance with all local regulations and ordinances.
CGI will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. However, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with CGI's legal duty to furnish information.
Skills
Recommended Skills
Storage (Computing)Pyspark
Relational Database Management Systems
Workflows
Dimensional Modeling
Extract Transform Load (Etl)