Role: Sr. Platform / DataOps Engineer
Contract Length: 12 months
Days in Office: 3 days on-site / 2 days remote
Job Description
What is the Opportunity?
We are seeking a hands-on, highly skilled, and motivated individual to join our dynamic team as a Senior Data Platform/DataOps Engineer. As a key member of our data engineering team, you will play a crucial role in the development and implementation of our Asset Management Data Platform. Your primary focus will be on designing, implementing, and maintaining robust data infrastructure to support our organization’s data needs and advance our capabilities pertaining to data engineering, BI, Machine Learning and AI.
What you will do:
- Work closely with Data Engineering and Analytics team to drive business decisions using data.
- Build scalable data platform infrastructure using cloud native technologies to enable Data Engineering, Data Science and Reporting.
- Take ownership of end-to-end deliverables and ensure high quality data solutions adhering to data platform architecture guidelines and engineering best practices.
- Enable the deployment of batch and event-driven data pipelines using latest technologies such as Spark, NiFi, Kafka, and Airflow.
- Configure and manage scalable and managed SQL based self-serve compute platform using Starburst Trino.
- Implement data management tools to streamline data processing, testing, preparing, deploying, and monitoring.
- Improve the speed and quality of the data development process by applying DevOps principles to data workflow.
- Ensure data security standards are applied across the data pipelines.
- Troubleshoot and resolve data integration and data processing issues.
- Perform proof-of-concepts to implement hybrid cloud data strategy involving on-prem systems like S3, OpenShift, Trino, Spark with Cloud based solutions like Databricks, Azure, Snowflake, etc.
- Apply design-thinking and agile mindset in working with other engineers and business stakeholders to continuously experiment, iterate and deliver on new initiatives.
- Stay informed about emerging technologies and trends in data engineering domain.
- Lead, mentor, and inspire a team of engineers to achieve high performance levels.
What you need:
- 3-5 years of hands-on experience in big data processing using Spark, Python, NiFi, Kafka, and Airflow for cloud and on-premises data platforms.
- 2-3 years of experience with solution design, data ingestion, scheduling & monitoring of ETL/ELT jobs.
- 2-3 years of system administration including Docker and Kubernetes platforms.
- Strong knowledge of UNIX-like operating systems and shell scripting skills.
- Experience with variety of SQL and NoSQL databases and ETL/ELT software.
- Proficiency in writing and optimizing SQL queries and at least one programming languages like Java, Scala and/or Python.
- Experience with cloud-based data platforms (Snowflakes, AWS, Azure, GCP)
- Experience with DevOps tools, automation, and using Data Ops to develop data flows and the continuous use of data.
- Continuously learning mindset and enjoy working on open-ended problems.
Nice-to-have
- Experience with OpenShift, AirFlow, S3, Trino, Ranger and Hive.
- Knowledge of networking, information security and vulnerability management.
- Knowledge of creating dashboards using Prometheus and Grafana.
- Knowledge about data science tools and libraries.
Don’t meet every single requirement? Apply anyways!
Jarvis Consulting Group is committed to developing a barrier-free recruitment process and work environment. We are committed to equality in the workplace and to growing a welcoming an inclusive team. We encourage applications from Indigenous persons, visible minority group members, and people of all sexual orientations and genders, and will consider all qualified applicants without regard to race, colour, religion, gender, gender identity or expression, sexual orientation, national origin, or disability or age. Jarvis is committed to compliance with all fair employment practices regarding citizenship and immigration status.
Accommodations are available on request for candidates partaking in any stage of the selection process. If you are a job seeker who requires assistance, please contact: po@jrvs.ca
Vous ne répondez pas à toutes les exigences ? Postulez quand même !
Jarvis Consulting Group s’engage à offrir un processus de recrutement et un environnement de travail sans barrières.
Nous nous engageons à favoriser l’égalité sur le lieu de travail et à former une équipe accueillante et inclusive. Nous encourageons les candidatures de personnes autochtones, de membres de minorités visibles et de personnes de toutes orientations sexuelles et de tous genres. Nous examinerons toutes les candidatures qualifiées sans égard à la race, la religion, au sexe, à l’identité ou à l’expression sexuelle, à l’orientation sexuelle, au handicap ou à l’âge. Jarvis s’engage à respecter toutes les pratiques équitables en matière d’emploi en ce qui concerne la citoyenneté et le statut d’immigrant.
Des aménagements sont disponibles sur demande, pour les candidats qui sont convoqués en entrevue, à n’importe quelle étape du processus de sélection. Si vous êtes à la recherche d’un emploi et que vous avez besoin d’aide, veuillez contacter : po@jrvs.ca