Job Description
Do you have Big Data, Spark ETL, Fabric, and Python experience, and want to be part of a new team? We are looking for you. You will be working with one of the best new consulting companies and have an integral role in our new data consulting practice.
Jarvis Consulting Group is a dynamic consulting firm specializing in cloud and data solutions. We work closely with clients across various industries to deliver tailored, innovative projects. Jarvis partners with some of the world's most impactful companies to shape the future of tomorrow through an innovative and supportive approach that fosters creativity and unrivalled collaboration, enabling our clients to win in the world of work. We are change agents bent on disruption by challenging the status quo.
Work you will do
- Build out scalable and reliable ETL pipelines and processes to ingest and transform data from a large number and variety of data sources.
- Understand Big Data and Cloud environments from a performance and capacity management standpoint, using the tools needed to onboard, monitor and move data into these environments
- Collaborate as the Big Data and Cloud Technical lead with multiple technology teams to ensure that the appropriate associated applications, integrations, infrastructure, and security architecture are designed to meet evolving business requirements, meet standards for reliability, scalability, performance, and availability
- Be the lead for internal and external products and projects.
- Responsible for the successful implementation of technical solutions for projects, supporting highly complex business applications with complex integration needs across multiple technology discipline
- Formulate and define project scope, and objectives based on a thorough understanding of the technical requirements of the projects
- Use sound Agile development practices (code reviews, unit testing, etc.) to develop and deliver quality code and data products
- Provide day-to-day support and technical expertise to both technical and non-technical teams
- Help to develop junior talent and transform them into experts
- Work with other engineers to brainstorm solutions to problems and support bank objectives
Who we are looking for
- 6+ years Spark Development experience
- Knowledge of Hadoop and Cloud Big Data ecosystem (Spark, Hive, HDFS, Azure/AWS/GCP, etc)
- Experience with streaming technologies such as Kafka and Spark Streaming
- Experience with Agile development (JIRA / Confluence)
- Experience with version control systems, such as Git
- Knowledge of dev tools like IDE’s, Maven, SBT
- Knowledge of orchestration tools like Airflow, Autosys, Oozie and Cron
- Experience with containerization (Docker) and orchestration (Kubernetes) is a plus
- Experience with developing and deploying applications to the cloud environment
Who you are
- Works well both individually and as part of a team
- Proven ability to work creatively in a problem-solving environment
- Strong communication skills
- Able to work closely with technical and non-technical team in a collaborative environment
- University degree in relevant STEM discipline (Computer Sciences, Electrical/Computer/Software Engineering and Mathematics)