Lead Data Engineer
Job Title: Lead Data Engineer
Reports To: Director, Software Development
Department: Technology & Product
Location: Remote
Prometric is a global leader in test development, test delivery, and candidate services for academic, professional, government, and corporate clients. We are looking for a highly skilled and motivated technology lead to join our team to work to on innovative technologies in the Data Engineering space.
Main Duties & Responsibilities:
- Design, develop, and support scalable data processing pipelines using Apache Spark and Java/Scala.
- Lead a talented team and make a significant impact on our data engineering capabilities.
- Implement and manage workflow orchestration with AirFlow for efficient data processing.
- Proficiently use SQL for querying and data manipulation tasks.
- Collaborate with cross-functional teams to gather requirements and ensure alignment with data engineering solutions.
Essential Criteria:
- Bachelor’s degree in computer science or another relevant discipline, and a minimum of five years relevant experience in Data Engineering.
- Solid experience with Apache Spark for large-scale data processing.
- Proficiency in Java or Scala programming languages.
- Strong knowledge of AirFlow for workflow orchestration.
- Proficient in SQL for data querying and manipulation.
Desirable Criteria:
- Familiarity with Avro and Schema Registry for efficient data serialization.
- Experience with containerization technologies such as Docker and orchestration tools like Kubernetes (K8S).
- Hands-on experience with cloud-based data engineering services, preferably Azure Data Factory or AWS Glue.
- Understanding of release management processes.
- Familiarity with version control systems such as Git and experience with release pipelines using Azure DevOps or similar tools.
- Ability to effectively work with teams across different geographical locations, including Ireland, the United States (Eastern Time Zone), and India.
If you are a passionate and experienced Tech Lead with a strong background in data engineering, we encourage you to apply. Join us to help shape the future of our data solutions at Prometric.
Autres détails
- Type de paie Salaire