Lead Data Engineer
Who we are
About this role
What you’ll do
- Lead the technical design and delivery of complex data products, pipelines, transformations, models and serving interfaces within agreed architecture and platform guardrails.
- Lead adoption and assurance of engineering practices across Python, SQL, data transformation, orchestration, CI/CD, automated testing, data quality, observability and documentation.
- Review critical designs and code, providing technical direction that improves engineering quality, consistency and maintainability.
- Resolve complex production issues and ensure solutions meet appropriate standards for scalability, security, resilience, maintainability and operational readiness.
- Define reusable product-engineering patterns while avoiding duplication of shared platform capabilities or bypassing established controls.
- Escalate cross-domain architectural decisions to the Lead Data Architect where appropriate.
- Collaborate with Data Science and ML & AI Engineering teams on trusted features, training data, inference data and outcome data.
- Design and support reliable batch, streaming and event-driven data pipelines.
- Provide technical leadership and mentoring to experienced engineers through critical design reviews, code reviews and production-incident resolution.
- Provide technical input into estimates, tooling and technology choices.
- Help reduce engineering defects, rework and duplicated patterns while improving conformance with architecture, data contracts, testing and security controls.
Who you are
- You are an experienced Data Engineer who combines advanced hands-on engineering capability with the technical judgement needed to guide complex production data solutions.
- Extensive production Data Engineering experience, including technical leadership of complex data solutions.
- Advanced Python and SQL capability.
- Strong experience with dbt or equivalent transformation and orchestration tooling.
- Strong knowledge of Google Cloud Platform data services or equivalent cloud technologies.
- Proven experience designing reliable batch, streaming and event-driven pipelines.
- Deep understanding of automated testing, CI/CD, observability, data quality, lineage, security and operational supportability.
- Experience designing scalable, resilient and maintainable data solutions within established architecture and platform standards.
- Sound technical judgement around data contracts, scalability, cloud cost, resilience and the reuse of shared platform services.
- Evidence of technical mentoring, critical design and code review, and leadership during production incidents.
- The ability to provide technical direction through influence and collaborate effectively across Architecture, Platform, Product, Data Science and ML & AI Engineering.
- Experience helping ensure data products are production-ready, secure and aligned with established engineering controls.
Not a perfect fit?
What's in it for you
- The opportunity to take a leading technical role in the design and delivery of complex, production-grade data products.
- Work across batch, streaming and event-driven architectures, addressing a broad range of Data Engineering challenges.
- Influence engineering standards and reusable patterns across Python, SQL, transformation, orchestration, testing, CI/CD and observability.
- Collaborate closely with Architecture, Platform, Product, Data Science and ML & AI Engineering teams.
- Influence technical choices covering scalability, resilience, security, cloud cost and maintainability.
- Apply AI-assisted engineering practices while maintaining human accountability, security, privacy, traceability and production discipline.
- Mentor experienced engineers and help improve technical quality across the wider Data Products & Insights team.
What skills you will learn
- Technical Data Engineering leadership, including critical design review, mentoring and engineering assurance.
- Advanced cloud-native Data Engineering, including the design of scalable batch, streaming and event-driven solutions.
- Production engineering practices, spanning automated testing, CI/CD, observability, data quality, lineage, security and resilience.
- Reusable data-product engineering patterns, balancing product requirements with shared platform capabilities and architectural controls.
- Cloud data engineering, through the use of Google Cloud Platform data services or equivalent technologies.
- Data Science and ML/AI integration, including collaboration around trusted features, training, inference and outcome data.
- Architecture and platform alignment, strengthening decision-making around data contracts, scalability, cost, resilience and reuse.
- AI-assisted engineering practices that support efficient delivery while maintaining production quality, traceability and accountability.
VOIS Equal Opportunity Employer Commitment
Join Us
Alert
Follow us on social media and #StayConnected
- LinkedIn: https://www.linkedin.com/company/vois/
- Facebook: https://www.facebook.com/voisglobal
- Instagram: https://www.instagram.com/voisglobal/
- You can also chat with our employees to learn more about our projects: https://lnkd.in/dpkrcvR2