Principal Data Engineer

 

We are building the Data Hub, a centralized data platform responsible for consolidating legacy data infrastructure, establishing enterprise-grade data foundations, and enabling advanced analytics and AI capabilities across Precision AQ.

The Principal Data Engineer is a senior technical leader and hands-on contributor responsible for the design, implementation, optimization, and operation of the Data Hub’s core platform capabilities. This role sits at the intersection of data engineering, platform engineering, DevOps, architecture, and data governance. You will help define the technical direction of the platform while remaining actively involved in designing solutions, building infrastructure, reviewing code, troubleshooting production issues, and improving engineering practices.

This is not a management role. While you will mentor and guide junior engineers, analytics engineers, and platform contributors, you will not have direct reports. We are looking for an experienced practitioner who enjoys solving complex technical challenges, setting high engineering standards, and leading through expertise and influence rather than organizational hierarchy.

As a member of the Data Hub leadership team, you will contribute to technology strategy, roadmap prioritization, architecture governance, vendor evaluation, and executive reporting while remaining deeply engaged in day-to-day engineering activities.

Main duties and responsibilities 

Infrastructure and Platform Engineering

• Design, build, and maintain scalable, secure, and reliable cloud-native data platform infrastructure.
• Develop infrastructure-as-code, CI/CD pipelines, deployment automation, and environment management processes.
• Partner with Corporate IT, Security, and platform vendors to ensure compliance, reliability, and operational excellence.
• Improve platform observability through monitoring, alerting, logging, and performance tracking.

Data Architecture and Modelling

• Design scalable data models supporting analytics, reporting, AI/ML, and operational use cases.
• Define and evolve data architecture standards, patterns, and best practices across the Data Hub ecosystem.
• Ensure data solutions align with governance, lineage, security, and regulatory requirements.
• Guide engineering teams in implementing maintainable and extensible data structures.

Data Engineering and Solution Delivery

• Build and optimize data ingestion, transformation, and delivery pipelines across multiple business domains.
• Lead technical design reviews and contribute directly to implementation of complex data engineering initiatives.
• Collaborate with Product, Analytics, AI/ML, and business stakeholders to translate requirements into scalable technical solutions.
• Provide hands-on support for critical platform initiatives, migrations, and modernization programs.

DevOps and Engineering Excellence

• Establish and enforce software engineering, GitOps, DevOps, testing, and deployment standards.
• Drive automation across development, deployment, monitoring, and operational processes.
• Promote best practices for code quality, documentation, technical debt management, and release management.
• Conduct architecture reviews and code reviews to ensure consistency and maintainability.

Quality, Performance, and Reliability Optimization

• Identify and resolve performance bottlenecks across pipelines, databases, and infrastructure.
• Define and implement data quality frameworks, automated validation processes, and operational controls.
• Optimize platform cost, scalability, reliability, and processing efficiency.
• Lead root-cause analysis and remediation efforts for production incidents and operational challenges.

Technical Leadership and Mentorship

• Mentor Data Engineers, Analytics Engineers, and other technical contributors through coaching, code reviews, pair programming, and design guidance.
• Act as a trusted technical advisor across multiple teams and business units.
• Share knowledge, develop engineering standards, and promote continuous improvement across the Data Hub organization.
• Help elevate the technical capabilities of the broader engineering team.

Strategic Planning and Cross-Functional Collaboration

• Participate in Data Hub leadership activities, including roadmap planning, prioritization discussions, technology evaluations, and executive reporting.
• Provide technical recommendations on platform investments, vendor selection, architecture direction, and engineering standards.
• Collaborate closely with Data Product Management, AI/ML teams, business stakeholders, and Corporate IT to ensure alignment between business priorities and technical execution.

Education and Experience

Education: Bachelor’s degree in computer science, engineering, Information Systems, or related field. Equivalent experience considered. Advance degree preferred but not required.

Required:

• 8+ years of experience in Data Engineering, Platform Engineering, Infrastructure Engineering, or related technical disciplines.
• Demonstrated expertise designing and implementing modern cloud-based data platforms.
• Strong hands-on experience with data warehousing, orchestration, transformation, and DevOps technologies.
• Experience building and optimizing large-scale data pipelines and data models.
• Deep understanding of software engineering principles, infrastructure automation, and production operations.
• Experience operating in highly regulated, compliance-sensitive, or enterprise environments.
• Proven track record of leading technical initiatives through influence rather than direct authority.

Preferred:

• Experience within healthcare, life sciences, pharmaceutical, or analytics organizations.
• Experience supporting AI/ML platforms, MLOps capabilities, feature stores, or model deployment frameworks.
• Familiarity with claims, formulary, commercial, or real-world evidence datasets.
• Experience leading enterprise-scale data migrations, modernization programs, or platform consolidations.
• Exposure to global engineering teams and distributed delivery models.

Knowledge, Skills and Competencies

Technical Expertise

• Advanced knowledge of cloud data platforms, modern data architecture, and platform engineering.
• Deep expertise in data modelling, query optimization, and analytical data design.
• Ability to evaluate and select appropriate technologies, frameworks, and architectural approaches.
• Expert-level SQL and strong Python skills.

Infrastructure and DevOps

• Strong understanding of CI/CD, GitOps, Infrastructure-as-Code, containerization, and deployment automation.
• Experience implementing operational monitoring, incident management, alerting, and observability solutions.
• Knowledge of security, access management, and platform governance best practices.
• Strong understanding of scalability, resilience, and performance engineering.

Leadership and Influence

• Leads through credibility, expertise, and collaboration rather than formal authority. Strong mentoring and coaching capabilities.
• Comfortable facilitating technical discussions and driving consensus among diverse stakeholders and ability to influence architecture and technology decisions across teams.

Communication and Collaboration

• Excellent written and verbal communication skills and able to communicate complex technical concepts to both technical and non-technical audiences.
• Effective stakeholder management across engineering, product, business, and executive teams. Strong decision-making and prioritization skills in ambiguous environments.

Technical Environment

You will work across a modern data and AI ecosystem that may include:

• Data Platforms: Snowflake, AWS Redshift, S3
• Transformation: dbt, Matillion
• Orchestration: Airflow, Dagster
• Data Governance: Snowflake Horizon
• Data Quality: dbt tests, Elementary
• Infrastructure & DevOps: Terraform, GitHub Actions, Azure DevOps
• AI/ML: Snowpark ML, MLflow, Snowflake Feature Store, LangChain
• Languages: SQL, Python