Lead Data Engineer (healthcare solution)

• Set the technical direction for healthcare data engineering across ingestion, transformation, validation, normalization, and delivery.
• Own the end-to-end architecture of large-scale distributed data pipelines transforming heterogeneous source data into clean, standards-conformant FHIR.
• Design and evolve scalable solutions using Python, Spark/PySpark, Databricks/Delta Lake, and modern data-processing technologies.
• Diagnose and eliminate complex performance and scalability bottlenecks across distributed data workloads.
• Establish standards and reusable frameworks for data quality, deduplication, patient/record linking, terminology normalization, and reliable idempotent processing.
• Define architectural patterns for FHIR transformation, validation, schema evolution, and production delivery.
• Build and evolve automated testing and validation capabilities for data quality and FHIR conformance.
• Establish engineering patterns and reusable tooling that allow the broader team to deliver new healthcare data capabilities faster and more consistently.
• Champion PHI-safe engineering practices and ensure healthcare data solutions meet applicable security, privacy, and compliance requirements.
• Make AI-native engineering a practical part of the team’s development model, using coding agents and reusable AI capabilities to automate repetitive work and accelerate complex engineering tasks.
• Establish engineering guardrails for AI-assisted development, ensuring AI-generated solutions remain correct, tested, secure, observable, and PHI-safe.
• Determine where deterministic processing provides better reliability than AI/LLM-based approaches and establish appropriate architectural patterns for both.
• Mentor Senior and Middle engineers through design reviews, pairing, technical guidance, and knowledge sharing.
• Drive complex, ambiguous engineering problems from initial investigation through architecture, implementation, and production operation.
• Collaborate with engineering, product, security, data, and healthcare-domain stakeholders to translate complex interoperability requirements into scalable technical solutions.
• Represent strong healthcare data engineering practices in relevant architecture, standards, and technical discussions.

• 8+ years of professional experience in data engineering, healthcare data platforms, HealthTech, or healthcare interoperability.
• Proven Lead-level technical ownership of complex production systems, including the ability to set technical direction and solve problems spanning multiple components and engineering teams.
• Strong hands-on expertise in Python and production-grade data engineering.
• Deep experience with Apache Spark/PySpark at scale, including partitioning, performance optimization, memory tuning, and distributed processing.
• Experience with modern lakehouse technologies such as Databricks and Delta Lake, or comparable platforms.
• Strong experience designing and operating large-scale ETL/ELT pipelines, including CDC, incremental/delta processing, idempotency, schema evolution, and batch/streaming workloads.
• Proven experience diagnosing and eliminating complex performance and scalability bottlenecks in distributed data-processing systems.
• Deep hands-on experience with FHIR R4, including resources, profiles, Bundles, validation, and large-scale healthcare data transformation.
• Strong knowledge of US Core, USCDI, HL7 v2.x, and C-CDA-to-FHIR transformation patterns.
• Strong understanding of healthcare terminology standards including SNOMED CT, LOINC, and RxNorm.
• Experience working with clinical, claims, coverage, eligibility, or comparable healthcare data models.
• Strong experience with data quality, deduplication, record/patient linking, terminology normalization, and validation.
• Strong SQL skills and understanding of data modeling, schema evolution, data quality, and production data architecture.
• Experience with Git, CI/CD, automated testing, observability, and production operations.
• Strong understanding of healthcare security, privacy, and regulatory requirements, including HIPAA and PHI-safe engineering practices.
• Proven ability to establish reusable engineering frameworks, architectural patterns, standards, and data-quality practices adopted by other engineers.
• Demonstrated technical leadership through architecture/design reviews, mentoring, pairing, and cross-team engineering influence.
• Practical experience using AI coding agents such as Claude Code or comparable agentic engineering tools as an integral part of the development lifecycle.
• Strong engineering judgment around when to use AI/LLM-based approaches versus deterministic rule-based solutions, particularly where correctness, cost, performance, or PHI safety are critical.
• Professional English sufficient for direct collaboration with U.S.-based technical and business stakeholders.

Nice to Have

• Deep experience with large-scale FHIR data modeling and healthcare interoperability platforms.
• Experience with Master Patient Index (MPI), patient matching, identity resolution, or large-scale record linkage.
• Experience with payer data exchange, including CARIN Blue Button, Da Vinci implementation guides, or similar standards.
• Experience working with pharmacy, laboratory, or other specialized healthcare datasets.
• Knowledge of ICD-10, CPT, CVX, and additional clinical terminology/code systems.
• Experience with 21st Century Cures Act, ONC certification, CMS Interoperability Rule, or related U.S. healthcare regulatory frameworks.
• Experience building LLM/agent-based capabilities for clinical data extraction, normalization, terminology mapping, or healthcare data quality.
• Experience designing reusable AI skills or agentic workflows adopted across engineering teams.
• Contributions to open-source healthcare, FHIR/HL7 communities, or relevant industry working groups.