Principal Data Engineer

Worth AI, a leader in the computer software industry, is looking for a talented and experienced Principal Data Engineer to join their innovative team. At Worth AI, we are on a mission to revolutionize decision-making with the power of artificial intelligence while fostering an environment of collaboration, and adaptability, aiming to make a meaningful impact in the tech landscape. Our team values include extreme ownership, one team and creating raving fans both for our employees and customers.Worth is looking for a Principal Data Engineer to own the company-wide data architecture and platform. Design and scale reliable batch/streaming pipelines, institute data quality and governance, and enable analytics/ML with secure, cost-efficient systems. Partner with engineering, product, analytics, and security to turn business needs into durable data products.ResponsibilitiesArchitecture & StrategyDefine end-to-end data architecture (lake/lakehouse/warehouse, batch/streaming, CDC, metadata)Set standards for schemas, contracts, orchestration, storage layers, and semantic/metrics modelsPublish roadmaps, ADRs/RFCs, and 'north star' target states; guide build vs. buy decisionsPlatform & PipelinesDesign and build scalable, observable ELT/ETL and event pipelinesEstablish ingestion patterns (CDC, file, API, message bus) and schema-evolution policiesProvide self-service tooling for analysts/scientists (dbt, notebooks, catalogs, feature stores)Ensure workflow reliability (idempotency, retries, backfills, SLAs)Data Quality & GovernanceDefine dataset SLAs/SLOs, freshness, lineage, and data certification tiersEnforce contracts and validation tests; deploy anomaly detection and incident runbooksPartner with governance on cataloging, PII handling, retention, and access policiesReliability, Performance & CostLead capacity planning, partitioning/clustering, and query optimizationIntroduce SRE-style practices for data (error budgets, postmortems)Drive FinOps for storage/compute; monitor and reduce cost per TB/query/jobSecurity & ComplianceImplement encryption, tokenization, and row/column-level security; manage secrets and auditsAlign with SOC 2 and privacy regulations (e.g., GDPR/CCPA; HIPAA if applicable)ML & Analytics EnablementDeliver versioned, documented datasets/features for BI and MLOperationalize training/serving data flows, drift signals, and feature-store governanceBuild and maintain the semantic layer and metrics consistency for experimentation/BILeadership & CollaborationProvide technical leadership across squads; mentor senior/staff engineersRun design reviews and drive consensus on complex trade-offsTranslate business goals into data products with product/analytics leadersRequirements10+ years in data engineering (including 3+ years as staff/principal or equivalent scope)Proven leadership of company-wide data architecture and platform initiativesDeep experience with at least one cloud (AWS) and a modern warehouse or lakehouse (e.g., Snowflake, Redshift, Databricks)Strong SQL and one programming language (Python or Scala/Java)Orchestration (Airflow/Dagster/Prefect), transformations (dbt or equivalent), and streaming (Kafka/Kinesis/PubSub)Data modeling (3NF, star, data vault) and semantic/metrics layersData quality testing, lineage, and observability in production environmentsSecurity best practices: RBAC/ABAC, encryption, key management, auditabilityNice to HaveFeature stores and ML data ops; experimentation frameworksCost optimization at scale; multi-tenant architecturesGovernance tools (DataHub/Collibra/Alation), OpenLineage, and testing frameworks (Great Expectations/Deequ)Compliance exposure (SOC 2, GDPR/CCPA; HIPAA/PCI where relevant)Model features sourced from complex 3rd-party data (KYB/KYC, credit bureaus, fraud detection APIs)BenefitsHealth Care Plan (Medical, Dental & Vision)Retirement Plan (401k, IRA)Life InsuranceUnlimited Paid Time Off9 paid HolidaysFamily LeaveWork From HomeFree Food & Snacks (Access to Industrious Co-working Membership!)Wellness Resources #J-18808-Ljbffr

Created: 2025-10-08