Data Engineer

Data Engineer (Q4 2025 hire)

Build and maintain ETL pipelines that process contracts and invoices at scale with high data quality

Technical Delivery Requirements

Data Pipeline Architecture

• ETL pipelines for contract ingestion: PDF/DOCX → structured data
• Invoice processing pipeline: CSV/Excel/PDF → validation queue
• Historical data migration for new customers (10K+ invoices per health system)
• Real-time data streaming for webhook integrations
• Data quality monitoring and validation

Data Warehouse

• Build analytics data warehouse for reporting
• Aggregate validation metrics, savings data, exception patterns
• Customer-specific data models for ROI reporting
• Time-series data for trend analysis
• Support for 245+ contract/validation schema fields

Data Quality & Governance

• Data validation: completeness, accuracy, consistency checks
• PII/PHI detection and masking
• Data lineage tracking: source → transformation → destination
• Error handling and dead letter queues
• Data retention policies (5-year audit trail)

Required Skills

• ETL tools: Airflow, Prefect, or similar for workflow orchestration
• Python: Data processing libraries (pandas, numpy, pyspark)
• SQL mastery: PostgreSQL, data modeling, query optimization
• Data warehousing: Redshift, BigQuery, or Snowflake
• Streaming: Kafka, Kinesis for real-time data processing
• Data quality: Great Expectations or similar frameworks

Key Deliverables (Next 6 months)

• Build ETL pipelines for 9 customer environments
• Process 100K+ historical invoices for new customers
• Achieve 99%+ data pipeline reliability
• Reduce data processing time by 30%
• Build analytics warehouse for reporting module