Sunset
Backend Engineer — Data Pipeline
$200K
New York · On-site · Full-time · Posted Sep 4, 2026
- Experience
- At least three years of professional software engineering experience.
About the job
Build backend systems for a high-volume pipeline that de-identifies enterprise data and makes processing, verification, recovery, and release safe and auditable.
What you'll do
- Design multi-stage processing systems and versioned contracts for manifests, artifacts, lineage, and state transitions.
- Build replay, reconciliation, verification, and recovery mechanisms for failures, backfills, migrations, and rollbacks.
- Diagnose production behavior and improve correctness and throughput while protecting privacy, security, and release confidence.
What they ask for
- At least three years of professional software engineering experience, including personal ownership of production backend systems.
- Strength in asynchronous or distributed systems, including queues, concurrency, state, storage, idempotency, partial failure, and recovery.
- Experience verifying correctness with methods such as reconciliation, control totals, diffs, replay, goldens, or independent sources.
- Ability to design versioned data and artifact contracts, migrate them safely, and debug across system boundaries.
Nice to have
- Experience with batch processing, workflow orchestration, event-driven systems, or data movement.
- Experience with sensitive or multi-tenant data, auditability, or model-based checks combined with deterministic checks and human review.
Pay
$200K USD base salary
Tools
Python, AWS, Airflow, Batch, SQS, S3, DynamoDB, PostgreSQL
AI focus
Use AI in daily development and in bounded verification systems, with explicit evaluation, independent checks, deterministic guardrails, and human review.
Model-written summary · the original posting is authoritative