À propos
Senior Data Engineer The Senior Data Engineer reports directly to the CTO and is responsible for delivering a reliable, production-ready market data platform that serves as the single source of truth for trading, risk, and business intelligence.
You will own the entire data platform from inception, deliver working historical backfill and real‑time Tardis pipelines within the first 60 days, and continue to evolve it into a best‑in‑class, cloud‑native, observable, self‑service system. The role includes mentoring junior staff and acting as the go‑to data expert for the firm.
What You’ll Do
Rapidly spin up the cloud environment and deliver historical backfill pipelines from Tardis.dev into a queryable database.
Deliver a real‑time Tardis WebSocket pipeline, ensuring data is normalized, cached for live consumption, accurate, replayable, and queryable by day 60.
Ensure all pipelines are idempotent, retryable, and use exactly‑once semantics. Implement full CI/CD, Terraform, automated testing, and secrets management.
Implement observability (structured logs, metrics, dashboards, alerting) from day one and provide self‑service access to the MVP database for Trading and BI teams via Tableau, Metabase, or internal REST APIs.
Develop specialized time‑series data, including USDe backing‑asset and a full opportunity‑surface time‑series for delta‑neutral, lending, and borrow opportunities.
Ingest data from additional sources (Kaiko, CoinAPI, on‑chain via TheGraph/Dune). Plan for 10x data growth with schema evolution, partitioning, and performance tuning, and establish enterprise‑grade governance, including a data quality framework, RBAC, audit logs, and a semantic layer.
Create full architecture documentation, runbooks, and a data dictionary. Onboard and mentor future junior staff.
What We’re Looking For
Proven track record of delivering production data in weeks, with the focus on a 60‑day MVP, and the ability to manage technical debt.
Experience building Tardis historical and real‑time pipelines (or equivalent high‑quality crypto market data feeds) with an understanding of rate limits and WebSocket structures.
Expertise in large‑scale, reliable ETL/ELT for financial or market data.
Fluency in provisioning full environments with Terraform and serverless technologies on AWS/GCP.
Advanced Python and SQL skills and proficiency with time‑series databases such as TimescaleDB or ClickHouse.
Expertise in WebSocket clients, message queues, low‑latency streaming, GitOps, automated testing/deployments, and observability practices.
Significant knowledge of stablecoins, lending protocols, and opportunity surface concepts, or a proven ability to ramp up quickly.
#J-18808-Ljbffr
Compétences linguistiques
- English
Avis aux utilisateurs
Cette offre provient d’une plateforme partenaire de TieTalent. Cliquez sur « Postuler maintenant » pour soumettre votre candidature directement sur leur site.