REGISTRIEREN

Jobbörse

Finde Jobs in deiner Nähe – ob vor Ort, hybrid oder remote.
  • Ähnliche Jobs zu: Senior Software Engineer - Data & Analytics Federation
XX
Senior Software Engineer - Analytics Data Platform LakehouseUnited States Digital Space LLC • New York, New York, United States
Jetzt Bewerben
XX

Senior Software Engineer - Analytics Data Platform Lakehouse

United States Digital Space LLC
  • US
    New York, New York, United States
  • US
    New York, New York, United States
Jetzt Bewerben

Über

Analytics Data Platform Lakehouse team builds and operates the foundations that power data engineers, applied AI, and product teams—managing millions of tables on their behalf and simplifying operations from maintenance and observability to governance, for both internal and customer-facing use cases. If you're excited by the intersection of petabyte data processing scale, open-source query engines, and building platforms with real product stakes, this is the team for you.
At the company, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them.
What You’ll Do:
Design, build, and operate core components of our lakehouse platform, including Apache Iceberg table management (data compaction, data layout optimization, materialized view scheduling…) and Iceberg catalog
Drive adoption of open table formats across internal teams, owning the integration of Trino, Spark and other query engines (DuckDB, Puppygraph…) with our Iceberg-based lakehouse at petabyte scale
Build observability for managed iceberg tables, to identify query performance bottlenecks, cost drivers and contribute fixes back to upstream open-source projects (Iceberg, Trino, Spark, Open Lineage) where relevant
Build self-serve tooling and abstractions that allow data engineering teams to reliably run thousands of pipelines per day against our lakehouse
Collaborate with data engineers, analysts, and infrastructure teams to define the roadmap for our lakehouse architecture and shape how the company manages analytic data at scale
Who You Are:
You have a BS/MS/PhD in Computer Science, Engineering, or a related field, or equivalent professional experience
You have deep, production-grade experience with one or more of Apache Iceberg, Trino, or Apache Spark, ideally demonstrated through significant open-source contributions: merged PRs, committer status, or PMC membership on projects
You have built or operated large-scale distributed data systems
You have a solid grasp of query planning, columnar file formats (Parquet, ORC), and table format internals (snapshots, manifests, partition evolution)
You are fluent in Java, Scala or Go and comfortable with Python for pipeline tooling
You have experience deploying and running data infrastructure on Kubernetes in cloud environments
The company offers a competitive salary and equity package, and may include variable compensation. Actual compensation is based on factors such as the candidate's skills, qualifications, and experience. In addition, the company offers a wide range of best in class, comprehensive and inclusive employee benefits for this role including healthcare, dental, parental planning, and mental health benefits, a 401(k) plan and match, paid time off, fitness reimbursements, and a discounted employee stock purchase plan.
The reasonably estimated yearly salary for this role at the company is: $130,000 — $300,000 USD.
About the company: the company is the leading observability and security platform for the AI era, providing businesses with unified visibility across the technology stack to manage complexity at scale. It brings applications, infrastructure, data, models, and security into one place, using AI to detect and resolve issues before they impact customers. Trusted globally by Fortune 500 companies and high-growth AI leaders, the company enables businesses to move faster with clarity and confidence. Learn more about #DatadogLife on Instagram, LinkedIn, and the company Learning Center.
Equal Opportunity at the company: the company is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and other characteristics protected by law. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. Here are our Candidate Legal Notices for your reference.
#J-18808-Ljbffr
  • New York, New York, United States

Sprachkenntnisse

  • English
Hinweis für Nutzer

Dieses Stellenangebot stammt von einer Partnerplattform von TieTalent. Klick auf „Jetzt Bewerben”, um deine Bewerbung direkt auf deren Website einzureichen.