Senior Hadoop Software Engineer

At eBay, we're more than a global ecommerce leader - we're changing the way the world shops and sells. Our platform empowers millions of buyers and sellers in more than 190 markets around the world. We're committed to pushing boundaries and leaving our mark as we reinvent the future of ecommerce for enthusiasts.

Our customers are our compass, authenticity thrives, bold ideas are welcome, and everyone can bring their unique selves to work - every day. We're in this together, sustaining the future of our customers, our company, and our planet.

Join a team of passionate thinkers, innovators, and dreamers - and help us connect people and build communities to create economic opportunity for all.

The team builds and evolves eBay's next-generation lakehouse platform, powering large-scale analytics across the business. With a focus on Spark stability, usability, and performance, the team enhances Iceberg capabilities and develops internal storage and metadata services that form the foundation of eBay's big data ecosystem.

As a Software Engineer, you will help design, optimize, and operate distributed data systems at global scale. You will partner with engineers across regions to improve platform reliability, address production challenges, and strengthen the performance of critical analytics infrastructure. This role follows eBay's hybrid work model, with three days per week in the office.

• Improve Spark stability, usability, and performance to support reliable, large-scale data processing.

• Extend and enhance Iceberg capabilities to enable scalable and efficient lakehouse operations.

• Design and build internal storage and metadata services that power eBay's lakehouse architecture.

• Troubleshoot and resolve production issues, reducing downtime and improving system resilience.

• Enhance monitoring, logging, and alerting to proactively identify and address system and performance risks.

• Collaborate with global engineering teams to solve complex distributed systems challenges and optimize platform performance.

• Contribute to the evolution of eBay's Hadoop and big data ecosystem, supporting analytics at global scale.

• 5+ years of software engineering experience with strong programming skills in Java or Scala.

• Deep knowledge of Apache Spark internals, including Catalyst optimizer, Tungsten execution engine, shuffle mechanics, Adaptive Query Execution (AQE), and memory management.

• Strong understanding of file formats such as Apache Parquet and table formats including Apache Iceberg, Delta, or Apache Hudi, including snapshots, manifests, partition specifications, and table metadata.

• Experience working with Linux/Unix systems, including shell scripting and system commands, and an understanding of networking principles in distributed environments.

• Proven ability to diagnose and resolve complex defects and performance issues in distributed systems using strong analytical and problem-solving skills.

• Experience designing and implementing distributed systems is a plus.

• Familiarity with the Hadoop ecosystem, including HDFS, HBase, or YARN, and contributions to open-source projects such as Apache Spark or Apache Iceberg, are advantageous.

#LI-CH2

Additional Details

eBay is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, sex, sexual orientation, gender identity, veteran status, and disability, or other legally protected status.If you have a need that requires accommodation, please contact us We will make every effort to respond to your request for accommodation as soon as possible. View our accessibility statement to learn more about eBay's commitment to ensuring digital accessibility for people with disabilities.

We use cookies to enhance your experience and may use AI tools for administrative tasks in the hiring process. xcfaprz To learn how we handle your personal data and use AI responsibly, please visit ourTalent Privacy Notice, Privacy Center, and AI Hiring Guidelines.