Big Data Engineer with Hive, Spark, and Scala - Contract W26AM City • Phoenix, Arizona, United States
Dieses Stellenangebot ist nicht mehr verfügbar
Big Data Engineer with Hive, Spark, and Scala - Contract W2
6AM City
- Phoenix, Arizona, United States
- Phoenix, Arizona, United States
Über
Qualifications
Minimum 7 years experience as a Big Data Engineer or Scala Developer, with a strong focus on big data technologies.
Proficient in Scala programming language, including experience with functional programming concepts, type systems, and concurrency.
Hands‑on experience with big data technologies, such as Apache Hadoop, Apache Spark, Apache Kafka, and Apache Hive.
Knowledge of data processing patterns and architectural patterns used in big data systems, such as batch processing, stream processing, and lambda architecture.
Familiarity with data modeling techniques and experience in designing efficient data storage solutions, including relational databases, NoSQL databases, and data lakes.
Strong analytical and problem‑solving skills, with ability to identify and address performance bottlenecks and data quality issues.
Excellent communication and collaboration skills, ability to work cross‑functionally and translate technical concepts to non‑technical stakeholders.
Experience with cloud‑based big data platforms, such as Amazon EMR, Google Cloud Dataproc, or Microsoft Azure HDInsight.
Familiarity with data visualization and business intelligence tools, such as Tableau, Power BI, or Looker.
Knowledge of machine learning and data science techniques, and experience in integrating them into big data pipelines.
Exposure to DevOps practices and tools, such as Docker, Kubernetes, or CI/CD pipelines.
Responsibilities
Design and implement scalable and efficient big data processing pipelines using Scala and big data technologies like Apache Spark and Apache Hive.
Develop and maintain batch and real‑time data ingestion, transformation, and processing workflows.
Integrate and optimize big data technologies, such as Apache Hadoop, Apache Spark, Apache Kafka, and Apache Hive, to create a robust and high‑performing data ecosystem.
Write efficient and optimized Scala code for data processing and transformation tasks using Spark's DataFrame and Dataset APIs.
Develop and maintain complex SQL queries and data transformations using Apache Hive.
Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
Ensure data quality, consistency, and security throughout the data lifecycle, implementing best practices for data governance and compliance.
Optimize the performance and scalability of big data systems, leveraging techniques like partitioning, indexing, and caching.
Automate and streamline data processing tasks using tools like Apache Airflow, Jenkins, or custom scripts.
Stay up‑to‑date with the latest trends and advancements in the big data and Scala technology landscape, and recommend improvements to the company's data architecture.
#J-18808-Ljbffr
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot wurde von einem unserer Partner veröffentlicht. Sie können das Originalangebot einsehen hier.