Remote Data Engineer – GCP, BigQuery & Spark

Datumo

Poland, Ohio, United States

Poland, Ohio, United States

À propos

We’re looking for a
Data Engineer
ready to push boundaries and grow with us. Datumo specializes in providing Data Engineering and Cloud Computing consulting services to clients from all over the world, primarily in Western Europe, Poland and the USA. Core industries we support include
e-commerce
,
telecommunications
and life sciences
. Our team consists of exceptional people whose commitment allows us to conduct highly demanding projects. Our team members tend to stick around for more than 3 years, and when a project wraps up, we don't let them go - we embark on a journey to discover exciting new challenges for them. It's not just a workplace; it's a community that grows together! Must-have: ✅ at least
3-4 years
of commercial experience in programming, ✅ proven record with a cloud provider -
Google Cloud Platform, ✅ strong knowledge of
Python, SQL and JVM languages
(Scala or Java or Kotlin), ✅ experience in
BigQuery
data warehousing solution, ✅ in-depth understanding of big data aspects like data storage,
modeling ,
processing ,
scheduling
etc., ✅ understanding of Apache Spark (or similar distributed data processing framework), ✅
data modeling
and
data storage
experience, ✅ ensuring solution quality through
automatic tests, CI/CD
and
code review, ✅ proven
collaboration with businesses, ✅
English
proficiency at min. B2 level, proficient in
Polish. Nice to have: knowledge of dbt, Docker and Kubernetes, Apache Kafka, familiarity with Apache Airflow or similar pipeline orchestrator, another JVM (Java/Scala/Kotlin) programming language, experience in Machine Learning projects, familiarity with one of BI tools: Power BI/Looker/Tableau, willingness to share knowledge (conferences, articles, open-source projects). What’s on offer: 100% remote work, with workation opportunity, onboarding with a
dedicated mentor, benefits:
Medicover Private Medical Care , co-financing of the
Medicover Sport
card, opportunity to
learn English with a native speaker, regular
company trips and informal get-togethers. Development opportunities in Datumo: participation in industry conferences, establishing Datumo's online brand presence, support in obtaining certifications (e.g. GCP, Azure, Snowflake), involvement in internal initiatives, like building technological roadmaps, access to internal technological training repositories. Discover our exemplary project:
IoT data ingestion to cloud The project integrates data from edge devices into the cloud using Azure services. The platform supports data streaming via either the IoT Edge environment with Java or Python modules, or direct connection using Kafka protocol to Event Hubs. It also facilitates batch data transmission to ADLS. Data transformation from raw telemetry to structured tables is done through Spark jobs in Databricks or data connections and update policies in Azure Data Explorer. ☁️ Petabyte-scale data platform migration to Google Cloud The goal of the project is to improve scalability and performance of the data platform by transitioning over a thousand active pipelines to GCP. The main focus is on rearchitecting existing Spark applications to either Cloud Dataproc or Cloud BigQuery SQL, depending on the Client’s requirements and automate it using Cloud Composer. Data analytics platform for investing company The project centers on developing and overseeing a data platform for an asset management company focused on ESG investing. Databricks is the central component. The platform, built on Azure cloud, integrates various Azure services for diverse functionalities. The primary task involves implementing and extending complex ETL processes that enrich investment data, using Spark jobs in Scala. Integrations with external data providers, as well as solutions for improving data quality and optimizing cloud resources, have been implemented. The initiative involves constructing a consumer data platform (CDP) for a major Polish retail company. Datumo actively participates from the project’s start, contributing to planning the platform’s architecture. The CDP is built on Google Cloud Platform (GCP), utilizing services like Pub/Sub, Dataflow and BigQuery. Open-source tools, including a Kubernetes cluster with Apache Kafka, Apache Airflow and Apache Flink, are used to meet specific requirements. This combination offers significant possibilities for the platform. Recruitment process: If you like what we do and you dream about creating this world with us - don’t wait, apply now! I hereby give my consent for the processing of my personal data included in the submitted CV and attached documents by DATUMO sp. z o.o. with its registered office in Warsaw for the purposes of the recruitment process, in accordance with the provisions of Article 6(1)(a) of the General Data Protection Regulation (GDPR), and - if the provided data include special categories of personal data as referred to in Article 9(1) of the GDPR - in accordance with Article 9(2)(a) of GDPR. This consent can be revoked at any time by sending a notification to the email address contact@datumo.pl; however, the withdrawal of consent does not affect the lawfulness of data processing based on the consent before its withdrawal. l declare that have read the information clause available HERE .* I hereby give consent for the storage of my personal data in the Candidates' Database maintained by DATUMO sp. z o.o. if the current recruitment process does not result in collaboration and DATUMO sp. z o.o. would like to contact me in the future for potential collaboration. I declare that I have read the information clause available HERE . Attach resume Max file size 10MB. Uploading... fileuploaded.jpg Upload failed. Max size for files is 10 MB. Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. Contact Apply , don’t leave it all to chance! cv@datumo.pl +48 789 566 177 Dziekońskiego 1 street, 00-728 Warsaw Full name Email Message I hereby give my consent for the processing of my personal data included in the submitted CV and attached documents by DATUMO sp. z o.o. with its registered office in Warsaw for the purposes of the recruitment process, in accordance with the provisions of Article 6(1)(a) of the General Data Protection Regulation (GDPR), and - if the provided data include special categories of personal data as referred to in Article 9(1) of the GDPR - in accordance with Article 9(2)(a) of GDPR. This consent can be revoked at any time by sending a notification to the email address contact@datumo.pl; however, the withdrawal of consent does not affect the lawfulness of data processing based on the consent before its withdrawal. l declare that have read the information clause available HERE .* I hereby give consent for the storage of my personal data in the Candidates' Database maintained by DATUMO sp. z o.o. if the current recruitment process does not result in collaboration and DATUMO sp. z o.o. would like to contact me in the future for potential collaboration. I declare that I have read the information clause available HERE . Attach resume Max file size 10MB. Uploading... fileuploaded.jpg Upload failed. Max size for files is 10 MB. Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. https://datumo.traffit.com/public/form/a/6bedf085208e3c87ec4d1b2a26b761d65754493d
#J-18808-Ljbffr

Poland, Ohio, United States

Compétences linguistiques

English

Avis aux utilisateurs

Cette offre a été publiée par l’un de nos partenaires. Vous pouvez consulter l’offre originale ici.

Trouver des emplois similaires