Lead Data Engineer

About Client: Client is a company that helps other businesses use technology to improve and grow. They provide services like creating computer programs, managing computer systems, and making sure everything is secure. Client works with different industries like telecommunications, healthcare, banking, and retail to help them use new and exciting technologies like artificial intelligence, cloud computing, and the Internet of Things. These technologies can make businesses more efficient and innovative. They have offices and people working all around the world, and their team of experts helps businesses solve their technology challenges. Client also cares about making a positive impact on society and the environment, so they support social causes and work to conserve resources.

Rate Range: $60-$63/Hr

Job Description: Responsible for designing, developing, and managing data engineering solutions to enable the efficient collection, transformation, and storage of data. The Lead Data Engineer will work closely with cross-functional teams, including data architects, data scientists, and business analysts, to build and optimize data pipelines, ensuring data is accessible, reliable, and of high quality for analytical and operational purposes. Key Responsibilities:

Architect, design, and develop scalable and high-performance data pipelines for data ingestion, transformation, and storage using modern cloud technologies like Azure Data Factory, Databricks, and SQL-based solutions. Integrate and manage structured, semi-structured, and unstructured data sources from multiple platforms, ensuring data is appropriately cleansed, transformed, and loaded (ETL/ELT processes). Design and develop data ingestion pipelines utilizing Delta Live Tables (DLT) and Structured Streaming to handle both batch and real-time data ingestion requirements. Collaborate with data engineers, analysts, and business teams to understand data needs and translate those into technical solutions. Optimize data pipelines and workflows to ensure high performance, scalability, and cost-efficiency across large-scale data environments. Lead the design and development of cloud-based data solutions on platforms such as Azure, ensuring data systems are reliable, secure, and compliant with industry standards. Implement data quality checks, validation rules, and data governance processes to maintain data integrity and consistency. Provide technical leadership and mentorship to junior and mid-level data engineers, fostering a culture of continuous learning and improvement. Set up monitoring systems for data pipelines and troubleshoot data issues to ensure smooth operations and prompt issue resolution. Document data engineering processes, architectures, and workflows to ensure transparency, traceability, and maintainability. Stay up to date with new technologies, tools, and best practices in data engineering, and continuously improve the performance, scalability, and efficiency of data systems. Skills/Qualifications:

Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field. 8+ years of experience in data engineering or data-related roles. 5+ years of experience in designing and implementing data pipelines using Azure Data Factory, Databricks, SQL, and other relevant technologies. Strong experience with building, managing, and optimizing ETL/ELT workflows. Strong experience in designing and developing data ingestion pipelines using Delta Live Tables (DLT) and Structured Streaming. Proficiency in programming languages such as Python, Scala, and SQL for data engineering tasks. Experience in handling large datasets and working with distributed computing frameworks (e.g., Spark, Databricks). Familiarity with data warehousing concepts, star/snowflake schemas, and cloud-based storage solutions. Azure Data Engineer, Azure Solutions Architect, or related cloud certifications are highly desirable. Demonstrated ability to lead data engineering teams, guide technical projects, and mentor junior engineers. Excellent analytical and troubleshooting skills, with a focus on optimizing performance and resolving data pipeline issues. Strong interpersonal and communication skills, with the ability to effectively communicate technical concepts to both technical and non-technical stakeholders.

About ApTask: ApTask is a leading global provider of workforce solutions and talent acquisition services, dedicated to shaping the future of work. As an African American-owned and Veteran-certified company, ApTask offers a comprehensive suite of services, including staffing and recruitment solutions, managed services, IT consulting, and project management. With a focus on excellence, collaboration, and innovation, ApTask provides unparalleled opportunities for professional growth and development. As a member of the ApTask team, you will have the chance to connect businesses with top-tier professionals, optimize workforce performance, and drive success across diverse industries. Join us at ApTask and be part of our mission to empower organizations to thrive while fostering a diverse and inclusive work environment.

Applicants may be required to attend interviews in person or by video conference. In addition, candidates may be required to present their current state or government issued ID during each interview. Candidate Data Collection Disclaimer: At ApTask, we prioritize safeguarding your privacy. As part of our recruitment process, certain Personally Identifiable Information (PII) may be requested by our clients for verification and application purposes. Rest assured, we strictly adhere to confidentiality standards and comply with all relevant data protection laws. Please note that we only collect the necessary information as specified by each client and do not request sensitive details during the initial stages of recruitment. If you have any concerns or queries about your personal information, please feel free to contact our compliance team at businessexcellence@aptask.com .

Applicant Consent: By submitting your application, you agree to ApTask's (www.aptask.com) Terms of Use and Privacy Policy , and provide your consent to receive SMS and voice call communications regarding employment opportunities that match your resume and qualifications. You understand that your personal information will be used solely for recruitment purposes and that you can withdraw your consent at any time by contacting us at 732-355-8000 or help@aptask.com. Message frequency may vary. Msg & data rates may apply.