Senior Data Engineer/Architect - DatabricksGreystar • Columbia, South Carolina, United States

This job offer is no longer available

Senior Data Engineer/Architect - Databricks

Greystar

Columbia, South Carolina, United States

Columbia, South Carolina, United States

About

Job Description Summary Greystar’s D2AI team is responsible for the platforms, processes, and practices that power AI across the organization. This role goes beyond traditional delivery—your decisions influence how data is transformed into intelligent, scalable solutions used by teams company‑wide. We require AI fluency because this role sits at the intersection of data, technology, and business outcomes. That means understanding how AI systems are designed and operationalized, using AI‑enabled tools in day‑to‑day work, and partnering effectively with engineering, analytics, and business teams to ensure AI solutions are reliable, responsible, and impactful. What You Will Do Hands‑on development of Azure SQL, Cosmos DB, Databricks, and Azure OpenAI—develop and unit test database and AI pipeline code, including T‑SQL, stored procedures, functions, views, and LLM prompt orchestration layers. Own and maintain the Databricks data ingestion & output pipelines for end programs such as greystar.com and Microsoft Customer Insights CDP platform, including Delta Lake table optimization, schema evolution, and medallion architecture (Bronze/Silver/Gold) design. Architect and maintain AI‑ready data structures—clean, well‑typed, and optimized for feature engineering and model consumption. Participate in the design of databases and feature stores, applying normalization or denormalization as appropriate to support both operational and ML workloads. Create, deploy, and maintain ADF and Databricks Workflows pipelines, adhering to Greystar's standards and documented best practices. Perform analysis of complex data and document findings, leveraging AI‑assisted tooling and notebooks to surface insights faster. Prepare and curate data for prescriptive and predictive modeling—including feature extraction, data labeling pipelines, and training/test dataset construction. Combine raw data from disparate external sources; build and support complex ingestions including real‑time streaming (Event Hub) and batch patterns. Collaborate closely with data scientists, ML engineers, and application developers consuming the data—ensuring outputs are well‑documented, versioned, and model‑ready. Integrate and support AI/ML model outputs back into data products—scoring pipelines, inference result storage, and feedback loops for model monitoring. Play a direct role in the maintenance, technical support, documentation, and administration of databases and Databricks environments, including Unity Catalog governance. Ensure standards are followed by participating in code reviews—including review of AI prompt logic, notebook structure, and pipeline configuration. Act as a thought leader: bring forward new ideas in AI, data engineering, and LLM integration—and then build them. Technical Qualifications 6+ years relevant and progressive data engineering experience. Deep technical knowledge and experience in Microsoft Azure architecture, including Azure PaaS databases, Synapse, ADF pipelines, Azure functions, Event Grids, etc. 3+ years of experience with Cosmos DB. 3+ years of experience with Data Bricks. Hands‑on skills working with data pipelines using SQL and No‑SQL databases. Minimum of 1 year of relevant experience working with Azure Data Lakes Gen 2. Experience with Power Platform / PowerBI. Experience in engineering practices such as code refactoring, design patterns, CI/CD, and highly scalable data applications. Experience developing batch ETL pipelines; real‑time pipelines are a plus. Knowledge of advanced data engineering concepts such as dimensional modeling, ETL, data governance, data warehousing, structured and unstructured data. Good experience with Python, machine learning frameworks, and statistics. Experience / familiarity with D365 Customer Insights platform / Dataverse is a plus. Overall Qualifications Knowledge of Agile software development process. Excellent problem‑solving skills and experience. Strong communication and collaboration skills. Self‑starter attitude and the ability to make decisions with minimal guidance from others. Innovative and passionate about your work and the work of your teammates. Ability to comprehend and analyze operational systems and ask appropriate questions to determine how to improve, migrate or modify the solution to meet business needs. Working Conditions Occasional time outside of core hours depending on priorities and deliverables. Some travel (with advance notice) may be required for training, team meetings or other events. Required Licenses or Certifications Databricks or other certifications are nice to have but not required. Education Bachelor’s Degree in computer science, information technology, business management, information systems, or equivalent experience. Advanced degree preferred. What the Right Candidate will Enjoy! 100% remote flexibility. Competitive pay, benefits, and overall compensation packages. The chance to be part of a technology team for a thriving organization that prioritizes accountability, respect, and operational excellence. The opportunity to join a thriving, highly visible organization during its technology transformation. Salary Range The salary range for this position is $150,000 - $170,000 USD annually. Additional Compensation Many factors go into determining employee pay within the posted range including business requirements, prior experience, current skills and geographical location. Corporate positions may be eligible to participate in a quarterly or annual bonus program based on individual and company performance. Benefits Competitive Medical, Dental, Vision, and Disability & Life insurance benefits. Low (free basic) employee medical costs for employee‑only coverage; costs discounted after 3 and 5 years of service. Generous paid time off—15 days of vacation, 4 personal days, 10 sick days, and 11 paid holidays. Birthday off after 1 year of service; additional vacation accrued with tenure. 6‑Week paid sabbatical after 10 years of service (and every 5 years thereafter). 401(k) with company match up to 6% of pay after 6 months of service. Paid parental leave and lifetime fertility benefit reimbursement up to $10,000 (includes adoption or surrogacy). Employee assistance program. Critical illness, accident, hospital indemnity, pet insurance and legal plans. Charitable giving program and benefits. EEO Statement Greystar will consider for employment qualified applicants with arrest and conviction records.
#J-18808-Ljbffr

Columbia, South Carolina, United States

Languages

English

Notice for Users

This job was posted by one of our partners. You can view the original job source here.

Find similar jobs