Support a data platform migration from Amazon Redshift to Databricks by building and maintaining Databricks pipelines and jobs, working with AWS services (S3, Glue, Athena), supporting data lake architecture, and applying data governance (lineage, catalogs, RBAC) while collaborating with stakeholders. 5-6 month engagement requiring 3-5 years of data engineering experience.
AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
WHY JOIN US
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!
ABOUT THE ROLE
We are looking for a Data Engineer to support a strategic data platform migration, moving tables, views, and stored procedures from Amazon Redshift to Databricks. You will build and support Databricks pipelines and jobs, work with AWS services including S3, Glue, and Athena, and apply data governance practices including data lineage, cataloging, and role-based access controls throughout the migration. The role requires 3–5 years of data engineering experience. This is a 5–6 month project engagement.
WHAT YOU WILL DO
- Migrate tables, views, and stored procedures from Amazon Redshift to Databricks.
- Build and support Databricks pipelines and jobs.
- Work with AWS services, including Amazon S3, AWS Glue, and Amazon Athena.
- Support data migration activities within a modern data lake architecture.
- Apply data governance and data management practices throughout the migration.
- Collaborate with project stakeholders to support migration activities.
MUST HAVES
- 3–5 years of experience as a Data Engineer.
- Hands-on experience with Databricks.
- Experience with Amazon Redshift.
- Strong SQL skills.
- Experience with AWS services, including Amazon S3, AWS Glue, and Amazon Athena.
- Experience building and supporting Databricks pipelines and jobs.
- Understanding of data lake architectures.
- Familiarity with data governance and data management practices, including data lineage, data catalogs, and role-based access controls (RBAC).
- Upper-intermediate English level
NICE TO HAVES
- Experience with Databricks Lakebridge transpiling capabilities.
PERKS AND BENEFITS
- Growth without limits: build your skills through mentorship, internal TechTalks, challenging projects, and a dedicated annual learning budget
- Competitive compensation: get recognition that reflects your skills and impact, with regular performance and compensation reviews
- Flexibility: work 100% remotely with flexible hours that support focus, autonomy, and a healthy work rhythm
- Meaningful, modern projects: build impactful products using modern technologies alongside global teams and leading brands
- Collaborative culture: join a supportive environment with zero micromanagement where ideas are welcomed and contributions are recognized
- Well-being & support: access local well-being programs and people-focused support tailored to your location
Similar Jobs
Software
Support a data platform migration from Amazon Redshift to Databricks by migrating tables, views, and stored procedures; build and maintain Databricks pipelines/jobs; work with AWS services (S3, Glue, Athena); enforce data governance, lineage, cataloging, and RBAC; collaborate with stakeholders during a 5-6 month engagement.
Top Skills:
Amazon AthenaAmazon RedshiftAmazon S3Aws GlueDatabricksDatabricks LakebridgeSQL
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Lead Samsara Mexico communications with a 360° strategy across external media, internal comms, and talent brand. Serve as primary media contact, develop localized storylines, manage executive visibility, support customer storytelling, run crisis comms, manage agencies, and measure program impact in coordination with marketing, legal, and global comms.
Top Skills:
LinkedIn
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
The Account Executive will engage with small to medium customers, focusing on IoT sales. Responsibilities include prospecting, closing deals, and building relationships. Requires high-volume cold calling and a collaborative approach.
Top Skills:
SFDC
What you need to know about the Bristol Tech Scene
Along with Gloucester, Swindon and Bath, Bristol is part of the "Silicon Gorge" tech hub, a region in the U.K. renowned for its high-tech and research-driven industries, with a particular emphasis on sustainability and reducing environmental impact. As the European Green Capital, Bristol is home to 25,000 cleantech companies, including Baker Hughes and unicorn Ovo Energy. The city has committed to achieving net-zero emissions within the next decade.
