Data Engineer building and maintaining scalable data pipelines for AI Search Infrastructure at You.com. Collaborating across teams to ensure data quality and enable AI capabilities.
About the role
- Data Engineer creating and implementing Big Data applications at Absa with business stakeholders and technology leaders. Involves ETL/ELT pipeline design, data automation, and team mentorship.
Responsibilities
- Design, implement, and maintain scalable, high-performance ETL/ELT pipelines for structured and unstructured data
- Understand the technical landscape and bank wide architecture to effectively design & deliver data solutions (architecture, pipeline etc.)
- Create & Maintain CI / CD Pipelines (authoring & supporting CI/CD pipelines within Git Hub Actions and deploy to production)
- Automate data applications using orchestration tools
- Debug and improve existing source code
- Support the continuous optimisation, improvement & automation of data pipelines
- Coach & mentor other data engineers
- Conduct peer reviews, testing, problem solving within the team
- Identify technical risks and mitigate these (pre, during & post deployment)
- Update / Design all application documentation aligned to the organization technical standards and risk / governance frameworks
- Create business cases & solution specifications for various governance processes (e.g. CTO approvals)
- Participate in incident management & DR activity – applying critical thinking, problem solving & technical expertise to get to the bottom of major incidents
- Deliver on time & on budget (always)
Requirements
- BSc Honours, BCom Honours, BEng, BBusSc in Computer Science, Information Systems or any Information Technology qualification that is at NQF level 8 or higher
- 3 or more years of experience as a Data Engineer
- Understanding of and experience of using Big Data technologies (Hadoop) is essential
- Experience with designing and developing Scala/ Apache Spark data applications
- Understanding of Linux and Bash scripting
- Understanding of Git and GitHub Actions
- Experience in CA Wade or any other orchestration tool
- Great SQL skills
- Ability to work in either an Agile or project methodology to deliver tasks
- Datawarehouse experience is beneficial but not a must
- Cloud skills (AWS preferable) and Databricks are beneficial but not a must
Benefits
- Equal opportunity employer
- Preference to designated groups for demographics
- Contribution to diversity of workforce
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Engineer developing and managing technology - based data solutions for clients in different industries in Greece. Participating in software development lifecycle within Agile team setting.
Data Architect leading design and governance of high - quality data architectures for clients. Collaborating with engineering teams and stakeholders to transform business challenges into scalable data solutions.
Data Engineer supporting vehicle buying and selling solutions through integration pipelines. Collaborating with teams to build digital vehicle platforms and optimize data processes in São Paulo.
Senior Advanced Data Engineer designing and optimizing data architecture for Honeywell. Collaborating with cross - functional teams to drive data - driven decision - making and operational efficiency.
Senior Data Engineer building and operating data platforms at bsport for analytics and AI/ML. Collaborating with Data team to enrich data layers and maintain platform observability.
Principal HR Data Engineer specializing in Microsoft Azure and Databricks Lakehouse platforms. Responsible for designing, implementing, and maintaining scalable data pipelines and architectures for analytics.
Manager, Business Solutions Data overseeing data processing lifecycle and team development at Ryan. Partnering with clients and teams to drive data solutions and operational excellence.
Data Engineer creating data pipelines at Data Wow, working closely with Data Scientists and Analysts to deliver data products.
Senior Data Engineer at Trainline responsible for data pipelines and insightful analytics. Collaborate cross - functionally to enable impactful data - driven decisions.