Architect leading initiatives in data platform engineering at F5, solving challenges in unstructured data and enhancing governance frameworks.
About the role
- Data Engineer responsible for designing and maintaining enterprise data warehouse for various projects. Collaborating with stakeholders to ensure efficient data flow and integration.
Responsibilities
- Design, develop, and maintain scalable Synapse pipelines for data ingestion, processing, and transformation in Azure data platform.
- Develop and optimize T-SQL and Python-based data processing scripts and integrate them with Synapse pipelines for ETL tasks and data transformations.
- Design and develop mapping dataflow solutions for complex data transformations.
- Develop and maintain data storage solutions, ensuring efficient data retrieval and storage on Azure Data Lake and databases.
- Implement and maintain API integrations, including REST APIs, for seamless data flow between systems.
- Design and test the end-to-end process for data ingestion, data transformation, data cleansing, data delivery, data quality and data timeliness.
- Implement and ensure data security and privacy measures, complying with industry standards and regulations.
- Monitor and optimize the performance of Azure data services, Azure Synapse pipelines, Function Apps, and Logic Apps.
- Improve the scalability, efficiency, and cost-effectiveness of data pipelines.
Requirements
- Bachelor's degree or equivalent in the field of Computer Science or Engineering
- 3+ years of Proven experience as a Data Engineer, specifically working with Microsoft Azure technologies.
- Proficient in T-SQL, Pyspark, Mapping Dataflow and Python programming language
- Hands-on experience with Azure data services, including Azure Data Factory, Azure Data Lake, Azure Function App, Azure Logic App, Azure SQL Database, Azure Synapse Analytics, mapping dataflow, and Event Hub.
- Understanding of data warehousing concepts and best practices.
- Experience with Data modeling.
- Experience in working with structured, semi-structured and unstructured data.
- Excellent written and verbal communication skills
- Excellent multi-tasking and organizational skills
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Architect designing and implementing data architectures supporting analytics and ML for federal clients. Collaborating with teams to translate mission needs into robust data solutions.
IT Data Engineer developing data pipelines and integrations for Scanfil Group's global IT organization. Collaborating across teams to enhance data solutions and reporting capabilities.
Data Engineer developing Azure data solutions at PwC New Zealand. Responsibilities include data quality monitoring, pipeline development, and collaboration with stakeholders in a supportive environment.
Senior Data Engineer designing and implementing the Enterprise Data Platform at Stellix. Focusing on analytics and insights with a growth path to Principal Data Engineer or Data Architect.
R&D Data Engineer at DXC, transforming complex data into digital assets for global analytics and Smart Lab solutions. Collaborating on ELN and LIMS tools for enhanced data management.
Senior Data Engineer at mobility AI company designing large - scale data processing pipelines. Leading technical decisions and mentoring junior engineers in data architecture.
Data Engineer role focusing on data pipelines and processing at 42dot, a mobility AI company. Responsibilities include data collection, schema management, and pipeline monitoring.
Senior Data Engineer at Booz Allen building advanced tech solutions for mission - driven projects. Utilizing data engineering activities, pipelines, and platforms for impactful data insights.
Senior Software Engineer contributing to Workday's AI/MLOps cloud ops platform. Involves data ingestion, computation, and generation of curated data sets with modern technologies.