Senior Data Engineer for the MDM Data Product Team at Avenga. Responsible for provisioning data from master data customer and product systems.
About the role
- Data Engineer responsible for backend data engineering tasks, developing ETL processes for legal data integration. Collaborating with teams to ensure efficient data pipelines and governance.
Responsibilities
- Design, construct, and maintain data pipelines (ETL processes) to integrate large volumes of legal data efficiently and securely from various sources
- Develop data lake and data warehouse solutions, implementing best practices for data ingestion, storage, and retrieval
- Establish and deploy comprehensive data governance and security frameworks including data access controls, compliance measures, and data classification
- Collaborate with cross-functional teams, including analysts, attorneys, and IT professionals, to understand data requirements and design backend solutions that meet their needs
- Assess and prioritize data initiatives based on business objectives, best practices, and partner/practice need to ensure alignment with strategic goals
- Design and implement testing strategies for data pipelines, validating data integrity, accuracy, and performance throughout the workflow
- Create and maintain thorough documentation for data architectures, pipelines, and transformations, and processes to ensure transparency and knowledge sharing
- Build and optimize automated data workflows to streamline ingestion, transformation, and processing, reducing manual effort and improving efficiency
- Optimize data storage and retrieval processes to improve performance and scalability, leveraging cloud-based technology such as Azure
- Participate in and contribute to data quality enhancement planning and implementation for new projects
- Stay current with the latest industry trends and best practices in data engineering and analytics, continuously evaluating and implementing new tools and techniques to enhance our data infrastructure
- Provide technical support and troubleshooting assistance for data-related issues, working proactively to identify and resolve potential problems
- Collaborate with cybersecurity and compliance teams to ensure data security and regulatory compliance
Requirements
- Bachelor’s degree in Computer Science, Engineering, or a technology-related field
- 7+ years of related work experience, preferably in a legal or regulatory environment
- Proven experience leading data engineering projects, including requirements gathering, architecture design, development, testing, and deployment
- Experience with Microsoft Fabric or a combination of Azure Synapse Analytics, ADLS Gen 2, Synapse SQL Pools, Azure Data Factory (ADF) and/or other Azure data platforms
- Experience with the Microsoft Power Platform, such as Power BI, Power Automate, Power Apps, or other similar tools
- Strong SQL Experience including complex queries, stored procedures, SSIS, SSRS
- Experience building data pipelines using Python/PySpark
- Proven experience designing and implementing data architectures, including system requirements, technology selection, and governance
- Expertise in end-to-end multi-layered data architectures (e.g., bronze/silver/gold), managing data from raw ingestion to business-ready datasets while ensuring quality, lineage, and documentation
- Strong data modeling skills, including conceptual, logical, and physical models for optimized storage and retrieval
- Proven ability to test data pipelines for accuracy, consistency, and performance, including unit and integration testing
- Experience in migrating data from on-premises databases and servers to Azure Cloud environments is preferred
- Experience with version control; knowledge and experience with GitHub is a plus
- Ability to present technical ideas in business-friendly and user-friendly language.
Benefits
- Health insurance
- 401(k) matching
- Flexible work arrangements
- Professional development opportunities
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Tech skills
Location requirements
Data Warehouse Modelling Engineer designing and maintaining data models using Data Vault 2.0 for iGaming industry. Collaborating with stakeholders and optimizing data models in a hybrid work environment.
Senior Data Engineer driving impactful data solutions for the climate logistics startup HIVED's core data platform. Collaborating with cross - functional squads to enhance analytics and delivery.
Data Engineer developing and maintaining CRE forecasting infrastructure for Cushman & Wakefield. Collaborates with senior economists and technical teams to ensure high - quality data solutions.
Data Engineer at PwC, engaging with Azure cloud services to enhance data handling and integrity. Responsibilities include pipeline optimizations, documentation, and collaboration with stakeholders.
Data Engineer Manager at PwC focusing on building data infrastructure and solutions. Leading data engineering projects to transform raw data into actionable insights and drive business growth.
Junior Data Engineer at OneMarketData focusing on data quality and integrity in financial datasets. Collaborating with senior analysts and assisting in data management and analysis tasks.
Data Engineer Manager leading robust data solutions for PwC's clients. Transforming raw data into actionable insights and coaching team members.
Senior Data Engineering Analyst developing and implementing data solutions. Collaborating in a diverse environment focused on data processing and analysis for clients' digital transformation.
Principal Software Engineer in Threat Data Platform developing AI - driven tools for threat intelligence automation. Collaborating on robust data pipelines for PANW’s product ecosystem.