Data Engineer responsible for designing and maintaining data pipelines using Azure components and technologies. A role to ensure seamless data flow and optimize ETL processes.
Responsibilities
Design, build, and maintain efficient, reusable, and reliable data pipelines.**Ensure data flow between different systems and databases is seamless and error-free.
Develop and maintain ETL (Extract, Transform, Load) processes to aggregate data from various sources.**Ensure data quality and integrity throughout the ETL process.
Optimize data pipelines and queries for performance.**Identify and resolve bottlenecks in data processing workflows.
Requirements
Experience - 2 years or above
Qualification - B.Tech, MTech
Must have strong knowledge & hands on in Apache Spark and Python programming, working with Delta Tables, Experience in Databricks is must
Strong SQL Skills: Data modelling, Developing SQL Store Procedures, Functions, Dynamic SQL queries, Joins
Must be aware of development of components for data fetching from APIs
Strong knowledge of Data Warehousing Concepts.
Hands on experience in ingesting data from various data sources and data types & file types
Should have good knowledge on development lifecycle , best practice, and coding standards. Should be able to help team members, review the code.
Good knowledge in Azure DevOps, understanding of build and release pipelines
Consultant/Senior Consultant developing transportation planning solutions for Steer North America. Collaborating on multimodal planning and managing client projects across North America.
Senior Consultant working on transport planning for major developments in London. Collaborating with clients and multi - disciplinary teams to deliver efficient and sustainable projects.
Media Consultant focusing on outside sales for Miami Radio market. Responsible for generating new clients and managing existing accounts while collaborating with a sales team.
Senior Lead Business Execution Consultant at Wells Fargo, leading enterprise initiatives and strategic planning. Collaborating with senior leaders to improve performance and culture.
Consultant focusing on health mental projects for corporate clients at Mental Clean. Engaging strategic partnerships to enhance organizational well - being.
Consultor de Sinistro I managing inspections of client insurance claims in São Paulo. Engaging with clients and service providers to ensure compliance and quality assurance standards.
Consultor(a) de Societário in a tech company focused on corporate governance. Involving legal documentation and compliance duties with corporate entities.
Commission and incentive System Consultant at HCSC responsible for report development and project management. Ensure accuracy in payment processes and mentor technical staff.
Senior Digital Analytics Consultant at fifty - five delivering data consultancy projects across clients in Hong Kong. Engaging in digital marketing optimization and ML - powered initiatives.
Digital Analytics Consultant managing data - led projects in marketing analytics for clients at fifty - five. Collaborating with experts to optimize data collection and marketing strategies.