Skill diminta
AirflowApacheSQLSQL Server
Deskripsi
Tanggung Jawab
- Develop and maintain
- data pipelines
- from multiple data sources
- Perform
- ETL (Extract, Transform, Load)
- processes for data integration
- Process and manage large volumes of data efficiently
- Develop and optimize database queries using Microsoft SQL Server
- Monitor and troubleshoot data pipelines
- Collaborate with Data Analysts, Business Analysts, and Developers for data requirements
- Perform data validation and ensure data quality
- Create documentation for data pipeline and ETL processes
- Optimize pipeline performance and database queries
Kualifikasi
- Minimum
- 1 year of experience as a Data Engineer
- Experience in building
- data pipelines
- Must have experience using Microsoft SQL Server
- Experience with
- ETL (Extract, Transform, Load)
- processes
- Familiar with
- Data Warehouse
- and
- Data Integration
- concepts
- Strong SQL skills (Join, Stored Procedure, Query Optimization)
- Experience using Informatica is a plus (Preferred)
- Experience with other ETL tools is an advantage
- Strong analytical and problem-solving skills
- Able to work independently and in a team environment
- Nice to Have
Experience with ETL tools such as
- SQL Server Integration Services
- Pentaho Data Integration
- Apache Airflow
- Experience handling large datasets
- Experience in Banking or Financial industry (preferred)