Skill diminta
AWSCommunicationMariaDBMySQLPandasPostgreSQLPower BIPythonSAPSQLSnowflakeTableau
Deskripsi
Deskripsi pekerjaan Data Engineer Insignia
You’ll be part of a hands-on consulting team building data platforms that clients actually run in production. This isn’t a support role — you’ll own workstreams, collaborate with client engineers, and be expected to ship. You’ll work across the full data lifecycle: ingestion, modeling, orchestration, and serving — on real enterprise-scale data.
Responsibilities
- Design, develop, and maintain end-to-end data pipelines for large-scale processing on cloud platforms (primarily Databricks on AWS)
- Build and optimize pipelines using Databricks — Delta Lake, Auto Loader, Workflows, and SQL Warehouses
- Implement medallion architecture (bronze/silver/gold) with proper data modeling practices
- Collaborate with client stakeholders and internal team to ensure data quality and deliver actionable insights via BI tools
- Monitor and troubleshoot data workflows in production, resolving issues in a structured and timely manner
- Write clean, maintainable PySpark and SQL code that other engineers can read and extend
- Support technical scoping and solution design during pre-sales or project kickoff phases
Requirements
- Must Have
- Min. 2 years of experience as a Data Engineer, preferably in a consulting or multi-client environment
- Strong proficiency in Python (PySpark, Pandas) and SQL — CTAS, Window Functions, CTEs
- Hands-on Databricks experience: notebooks, Delta Lake, Workflows, and SQL Warehouses
- Experience with AWS data services: S3, Glue, RDS/Aurora, or Redshift
- Solid understanding of data modeling — star schema, medallion architecture, normalization
- Version control with GitHub or GitLab in a team environment
- Strong communication skills — ability to explain technical decisions to non-technical stakeholders
- Nice to Have
- Databricks Unity Catalog, Auto Loader, Delta Sharing, or lakehouse concepts
- Snowflake experience — schema design, data sharing, cost optimization
- SAP data extraction via OData, PyRFC, or similar connectors
- BI tools: Power BI, QuickSight, Tableau, or Databricks AI/BI
- Databricks Certified Data Engineer Associate or above
- API-based data ingestion patterns
- Hands-on experience with MySQL, PostgreSQL, or MariaDB