If you are an experienced and passionate Data Engineer who enjoys working in a highly entrepreneurial work environment, this is the opportunity for you. Excellent career opportunity to manage, design, develop and deliver the data integration environment for a fastgrowing organisation in Malaysia!
- Bachelors in IT, Computer Science or Engineering;
- At least 4 years of experience in data warehouse, operational data store and large scale data architecture implementations in Unix/ Windows;
- At least 3 years of big data technologies (e.g. Hadoop, Hive, Spark);
- At least 3 years of experience in doing data modeling, data mart designing and implementing;
- Strong handson development experience with Talend ETL (or similar tools) to transform complex data structure in a multidata source environment;
- Familiar with ETL/ ELT framework, data warehouse concepts, data management framework and data lifecycle processes;
- Strong knowledge in handling different database technologies (RDBMS, NoSQL and Columnar);
- Extensive experience working with data visualization (e.g. Tableau);
- Strong understanding of programming and scripting languages like Python, Java, Shell and PLSQL;
- Proven track record of solving challenging problems by applying advanced analytics;
- Good analytical ability, conceptual thinking, planning and organisational skills.
- Experience in the financial sector (e.g. insurance/banks) will be preferred.
- Design, develop, document and implement endtoend data pipelines and data integration processes, both batch and realtime;
- Perform data analysis, data profiling, data cleansing, data lineage, data mapping and data transformation;
- Develop ETL / ELT jobs and workflows, and deployment of data solutions;
- Monitor, recommend, develop and implement ways to improve data quality including reliability, efficiency and cleanliness to optimize and finetune ETL/ ELT processes;
- Recommend, execute and deliver best practices in data management and data lifecycle processes, including modular development of ETL/ ELT processes, coding and configuration standards, error handling and notification standards, auditing standards, and data archival standards;
- Prepare test data, create and execute test plans, test cases and test scripts;
- Collaborate with different stakeholders to understand data needs, gather requirements and implement data solutions to deliver business goals;
- Provide technical support for any data issues and change requests, document all investigations, findings, recommendations and resolutions.