WAHDP - Senior Data Engineer

Scope: The RDDP Program is a 5-year program, which was initiated in the FY2021/2022. It has progressed through the data platform proof-of-concept and early pilot stages. The Program Blueprint defines three distinct waves to transition from the pilot data platform to a secure, reliable, and scalable enterprise data platform service. The following 2 projects are currently being delivered. The Real-time Demand Data Platform (RDDP) Program is responsible for developing and implementing an Enterprise Data Platform, with near real-time capabilities, to support the new State Health Operations Centre (SHOC), while also laying the necessary foundations for a scalable platform to meet the future strategic needs of WA Health. Platform Architecture and Capabilities (PAC) The Platform Architecture and Capabilities (PAC) project aims to establish best practices in architecture, design and operations for the WA Health Data Platform, to support development of a sustainable and scalable enterprise-level data asset. Data Ingestion and Modelling The Data Ingestion and Modelling (DIM) project will deliver the ingestion, integration and modelling of select source system data in the WA Health Data Platform, for the purposes of providing a single view across WA Health of system demand and capacity. It has been identified that these projects need the services of suitably skilled and experienced data engineer services to complete the following: Setting up data ingestion processes into the Snowflake data warehouse, using Oracle GoldenGate and Matillion solutions, and supporting transition to the BAU support team. Implementing automatic job monitoring, triage, and issues resolution activities. Establishing data integrity monitoring and associated process improvements.Ensuring the delivery of robust data ingestion processes and supporting data lifecycle management activities.Developing and supporting data modelling activities.It is expected that suitable data engineering resources will Extensive experience in delivering robust and scalable data ingestion pipelines using both near real time (Oracle GoldenGate) and batch (Matillion) solutions.Experience in data warehouse monitoring and data lifecycle management activities utilising Snowflake warehousing.Experience in data modelling techniques, such as Data Mesh, Inmon, Kimball. Demonstrated experience in cloud-based Data Warehousing as a Service (DWaaS) using Microsoft Azure, Snowflake, Matillion, Oracle GoldenGate, AWS. Advanced technical experience in SQL