Overall Purpose of the Job (Please provide a summary of why the job was created and what the main responsibilities are)

The data engineer is responsible for ingesting and transforming a wide range of data across the business into formats that can be used by end-users to develop differentiating business insights.  By structuring  data requirements into logical data constructs, the Data Engineer enables optimal, physical implementation of the data structure by generating data models. A key deliverable is the development of visual reports, dashboards and KPI scorecards.
The role will involve collaborating with internal and external resources to ensure the data analytics team have the required data from the source systems as well as be responsible for the accuracy of data loaded into the Data Warehouse

Education (Formal Qualification required)

Diploma in IT or related field
Bachelor’s degree in (Information Systems, Informatics, Computer Systems or Information Technology) – NQF 7  Or National Diploma / Diploma in Software Development /IT- NQF 6
SQL/ Data engineering certification
Azure Data Certification

Experience (Minimum Experience required - type and number of years- experience can run concurrently)

Job Title / Function	Minimum time spent in job
·   SQL Developer	Year(s)	5
·   BI Developer	Year(s)	5
·   Data Engineering	Year(s)	3
·        Synapse pipelines / Data factory / SSIS	Year(s)	3

Data analysis / data modelling/  report building

• Creates data model designs in accordance with recommended best practice
• Collaborates with team members and business to translate business requirements into data requirements
• Analyzes and profiles the source data to understand data quality issues, relationships, patterns, and rules in the data
• Structures data requirements into logical data constructs based on recommended best practice including ERD models and dimensional models to ensure optimal implementation
• Compiles Source to Target Mapping Specifications with the Business Intelligence Business Analyst (BIBA)
• Ensures complete and correct data by identifying definitive or authoritative sources of data; analyzing source data; and identifying gaps
• Enables physical implementation of the data structure by generating the physical data model from the logical data model
• Cleans and maintains databases by restructuring and deleting old data/stored procedures
• Develops visual reports, dashboards and KPI scorecards using Power BI desktop.
• Connects to data sources, importing and transforming data for Business Intelligence
• Implements row level security on data, demonstrating an understanding of application security layer
• Writes DAX queries in Power BI.
  

ETL and pipeline development

• Assists in designing and implementing scalable and robust processes for ingesting and transforming complex datasets.
• Develops the work packages as per the design to deliver elements of the data warehouse architecture (data ingestion, ETL, etc.,)
• Provides technical support, troubleshooting and upgrades setup or support on ETL and data warehouse related issues
• Analyzes and maps data from source systems to target data warehouse
• Participates in all aspects of quality assurance
• Monitors and maintains ETL pipelines and data warehouse
• Develops the data dictionary

Task tracking and Support Resolution

• Makes use of DevOps to track tasks, effort estimates, effort completed and repos
• Responds timeously to support calls and tracks issues to completion

Continuous improvement

• Owns the development of elements of centralised, automated data improvement initiatives under the guidance of the Team Lead
• Makes increased contributions by broadening and enhancing individual skills.
• Keeps abreast of industry trends and best practice and makes recommendations to management
• Investigates new technologies to improve and optimize standard reporting.

Skill	Weighting Importance (Please note that this should add up to 100%)
·        Data modelling	15%
·        Excellent communication skills	5%
·        Analytical and problem solving	15%
·        Microsoft Power BI	15%
·        Azure Synapse SQL and pipelines	20%
·        Microsoft SQL Server Technologies	30%

 

Behavioural Attribute

·        Assertive

·        Timeous

·        Decisiveness

·        Team player

·        Innovation