ECMS are looking for skilled Data Engineers to work across a couple of our clients delivering technical and advisory services around complex data transformations. The role will involve understanding our client’s current data and data pipeline architecture and looking at how they can optimise this for cross-functional teams.
Key responsibilities:
Build out complex data transformation pipelines and processes
Advise on best practices around data lakes and related technologies in the cloud
Maintain and support creation of data warehouses in the cloud
Assemble large, complex data sets to meet functional / non-functional requirements
Support in the quality assurance of data to ensure it is fit for purpose
Define data access policies and help police them
Create data tools for analytics and data science teams
Essential technical experience:
Excellent SQL knowledge and experience working with relational databases
Manipulating, processing and extracting data from disparate data sources
Experience building data pipelines, ideally in the cloud
Strong analytical experience
Knowledge of queues and big data stores such as Hadoop
Experience with NoSQL databases and AWS services such as RDS, EC2 and Redshift
Experience with stream processing services such as Apache Spark etc.
Experience with programming languages such as Python or Java and ETL tools like Talend
