Data Engineer - Sales Data - Azure Databricks

We have an exciting opportunity now available with one of our sector-leading data analytics clients! They are currently looking for a skilled Data Engineer to join their team for a four-month contract., We are seeking an experienced data analyst to support a critical business project which uses sales and product data to assess year on year data subscriptions growth by agreed business definitions of uplift, upsell, cross sell and cancellation. The work required involves mapping product sets to ensure retired products with superseded replacements are categorised correctly along with ensuring the base data set correctly captures sales for the previous year and the current year. We work in an Azure/Databricks environment so experience in this technical set up is essential along with a good ability to understand the sales data architecture (from Salesforce).

The ideal candidate will have the following:

• Proficiency in programming languages such as Python and SQL is essential. Python, in particular, has emerged as the language for Data and AI applications commonly used for data manipulation, building data pipelines, and developing algorithms for data processing.
• A strong understanding of data formats such as Delta and Iceberg is vital, in addition to both SQL and NoSQL databases. Data engineers should be skilled in designing, querying, and optimising data to ensure efficient data storage and retrieval. Familiarity with database technologies like PostgreSQL, MongoDB, CosmosDB, and SQL Server is also beneficial.
• Knowledge of data warehousing technologies such as Databricks SQL, Snowflake Warehouses, Amazon Redshift, and Google BigQuery is important for structuring and managing large volumes of data.
• Data engineers should be adept at Extract, Transform, Load (ETL) processes. This includes using frameworks such as the Medallion Architecture, as well as tools like Apache Airflow, Fivetran, Azure Data Factory, and Amazon Glue to automate data workflows and ensure data quality.
• Familiarity with cloud computing services, such as AWS, Azure, or Google Cloud Platform, is increasingly important. Data engineers often leverage these platforms for scalable data storage and processing capabilities.
• Knowledge of big data frameworks like Apache Spark, Beam is crucial for handling large datasets and Real Time data processing.
• Understanding data modelling techniques helps data engineers design efficient data structures that meet the analytical needs of the organisation.
• Proficiency in using version control systems like Git is important for collaboration and maintaining code integrity.
• Awareness of data security practices and compliance regulations (such as GDPR) is essential to protect sensitive information and ensure ethical data handling. In addition, knowledge of data access methods, and attribute-based access control (ABAC) is vital to securing enterprise data.

While not always required, having a foundational understanding of machine learning concepts can help data engineers collaborate effectively with data scientists and contribute to the development of predictive models.

Square One is acting as both an employment agency and an employment business, and is an equal opportunities recruitment business. Square One embraces diversity and will treat everyone equally. Please see our website for our full diversity statement.