Big Data Architect IT Consultant Master

Our client seeks an experienced IT Consultant to support the design, development, implementation and maintenance of an enterprise Big Data solution as part of a Data Modernization Effort.

This role will provide expertise to support the development of a Big Data / Data Lake system architecture that supports enterprise data operations for a government organization, including Internet of Things (IoT) / Smart City projects, the enterprise data warehouse, the open data portal, and data science applications. This is an exciting opportunity to work as part of a collaborative senior data team supporting the Chief Data Officer. This architecture includes Databricks, Microsoft Azure platform tools (including Data Lake, Synapse) and data pipeline/ETL development tools (including StreamSets, Azure Data Factory). The platform will be designed for enterprise-wide use and integration with other Enterprise Data tools such as Esri, Tableau, MicroStrategy, API Gateways, and Oracle databases and integration tools.

Responsibilities

• Coordinates IT project management, engineering, maintenance, QA, and risk management.
• Plans, coordinates, and monitors project activities.
• Develops technical applications to support users.
• Develops, implements, maintains and enforces documented standards and procedures for the design, development, installation, modification, and documentation of assigned systems.
• Provides training for system products and procedures.
• Performs application upgrades.
• Performs monitoring, maintenance, or reporting on real-time databases, real-time network and serial data communications, and real-time graphics and logic applications.
• Troubleshoots problems.
• Ensures project life-cycle is in compliance with established standards and procedures.

• Bachelor’s degree in Information Technology or related field or equivalent experience, * Experience implementing modern Big Data storage and analytics platforms such as Databricks and Data Lakes (5 years)
• Knowledge of modern Big Data and Data Architecture and Implementation best practices (5 years)
• Knowledge of architecture and implementation of networking, security and storage on cloud platforms such as Microsoft Azure (5 years)
• Experience with operations (runtime, warehousing, pipelines, auditing, sharing, cost management, user management) for data platforms such as Databricks (5 years)
• Experience with deployment of data tools and storage on cloud platforms such as Microsoft Azure (5 years)
• Knowledge of data-centric systems for the analysis and visualization of data, such as Tableau, MicroStrategy, ArcGIS, Kibana, Oracle (5 years)
• Experience querying structured and unstructured data sources including SQL and NoSQL databases (5 years)
• Experience modeling and ingesting data into and between various data systems through the use of Data Pipelines (5 years)
• Experience with API / Web Services (REST/SOAP) (3 years)
• Experience with complex event processing and real-time streaming data (3 years)
• Experience with deployment and management of data science tools and modules such as Notebooks and R (3 years)
• Experience with ETL, data processing, analytics using languages such as Python, Java or R (3 years)
• Planning, coordinating, and monitoring project activities (16+ years)
• Leading projects, ensuring they are in compliance with established standards/procedures (16+ years), * Databricks Certified Data Engineer Professional (highly desired)
• Experience deploying apps, gateways, agents and models on data platforms such as Databricks (nice to have, 2 years)