Lead Data Platform Engineer (Palantir Or Databricks)

pbProject description /b /ppbr/ppRole Description - We are seeking an expert with deep proficiency as a Palantir Platform Engineer, possessing experience in data engineering and designing, building, and operationalizing AI-powered workflows, agents, and applications that drive tangible business outcomes.The ideal candidate is a self-starter, able to translate complex business needs into scalable technical solutions, and confident working directly with stakeholders to maximize the value of Foundry and AIP./ppbr/ppbResponsibilities /b /ppManage and optimize Palantir data platform./ppEnsure high availability, security, and performance of data systems./ppProvide valuable insights about data platforms usage./ppOptimize computing and storage for large-scale data processing./ppDesign and maintain system libraries (Python) used in ETL pipelines and platform governance./ppOptimize ETL Processes - Enhance and tune existing ETL processes for better performance, scalability, and reliability./ppAIP AI Enablement: /ppSupport the design and deployment of AIP use cases such as copilots, retrieval workflows, and decision-support agents./ppGround agents and logic flows using RAG (retrieval-augmented generation) by connecting to relevant data sources, embedding/vector search, ontology content./ppUse Ontology-Augmented Generation (OAG) when needed: operational decision-making where logic, data, actions and relationships are embedded in the Ontology./ppCollaborate with senior engineers on agent design, instructions, and evaluation using AIP's native features./ppbr/ppbSkills /b /ppbr/ppMust have /ppMinimum 10 Years of experience in IT/Data./ppMinimum 5 years of experience as a Data Platform Engineer/Data Engineer./ppMinimum 3 years of experience with Palantir Foundry./ppPractical experience using or supporting AIP features such as RAG workflows, copilots, or agent-based applications./ppBachelor's in IT or related field./ppInfrastructure Cloud: Azure, AWS (expertise in storage, networking, compute)./ppProficiency in PySpark for distributed computing /ppProficiency in Python for ETL development./ppSQL: Expertise in writing and optimizing SQL queries, preferably with experience in databases such as PostgreSQL, MySQL, Oracle, or Snowflake./ppETL Tools: Familiarity with ETL tools processes /ppData Modelling: Experience with dimensional modelling, normalization/denormalization, and schema design./ppVersion Control: Proficiency with version control tools like Git to manage codebases and collaborate on development./ppData Pipeline Monitoring: Familiarity with monitoring tools (e.g., Prometheus, Grafana, or custom monitoring scripts) to track pipeline performance./ppData Quality Tools: Experience implementing data validation, cleaning, and quality frameworks, ideally Monte Carlo./ppbr/ppNice to have /ppContainerization Orchestration: Docker, Kubernetes./ppInfrastructure as Code (IaC): Terraform./ppUnderstanding of Investment Data domain (desired)./p

ppbr/ppbSkills /b /ppbr/ppMust have /ppMinimum 10 Years of experience in IT/Data. /ppMinimum 5 years of experience as a Data Platform Engineer/Data Engineer. /ppMinimum 3 years of experience with Palantir Foundry. /ppPractical experience using or supporting AIP features such as RAG workflows, copilots, or agent-based applications. /ppBachelor's in IT or related field. /ppInfrastructure Cloud: Azure, AWS (expertise in storage, networking, compute). /ppProficiency in PySpark for distributed computing /ppProficiency in Python for ETL development. /ppSQL: Expertise in writing and optimizing SQL queries, preferably with experience in databases such as PostgreSQL, MySQL, Oracle, or Snowflake. /ppETL Tools: Familiarity with ETL tools processes /ppData Modelling: Experience with dimensional modelling, normalization/denormalization, and schema design. /ppVersion Control: Proficiency with version control tools like Git to manage codebases and collaborate on development. /ppData Pipeline Monitoring: Familiarity with monitoring tools (e.g., Prometheus, Grafana, or custom monitoring scripts) to track pipeline performance. /ppData Quality Tools: Experience implementing data validation, cleaning, and quality frameworks, ideally Monte Carlo. /ppbr/ppNice to have /ppContainerization Orchestration: Docker, Kubernetes. /ppInfrastructure as Code (IaC): Terraform. /ppUnderstanding of Investment Data domain (desired).