> Markdown version of [/jobs/ext/2579964-data-engineer](https://www.wearedevelopers.com/jobs/ext/2579964-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** New York Post - **Location:** New York, NY, United States - **Experience:** Experienced - **Salary:** $100,000.0 - $115,000.0 - **Contract:** Permanent contract - **Skills:** Clean Code Principles, Secure Shell (SSH), Application Programming Interfaces (APIs), Agile Methodology, Artificial Intelligence, Airflow, Amazon Web Services, Amazon S3, JIRA, Unit Testing, Batch Processing, Big Data, BigQuery, Command-Line Interface, Cloud Computing, Code Review, Continuous Integration, Data as a Services, Extract Transform Load (ETL), Data Warehousing, Amazon DynamoDB, Data Flow Control, Internet Services, Python (Programming Language), Linux System Administration, Machine Learning, Recommender Systems, E2e Testing, Shell Script, Software Engineering, SQL Databases, Web Standards, Datadog, Data Logging, Cloud Platform System, Large Language Models, Snowflake, Software Troubleshooting, Generative AI, AWS Lambda, Git, Containerization, Pyspark, Kubernetes, Information Technology, Google Cloud Functions, Graphql, Video Streaming, Cloudwatch, Splunk, Data Pipelines, Docker, Amazon Redshift - **Published:** August 3, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=a76e43ef1f5bbfd7 ## About the Role Must-Haves * MS or BS in computer science, or related field. * 2+ years of professional software engineering experience * Proficiency in Python, SQL, and PySpark * Working knowledge of common AWS and/or GCP services * Experience building and deploying data pipelines (e.g., Airflow, Dataflow, Glue, or similar tools) * Knowledge of data warehouse technologies (BigQuery, Snowflake, Redshift, etc.) * Firm grasp of the tools of the trade including AI tooling, IDEs, Git, JIRA, and agile methodologies. * Familiarity with web standards and protocols. * Comfort with code reviews, writing unit tests, QA and UAT processes, and exposure to end-to-end testing frameworks. * Strong problem-solving skills, attention to detail, thoroughness, and an eagerness to learn. * Proactive approach to problem-solving. * Highly organized with the ability to manage multiple priorities in a fast-paced environment. Great-to-Have Experience * Familiarity with content-management systems, media, news, publishing, video technology, ad technology, or other high-traffic consumer-facing industries. * Exposure to personalization and recommendation systems. * Comfortable with the CLI, Linux administration, shell scripting, ssh, crontabs, etc. * Hands-on experience with generative-AI models, LLM prompting, and/or machine learning * Experience with monitoring, logging, dashboard, and alarming tools (e.g., CloudWatch, Datadog, Splunk, etc.) * Familiarity with REST and GraphQL APIs, authentication/authorization, and API best practices. * Scaling techniques in cloud environments. * Troubleshooting and debugging experience with internet services. * Understanding of CI/CD workflows and containerization (Docker, Kubernetes a plus) Note: The New York Post follows a hybrid work model. This position would be expected to be in the office a minimum of 3 days per week (subject to change). ## Description You will build and maintain the foundational data services that power analytics and machine learning for our newsroom, and product teams to deliver cutting-edge thoughtful digital experiences at scale. You'll work alongside senior engineers, product managers, and data teams, contributing to high-performance systems., * Design, build, and operate cloud-based data pipelines supporting specialized needs of our consumer websites, native mobile apps, and information services. * Work daily with compute services, query services, storage services, and big data services with technologies like Vertex AI, Lambdas, DynamoDB, Cloud Functions, S3, Kubernetes, Glue, and BigQuery inside AWS and GCP. * Develop and maintain data ETL pipelines for real-time and batch processing of large-scale content and event data. * Write clean, testable, and maintainable code (Python preferred). * Collaborate with product, editorial, and data science teams to deliver infrastructure and services that meet business needs. * Monitor, optimize, and improve the reliability, security, and cost efficiency of cloud-based systems. * Write and present ideas clearly. Document functional requirements for both technical and non-technical audiences. * Participate in code reviews, architecture discussions, and agile sprint ceremonies. ## Related Videos - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Improving quality with Agentic AI with Rovo Dev and Xray](https://www.wearedevelopers.com/videos/2005-improving-quality-with-agentic-ai-with-rovo-dev-and-xray) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) ## Related Articles - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers) - [Dev Digest 168: Hacking Postgres, Blocking Meta and Fixing CSS](https://www.wearedevelopers.com/magazine/588-dev-digest-168-hacking-postgres-blocking-meta-and-fixing-css) - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Dev Digest 139 - Soft and hard queries](https://www.wearedevelopers.com/magazine/487-dev-digest-139-soft-and-hard-queries) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)