> Markdown version of [/jobs/ext/3532035-data-engineer-to-architect](https://www.wearedevelopers.com/jobs/ext/3532035-data-engineer-to-architect). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Engineer to architect

- **Company:** Transflo Terminal Services, Inc.
- **Location:** United States (Remote available)
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Query Performance, Application Programming Interfaces (APIs), Airflow, Amazon Web Services, Amazon S3, Apache HTTP Server, Data as a Services, Information Engineering, Data Governance, Data Infrastructure, Data Integrity, Extract Transform Load (ETL), Data Transformation, Data Mining, Data Normalization, Data Sharing, Data Warehousing, IBM DB2, Relational Databases, Dimensional Modeling, Amazon DynamoDB, JSON, Python (Programming Language), PostgreSQL, Metadata Repositories, MySQL, NoSQL, Performance Tuning, Power BI, SQL Stored Procedures, SQL Databases, Data Streaming, Tableau (Software), Workflow Management Systems, Parquet, Data Processing, Scripting, Business Intelligence Development Studio, Data Layers, Data Lakes, Avro, AWS Glue, Star Schema, AWS Data Analytics, Real Time Data, Apache Kafka, Video Streaming, Data Delivery, Restful APIs, Terraform, Stream Processing, Looker Analytics, Data Pipelines, Amazon Redshift
- **Published:** September 30, 2026
- **Apply:** https://www.thejobnetwork.com/job/a4d5f8b1-454d-4ecf-baee-1db6ddc2fcdb/senior-data-engineer

## About the Role

* 5+ years of professional data engineering experience with a track record of building and operating production-grade data warehouses and pipeline infrastructure
\n

 * Expert-level experience with Amazon Redshift including cluster sizing, WLM configuration, distribution and sort key optimization, vacuuming, and query plan analysis
\n

 * Deep proficiency in SQL for complex analytical queries, window functions, CTEs, stored procedures, and performance tuning across Redshift and ANSI-compatible engines
\n

 * Hands-on experience ingesting data from heterogeneous source systems: REST APIs, IBM DB2, MySQL, Amazon Aurora (MySQL and PostgreSQL-compatible), Amazon DynamoDB, PostgreSQL, and file-based sources (CSV, JSON, Parquet, Avro)
\n

 * Proven experience designing and implementing medallion (bronze/silver/gold) or equivalent layered data architectures at enterprise scale
\n

 * Strong working knowledge of star schema and snowflake schema design, dimensional modeling theory, slowly changing dimensions, and fact table granularity decisions
\n

 * Experience building real-time or near real-time data pipelines using streaming technologies such as Amazon Kinesis, Apache Kafka (or Amazon MSK), or equivalent
\n

 * Proficiency with ETL/ELT orchestration tools such as AWS Glue, dbt, Apache Airflow, or AWS Step Functions
\n

 * Demonstrated experience implementing data governance practices: data catalogs (AWS Glue Data Catalog, Apache Atlas, or equivalent), lineage, metadata tagging, and access control frameworks
\n

 * Infrastructure-as-code experience with Terraform for provisioning and managing data infrastructure on AWS
\n

 * Strong Python skills for pipeline development, data transformation logic, and automation scripting
\n

 * Deep understanding of data reliability engineering: idempotency, exactly-once processing, late-arriving data handling, schema evolution, and SLA-driven pipeline design
\n, * Experience in the transportation, logistics, trucking, or fleet management industry, or with high-volume transactional SaaS platforms processing operational telemetry data is a huge plus
\n

 * Experience building DaaS or data product offerings including governed external data APIs, Redshift Data Sharing, or AWS Data Exchange integrations
\n

 * Knowledge of columnar storage formats (Parquet, ORC) and lakehouse patterns using Amazon S3 as a data lake layer in conjunction with Redshift Spectrum or AWS Glue
\n

 * Familiarity with BI and analytics consumption tools such as Tableau, Power BI, Amazon QuickSight, or Looker and how data model design decisions impact end-user query performance, * Experience with data observability platforms such as Monte Carlo, Great Expectations, or dbt tests for automated data quality monitoring

## Description

This role demands both deep technical expertise and a strategic mindset. You will work across a wide range of source systems - APIs, relational databases, NoSQL stores, file-based feeds, and streaming data - normalizing and modeling data into reliable, governed, and high-performance analytical assets. You will build and scale systems designed for near real-time data environments supporting high-traffic, mission-critical workloads in the transportation and logistics industry., * Architect, build, and evolve a scalable enterprise data warehouse on Amazon Redshift, applying industry-standard concepts including star schemas, snowflake schemas, normalization, denormalization, referential integrity, and performance optimization strategies
\n

 * Design and implement bronze, silver, and gold data layer architecture (medallion architecture): raw ingestion, cleansed and standardized intermediate layers, and curated, business-ready data products optimized for analytics consumption
\n

 * Develop dimensional data models, fact and dimension tables, slowly changing dimensions (SCDs), and aggregate structures that support BI tooling, ad-hoc analytics, and downstream API consumption
\n

 * Apply rigorous data modeling practices including schema design, constraint definition, indexing strategy, sort keys, distribution keys, and query plan optimization within Redshift and connected systems
\n

 * Build, own, and maintain robust batch and streaming data pipelines that ingest data from disparate source systems including REST APIs, flat files, IBM DB2, MySQL, Amazon Aurora, Amazon DynamoDB, and PostgreSQL
\n

 * Implement real-time and near real-time data streaming architectures using AWS-native services such as Kinesis Data Streams, Kinesis Firehose, MSK (Managed Kafka), and EventBridge to support low-latency data delivery requirements


## Related Videos

- [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis)
- [Tips and Tricks for Working with JSON](https://www.wearedevelopers.com/videos/1229-tips-and-tricks-for-working-with-json)
- [From event streaming to event sourcing 101](https://www.wearedevelopers.com/videos/91-from-event-streaming-to-event-sourcing-101)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Introducing JSON Structure](https://www.wearedevelopers.com/videos/100219-introducing-json-structure)
- [Tomorrow's cloud data platforms - fully managed database-as-a-service (DBaaS)](https://www.wearedevelopers.com/videos/254-tomorrow-s-cloud-data-platforms-fully-managed-database-as-a-service-dbaas)

## Related Articles

- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)